Uses of Class
opennlp.tools.tokenize.TokenizerFactory
Packages that use TokenizerFactory
Package
Description
Contains classes related to finding token or words in a string.
-
Uses of TokenizerFactory in opennlp.tools.tokenize
Methods in opennlp.tools.tokenize that return TokenizerFactoryModifier and TypeMethodDescriptionstatic TokenizerFactory
TokenizerFactory.create
(String subclassName, String languageCode, Dictionary abbreviationDictionary, boolean useAlphaNumericOptimization, Pattern alphaNumericPattern) Factory method the framework uses instantiate a newTokenizerFactory
.TokenizerModel.getFactory()
Methods in opennlp.tools.tokenize with parameters of type TokenizerFactoryModifier and TypeMethodDescriptionstatic TokenizerModel
TokenizerME.train
(ObjectStream<TokenSample> samples, TokenizerFactory factory, TrainingParameters mlParams) Trains a model for theTokenizerME
.Constructors in opennlp.tools.tokenize with parameters of type TokenizerFactoryModifierConstructorDescriptionTokenizerCrossValidator
(TrainingParameters params, TokenizerFactory factory, TokenizerEvaluationMonitor... listeners) Creates aTokenizerCrossValidator
using the givenTokenizerFactory
.TokenizerModel
(MaxentModel tokenizerModel, Map<String, String> manifestInfoEntries, TokenizerFactory tokenizerFactory) Initializes aTokenizerModel
instance via aMaxentModel
and related resources.