class documentation

Implements Laplace (add one) smoothing.

Initialization identical to BaseNgramModel because gamma is always 1.

Method __init__ Creates new LanguageModel.

Inherited from Lidstone:

Method unmasked_score Add-one smoothing: Lidstone or Laplace.
Instance Variable gamma Undocumented

Inherited from LanguageModel (via Lidstone):

Method context_counts Helper method for retrieving counts for a given context.
Method entropy Calculate cross-entropy of model for given evaluation text.
Method fit Trains the model on a text.
Method generate Generate words from the model.
Method logscore Evaluate the log score of this word in this context.
Method perplexity Calculates the perplexity of the given text.
Method score Masks out of vocab (OOV) words and computes their model score.
Instance Variable counts Undocumented
Instance Variable order Undocumented
Instance Variable vocab Undocumented
def __init__(self, *args, **kwargs): (source) ΒΆ

Creates new LanguageModel.

of creating a new one when training. :type vocabulary: nltk.lm.Vocabulary or None :param counter: If provided, use this object to count ngrams. :type vocabulary: nltk.lm.NgramCounter or None :param ngrams_fn: If given, defines how sentences in training text are turned to ngram

sequences.
Parameters
*argsUndocumented
ngrams_fn:function or NoneUndocumented
pad_fn:function or NoneIf given, defines how senteces in training text are padded.
vocabularyIf provided, this vocabulary will be used instead
**kwargsUndocumented