class documentation
Implements Laplace (add one) smoothing.
Initialization identical to BaseNgramModel because gamma is always 1.
Method | __init__ |
Creates new LanguageModel. |
Inherited from Lidstone
:
Method | unmasked |
Add-one smoothing: Lidstone or Laplace. |
Instance Variable | gamma |
Undocumented |
Inherited from LanguageModel
(via Lidstone
):
Method | context |
Helper method for retrieving counts for a given context. |
Method | entropy |
Calculate cross-entropy of model for given evaluation text. |
Method | fit |
Trains the model on a text. |
Method | generate |
Generate words from the model. |
Method | logscore |
Evaluate the log score of this word in this context. |
Method | perplexity |
Calculates the perplexity of the given text. |
Method | score |
Masks out of vocab (OOV) words and computes their model score. |
Instance Variable | counts |
Undocumented |
Instance Variable | order |
Undocumented |
Instance Variable | vocab |
Undocumented |
overrides
nltk.lm.Lidstone.__init__
Creates new LanguageModel.
of creating a new one when training.
:type vocabulary: nltk.lm.Vocabulary
or None
:param counter: If provided, use this object to count ngrams.
:type vocabulary: nltk.lm.NgramCounter
or None
:param ngrams_fn: If given, defines how sentences in training text are turned to ngram
sequences.
Parameters | |
*args | Undocumented |
ngrams | Undocumented |
pad | If given, defines how senteces in training text are padded. |
vocabulary | If provided, this vocabulary will be used instead |
**kwargs | Undocumented |