class documentation
Implements Laplace (add one) smoothing.
Initialization identical to BaseNgramModel because gamma is always 1.
| Method | __init__ |
Creates new LanguageModel. |
Inherited from Lidstone:
| Method | unmasked |
Add-one smoothing: Lidstone or Laplace. |
| Instance Variable | gamma |
Undocumented |
Inherited from LanguageModel (via Lidstone):
| Method | context |
Helper method for retrieving counts for a given context. |
| Method | entropy |
Calculate cross-entropy of model for given evaluation text. |
| Method | fit |
Trains the model on a text. |
| Method | generate |
Generate words from the model. |
| Method | logscore |
Evaluate the log score of this word in this context. |
| Method | perplexity |
Calculates the perplexity of the given text. |
| Method | score |
Masks out of vocab (OOV) words and computes their model score. |
| Instance Variable | counts |
Undocumented |
| Instance Variable | order |
Undocumented |
| Instance Variable | vocab |
Undocumented |
overrides
nltk.lm.Lidstone.__init__Creates new LanguageModel.
of creating a new one when training.
:type vocabulary: nltk.lm.Vocabulary or None
:param counter: If provided, use this object to count ngrams.
:type vocabulary: nltk.lm.NgramCounter or None
:param ngrams_fn: If given, defines how sentences in training text are turned to ngram
sequences.
| Parameters | |
| *args | Undocumented |
| ngrams | Undocumented |
| pad | If given, defines how senteces in training text are padded. |
| vocabulary | If provided, this vocabulary will be used instead |
| **kwargs | Undocumented |