nltk.translate.ibm1.IBMModel1

class documentation

class IBMModel1(IBMModel): (source)

Constructor: IBMModel1(sentence_aligned_corpus, iterations, probability_tables)

Lexical translation model that ignores word order

>>> bitext = []
>>> bitext.append(AlignedSent(['klein', 'ist', 'das', 'haus'], ['the', 'house', 'is', 'small']))
>>> bitext.append(AlignedSent(['das', 'haus', 'ist', 'ja', 'groß'], ['the', 'house', 'is', 'big']))
>>> bitext.append(AlignedSent(['das', 'buch', 'ist', 'ja', 'klein'], ['the', 'book', 'is', 'small']))
>>> bitext.append(AlignedSent(['das', 'haus'], ['the', 'house']))
>>> bitext.append(AlignedSent(['das', 'buch'], ['the', 'book']))
>>> bitext.append(AlignedSent(['ein', 'buch'], ['a', 'book']))

>>> ibm1 = IBMModel1(bitext, 5)

>>> print(ibm1.translation_table['buch']['book'])
0.889...
>>> print(ibm1.translation_table['das']['book'])
0.061...
>>> print(ibm1.translation_table['buch'][None])
0.113...
>>> print(ibm1.translation_table['ja'][None])
0.072...

>>> test_sentence = bitext[2]
>>> test_sentence.words
['das', 'buch', 'ist', 'ja', 'klein']
>>> test_sentence.mots
['the', 'book', 'is', 'small']
>>> test_sentence.alignment
Alignment([(0, 0), (1, 1), (2, 2), (3, 2), (4, 3)])

Method	`__init__`	Train on `sentence_aligned_corpus` and create a lexical translation model.
Method	`align`	Determines the best word alignment for one sentence pair from the corpus that the model was trained on.
Method	`align_all`	Undocumented
Method	`prob_alignment_point`	Probability that word `t` in the target sentence is aligned to word `s` in the source sentence
Method	`prob_all_alignments`	Computes the probability of all possible word alignments, expressed as a marginal distribution over target words t
Method	`prob_t_a_given_s`	Probability of target sentence and an alignment given the source sentence
Method	`set_uniform_probabilities`	Undocumented
Method	`train`	Undocumented
Instance Variable	`translation_table`	Undocumented

def __init__(self, sentence_aligned_corpus, iterations, probability_tables=None): (source) ¶

Train on sentence_aligned_corpus and create a lexical translation model.

Translation direction is from AlignedSent.mots to AlignedSent.words.

Parameters
sentence_aligned_corpus:list(AlignedSent)	Sentence-aligned parallel corpus
iterations:int	Number of iterations to run training algorithm
probability_tables:dict[str]: object	Optional. Use this to pass in custom probability values. If not specified, probabilities will be set to a uniform distribution, or some other sensible value. If specified, the following entry must be present: `translation_table`. See `IBMModel` for the type and purpose of this table.

def align(self, sentence_pair): (source) ¶

Determines the best word alignment for one sentence pair from the corpus that the model was trained on.

The best alignment will be set in sentence_pair when the method returns. In contrast with the internal implementation of IBM models, the word indices in the Alignment are zero- indexed, not one-indexed.

Parameters
sentence_pair:AlignedSent	A sentence in the source language and its counterpart sentence in the target language

def align_all(self, parallel_corpus): (source) ¶

Undocumented

def prob_alignment_point(self, s, t): (source) ¶

Probability that word t in the target sentence is aligned to word s in the source sentence

def prob_all_alignments(self, src_sentence, trg_sentence): (source) ¶

Computes the probability of all possible word alignments, expressed as a marginal distribution over target words t

Each entry in the return value represents the contribution to the total alignment probability by the target word t.

To obtain probability(alignment | src_sentence, trg_sentence), simply sum the entries in the return value.

Returns
dict(str): float	Probability of t for all s in `src_sentence`

def prob_t_a_given_s(self, alignment_info): (source) ¶

Probability of target sentence and an alignment given the source sentence

def set_uniform_probabilities(self, sentence_aligned_corpus): (source) ¶

Undocumented

def train(self, parallel_corpus): (source) ¶

Undocumented

translation_table = (source) ¶

Undocumented