nltk.corpus.reader.EuroparlCorpusReader

class documentation

class EuroparlCorpusReader(PlaintextCorpusReader): (source)

Constructor: EuroparlCorpusReader(root, fileids, word_tokenizer, sent_tokenizer, ...)

View In Hierarchy

Reader for Europarl corpora that consist of plaintext documents. Documents are divided into chapters instead of paragraphs as for regular plaintext documents. Chapters are separated using blank lines. Everything is inherited from PlaintextCorpusReader except that:

  • Since the corpus is pre-processed and pre-tokenized, the word tokenizer should just split the line at whitespaces.
  • For the same reason, the sentence tokenizer should just split the paragraph at line breaks.
  • There is a new 'chapters()' method that returns chapters instead instead of paragraphs.
  • The 'paras()' method inherited from PlaintextCorpusReader is made non-functional to remove any confusion between chapters and paragraphs for Europarl.
Method chapters No summary
Method paras No summary
Method _read_para_block Undocumented
Method _read_sent_block Undocumented
Method _read_word_block Undocumented

Inherited from PlaintextCorpusReader:

Method __init__ Construct a new plaintext corpus reader for a set of documents located at the given root directory. Example usage:
Method raw No summary
Method sents No summary
Method words No summary
Instance Variable _para_block_reader Undocumented
Instance Variable _sent_tokenizer Undocumented
Instance Variable _word_tokenizer Undocumented

Inherited from CorpusReader (via PlaintextCorpusReader):

Method __repr__ Undocumented
Method abspath Return the absolute path for the given file.
Method abspaths Return a list of the absolute paths for all fileids in this corpus; or for the given list of fileids, if specified.
Method citation Return the contents of the corpus citation.bib file, if it exists.
Method encoding Return the unicode encoding for the given corpus file, if known. If the encoding is unknown, or if the given file should be processed using byte strings (str), then return None.
Method ensure_loaded Load this corpus (if it has not already been loaded). This is used by LazyCorpusLoader as a simple method that can be used to make sure a corpus is loaded -- e.g., in case a user wants to do help(some_corpus).
Method fileids Return a list of file identifiers for the fileids that make up this corpus.
Method license Return the contents of the corpus LICENSE file, if it exists.
Method open Return an open stream that can be used to read the given file. If the file's encoding is not None, then the stream will automatically decode the file's contents into unicode.
Method readme Return the contents of the corpus README file, if it exists.
Class Variable root Undocumented
Method _get_root Undocumented
Instance Variable _encoding The default unicode encoding for the fileids that make up this corpus. If encoding is None, then the file contents are processed using byte strings.
Instance Variable _fileids A list of the relative paths for the fileids that make up this corpus.
Instance Variable _root The root directory for this corpus.
Instance Variable _tagset Undocumented
def chapters(self, fileids=None): (source)
Returns
list(list(list(str)))the given file(s) as a list of chapters, each encoded as a list of sentences, which are in turn encoded as lists of word strings.
def paras(self, fileids=None): (source)
overrides nltk.corpus.reader.PlaintextCorpusReader.paras
Returns
list(list(list(str)))the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word strings.
def _read_para_block(self, stream): (source)
overrides nltk.corpus.reader.PlaintextCorpusReader._read_para_block

Undocumented

def _read_sent_block(self, stream): (source)
overrides nltk.corpus.reader.PlaintextCorpusReader._read_sent_block

Undocumented

def _read_word_block(self, stream): (source)
overrides nltk.corpus.reader.PlaintextCorpusReader._read_word_block

Undocumented