nltk.corpus.reader.KNBCorpusReader

class documentation

class KNBCorpusReader(SyntaxCorpusReader): (source)

Constructor: KNBCorpusReader(root, fileids, encoding, morphs2str)

View In Hierarchy

This class implements:
  • __init__, which specifies the location of the corpus and a method for detecting the sentence blocks in corpus files.
  • _read_block, which reads a block from the input stream.
  • _word, which takes a block and returns a list of list of words.
  • _tag, which takes a block and returns a list of list of tagged words.
  • _parse, which takes a block and returns a list of parsed sentences.
The structure of tagged words:
tagged_word = (word(str), tags(tuple)) tags = (surface, reading, lemma, pos1, posid1, pos2, posid2, pos3, posid3, others ...)

Usage example

>>> from nltk.corpus.util import LazyCorpusLoader
>>> knbc = LazyCorpusLoader(
...     'knbc/corpus1',
...     KNBCorpusReader,
...     r'.*/KN.*',
...     encoding='euc-jp',
... )
>>> len(knbc.sents()[0])
9

Method __init__ Initialize KNBCorpusReader morphs2str is a function to convert morphlist to str for tree representation for _parse()
Instance Variable morphs2str Undocumented
Method _parse Undocumented
Method _read_block Undocumented
Method _tag Undocumented
Method _word Undocumented

Inherited from SyntaxCorpusReader:

Method parsed_sents Undocumented
Method raw Undocumented
Method sents Undocumented
Method tagged_sents Undocumented
Method tagged_words Undocumented
Method words Undocumented
Method _read_parsed_sent_block Undocumented
Method _read_sent_block Undocumented
Method _read_tagged_sent_block Undocumented
Method _read_tagged_word_block Undocumented
Method _read_word_block Undocumented

Inherited from CorpusReader (via SyntaxCorpusReader):

Method __repr__ Undocumented
Method abspath Return the absolute path for the given file.
Method abspaths Return a list of the absolute paths for all fileids in this corpus; or for the given list of fileids, if specified.
Method citation Return the contents of the corpus citation.bib file, if it exists.
Method encoding Return the unicode encoding for the given corpus file, if known. If the encoding is unknown, or if the given file should be processed using byte strings (str), then return None.
Method ensure_loaded Load this corpus (if it has not already been loaded). This is used by LazyCorpusLoader as a simple method that can be used to make sure a corpus is loaded -- e.g., in case a user wants to do help(some_corpus).
Method fileids Return a list of file identifiers for the fileids that make up this corpus.
Method license Return the contents of the corpus LICENSE file, if it exists.
Method open Return an open stream that can be used to read the given file. If the file's encoding is not None, then the stream will automatically decode the file's contents into unicode.
Method readme Return the contents of the corpus README file, if it exists.
Class Variable root Undocumented
Method _get_root Undocumented
Instance Variable _encoding The default unicode encoding for the fileids that make up this corpus. If encoding is None, then the file contents are processed using byte strings.
Instance Variable _fileids A list of the relative paths for the fileids that make up this corpus.
Instance Variable _root The root directory for this corpus.
Instance Variable _tagset Undocumented
def __init__(self, root, fileids, encoding='utf8', morphs2str=_morphs2str_default): (source)
overrides nltk.corpus.reader.CorpusReader.__init__

Initialize KNBCorpusReader morphs2str is a function to convert morphlist to str for tree representation for _parse()

morphs2str = (source)

Undocumented

def _parse(self, t): (source)
overrides nltk.corpus.reader.SyntaxCorpusReader._parse

Undocumented

def _read_block(self, stream): (source)
overrides nltk.corpus.reader.SyntaxCorpusReader._read_block

Undocumented

def _tag(self, t, tagset=None): (source)
overrides nltk.corpus.reader.SyntaxCorpusReader._tag

Undocumented

def _word(self, t): (source)
overrides nltk.corpus.reader.SyntaxCorpusReader._word

Undocumented