nltk.chunk.util.ChunkScore

class documentation

class ChunkScore(object): (source)

Constructor: ChunkScore(**kwargs)

View In Hierarchy

A utility class for scoring chunk parsers. ChunkScore can evaluate a chunk parser's output, based on a number of statistics (precision, recall, f-measure, misssed chunks, incorrect chunks). It can also combine the scores from the parsing of multiple texts; this makes it significantly easier to evaluate a chunk parser that operates one sentence at a time.

Texts are evaluated with the score method. The results of evaluation can be accessed via a number of accessor methods, such as precision and f_measure. A typical use of the ChunkScore class is:

>>> chunkscore = ChunkScore()           # doctest: +SKIP
>>> for correct in correct_sentences:   # doctest: +SKIP
...     guess = chunkparser.parse(correct.leaves())   # doctest: +SKIP
...     chunkscore.score(correct, guess)              # doctest: +SKIP
>>> print('F Measure:', chunkscore.f_measure())       # doctest: +SKIP
F Measure: 0.823
Method __init__ Undocumented
Method __len__ Undocumented
Method __repr__ Return a concise representation of this ChunkScoring.
Method __str__ Return a verbose representation of this ChunkScoring. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., missed()...
Method accuracy Return the overall tag-based accuracy for all text that have been scored by this ChunkScore, using the IOB (conll2000) tag encoding.
Method correct Return the chunks which were included in the correct chunk structures, listed in input order.
Method f_measure Return the overall F measure for all texts that have been scored by this ChunkScore.
Method guessed Return the chunks which were included in the guessed chunk structures, listed in input order.
Method incorrect Return the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.
Method missed Return the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.
Method precision Return the overall precision for all texts that have been scored by this ChunkScore.
Method recall Return the overall recall for all texts that have been scored by this ChunkScore.
Method score Given a correctly chunked sentence, score another chunked version of the same sentence.
Instance Variable kwargs Keyword arguments:
Method _updateMeasures Undocumented
Instance Variable _chunk_label Undocumented
Instance Variable _correct Undocumented
Instance Variable _count Undocumented
Instance Variable _fn List of false negatives
Instance Variable _fn_num Number of false negatives.
Instance Variable _fp List of false positives
Instance Variable _fp_num Number of false positives
Instance Variable _guessed Undocumented
Instance Variable _max_fn Undocumented
Instance Variable _max_fp Undocumented
Instance Variable _max_tp Undocumented
Instance Variable _measuresNeedUpdate Undocumented
Instance Variable _tags_correct Undocumented
Instance Variable _tags_total Undocumented
Instance Variable _tp List of true positives
Instance Variable _tp_num Number of true positives
def __init__(self, **kwargs): (source)

Undocumented

def __len__(self): (source)

Undocumented

def __repr__(self): (source)

Return a concise representation of this ChunkScoring.

Returns
strUndocumented
def __str__(self): (source)

Return a verbose representation of this ChunkScoring. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., missed() and incorrect()).

Returns
strUndocumented
def accuracy(self): (source)

Return the overall tag-based accuracy for all text that have been scored by this ChunkScore, using the IOB (conll2000) tag encoding.

Returns
floatUndocumented
def correct(self): (source)

Return the chunks which were included in the correct chunk structures, listed in input order.

Returns
list of chunksUndocumented
def f_measure(self, alpha=0.5): (source)

Return the overall F measure for all texts that have been scored by this ChunkScore.

Parameters
alpha:floatthe relative weighting of precision and recall. Larger alpha biases the score towards the precision value, while smaller alpha biases the score towards the recall value. alpha should have a value in the range [0,1].
Returns
floatUndocumented
def guessed(self): (source)

Return the chunks which were included in the guessed chunk structures, listed in input order.

Returns
list of chunksUndocumented
def incorrect(self): (source)

Return the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.

Returns
list of chunksUndocumented
def missed(self): (source)

Return the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.

Returns
list of chunksUndocumented
def precision(self): (source)

Return the overall precision for all texts that have been scored by this ChunkScore.

Returns
floatUndocumented
def recall(self): (source)

Return the overall recall for all texts that have been scored by this ChunkScore.

Returns
floatUndocumented
def score(self, correct, guessed): (source)

Given a correctly chunked sentence, score another chunked version of the same sentence.

Parameters
correct:chunk structureThe known-correct ("gold standard") chunked sentence.
guessed:chunk structureThe chunked sentence to be scored.
kwargs = (source)

Keyword arguments:

  • max_tp_examples: The maximum number actual examples of true positives to record. This affects the correct member function: correct will not return more than this number of true positive examples. This does not affect any of the numerical metrics (precision, recall, or f-measure)
  • max_fp_examples: The maximum number actual examples of false positives to record. This affects the incorrect member function and the guessed member function: incorrect will not return more than this number of examples, and guessed will not return more than this number of true positive examples. This does not affect any of the numerical metrics (precision, recall, or f-measure)
  • max_fn_examples: The maximum number actual examples of false negatives to record. This affects the missed member function and the correct member function: missed will not return more than this number of examples, and correct will not return more than this number of true negative examples. This does not affect any of the numerical metrics (precision, recall, or f-measure)
  • chunk_label: A regular expression indicating which chunks should be compared. Defaults to '.*' (i.e., all chunks).
def _updateMeasures(self): (source)

Undocumented

_chunk_label = (source)

Undocumented

_correct = (source)

Undocumented

_count: int = (source)

Undocumented

_fn: list(Token) = (source)

List of false negatives

_fn_num: int = (source)

Number of false negatives.

_fp: list(Token) = (source)

List of false positives

_fp_num: int = (source)

Number of false positives

_guessed = (source)

Undocumented

_max_fn = (source)

Undocumented

_max_fp = (source)

Undocumented

_max_tp = (source)

Undocumented

_measuresNeedUpdate: bool = (source)

Undocumented

_tags_correct: float = (source)

Undocumented

_tags_total: float = (source)

Undocumented

_tp: list(Token) = (source)

List of true positives

_tp_num: int = (source)

Number of true positives