edu.stanford.nlp.ie.hmm
Class AnswerChecker

java.lang.Object
  |
  +--edu.stanford.nlp.ie.hmm.AnswerChecker

public class AnswerChecker

extends Object

Utility class for checking whether words pulled from the HMM match the errors created from an AnswerConstructor.

Nested Class Summary

static class

AnswerChecker.Range Reprsents a range [from,to) (same semantics as substring).

Constructor Summary

AnswerChecker(TypedTaggedDocument source)
Constructs a new AnswerChecker to check answers for the given TypedTaggedDocument.

Method Summary

HashMap	checkAnswers(int[] guesses, boolean strict) Returns a map from Integer (state type) -> PrecisionRecallStats for that state type.
HashMap	checkBestAnswers(Map bestGuessWordSequenceByType) Takes a Map from Integer (type) -> List of Strings for best guess word sequence and returns a Map from type to PrecisionRecallStats for that state type.
static HashMap	getAnswerRanges(int[] answers) Takes the given list of answer types and converts them into a map from type to a list of Ranges (start/end) indices.
void	printAnswers() Prints answer word sequences for the correct answers for this document.
void	printAnswerWordSequences(int[] answers) Prints the list of unique answers for each type determined by the given type sequence.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

AnswerChecker

public AnswerChecker(TypedTaggedDocument source)

Constructs a new AnswerChecker to check answers for the given TypedTaggedDocument. The list of answer types is taken from source.getTypeSequence().

Method Detail

getAnswerRanges

public static HashMap getAnswerRanges(int[] answers)

Takes the given list of answer types and converts them into a map from type to a list of Ranges (start/end) indices. The Ranges within each state type are guaranteed to be in linear order. No ranges overlap.

checkBestAnswers

public HashMap checkBestAnswers(Map bestGuessWordSequenceByType)

Takes a Map from Integer (type) -> List of Strings for best guess word sequence and returns a Map from type to PrecisionRecallStats for that state type. For each type, if the best answer is among the correct answers, you get full credit for the type. Otherwise you get a false positive for guessing a bad answer and a false negative for not guessing the right answer. You also get a false negative for each answer type you didn't guess at.

checkAnswers

public HashMap checkAnswers(int[] guesses,
                            boolean strict)

Returns a map from Integer (state type) -> PrecisionRecallStats for that state type. guesses is a sequence of types (e.g. from HMM.getLabelsForSequence(int[])). If a state type is not in the map, it means no states of that type were among the given given guesses or answers. If strict is false, each unique answer need only be extracted once (i.e. slot-filling mode--if the same answer appears in two places and is only extracted once, no false negative is counted). If strict is true, every instance of every answer is scored as either a true positive or a false negative.

printAnswers

public void printAnswers()

Prints answer word sequences for the correct answers for this document.

See Also:

printAnswerWordSequences(int[])

printAnswerWordSequences

public void printAnswerWordSequences(int[] answers)

Prints the list of unique answers for each type determined by the given type sequence. Each line is a type, and all the answers are separated by commas. The line is prefixed with the type number.