edu.stanford.nlp.trees
Class AdwaitStreamTokenizer

java.lang.Object
  |
  +--java.io.StreamTokenizer
        |
        +--edu.stanford.nlp.trees.PennTagbankStreamTokenizer
              |
              +--edu.stanford.nlp.trees.AdwaitStreamTokenizer

public class AdwaitStreamTokenizer

extends PennTagbankStreamTokenizer

Builds a tokenizer for Adwait-style pos tagged directories.

Field Summary

Fields inherited from class java.io.StreamTokenizer

nval, sval, TT_EOF, TT_EOL, TT_NUMBER, TT_WORD, ttype

Constructor Summary

AdwaitStreamTokenizer(Reader r)
Create a tokenizer for Adwait-style sentences.

Methods inherited from class java.io.StreamTokenizer

commentChar, eolIsSignificant, lineno, lowerCaseMode, nextToken, ordinaryChar, ordinaryChars, parseNumbers, pushBack, quoteChar, resetSyntax, slashSlashComments, slashStarComments, toString, whitespaceChars, wordChars

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AdwaitStreamTokenizer

public AdwaitStreamTokenizer(Reader r)

Create a tokenizer for Adwait-style sentences. This sets up simple character meanings for all non-whitespace chars

Parameters:

r - The reader steam