edu.stanford.nlp.annotation
Class PTBLexer

java.lang.Object
  |
  +--edu.stanford.nlp.annotation.PTBLexer

public class PTBLexer

extends Object

This class is a scanner generated by JFlex 1.3.5 on 8/5/02 12:16 AM from the specification file file:/dfs/hake/0/grow/lexer/jflex/ptblexer.flex

Field Summary

static int	YYEOF This character denotes the end of file
static int	YYINITIAL lexical states

Method Summary

static void	main(String[] argv) Runs the scanner on input files.
void	yybegin(int newState) Enters a new lexical state
char	yycharat(int pos) Returns the character at position pos from the matched text.
void	yyclose() Closes the input stream.
int	yylength() Returns the length of the matched text region.
int	yylex() Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
void	yyreset(Reader reader) Closes the current stream, and resets the scanner to read from a new input stream.
int	yystate() Returns the current lexical state.
String	yytext() Returns the text matched by the current regular expression.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

YYEOF

public static final int YYEOF

This character denotes the end of file

See Also:

Constant Field Values

YYINITIAL

public static final int YYINITIAL

lexical states

See Also:

Constant Field Values

Method Detail

yyclose

public final void yyclose()
                   throws IOException

Closes the input stream.

IOException

yyreset

public final void yyreset(Reader reader)
                   throws IOException

Closes the current stream, and resets the scanner to read from a new input stream. All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to YY_INITIAL.

Parameters:

reader - the new input stream

IOException

yystate

public final int yystate()

Returns the current lexical state.

yybegin

public final void yybegin(int newState)

Enters a new lexical state

Parameters:

newState - the new lexical state

yytext

public final String yytext()

Returns the text matched by the current regular expression.

yycharat

public final char yycharat(int pos)

Returns the character at position pos from the matched text. It is equivalent to yytext().charAt(pos), but faster

Parameters:

pos - the position of the character to fetch. A value from 0 to yylength()-1.

Returns:

the character at position pos

yylength

public final int yylength()

Returns the length of the matched text region.

yylex

public int yylex()
          throws IOException

Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.

Returns:

the next token

Throws:

IOException - if any I/O-Error occurs

main

public static void main(String[] argv)

Runs the scanner on input files. This is a standalone scanner, i.e. it will print any unmatched text to System.out unchanged.

Parameters:

argv - the command line, contains the filenames to run the scanner on.