edu.stanford.nlp.process
Class SimpleTokenizer

java.lang.Object
  |
  +--edu.stanford.nlp.process.SimpleTokenizer

All Implemented Interfaces:

Tokenizer

public class SimpleTokenizer

extends Object

implements Tokenizer

A tokenizer has methods for tokenizing a String as words or sentences

Constructor Summary

SimpleTokenizer()

Method Summary

Collection	tokenize(InputStream textInputStream) Throws HeadlessException
Collection	tokenize(Reader textReader) Throws HeadlessException
Collection	tokenize(String text) Returns text as a Collection of Words.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleTokenizer

public SimpleTokenizer()

Method Detail

tokenize

public Collection tokenize(String text)

Returns text as a Collection of Words.

Specified by:

tokenize in interface Tokenizer

tokenize

public Collection tokenize(Reader textReader)

Throws HeadlessException

Specified by:

tokenize in interface Tokenizer

tokenize

public Collection tokenize(InputStream textInputStream)

Throws HeadlessException

Specified by:

tokenize in interface Tokenizer