| Interface | Description |
|---|---|
| Tokenizer<T> |
Tokenizers break up text into individual Objects.
|
| Class | Description |
|---|---|
| AbstractTokenizer<T> |
An abstract tokenizer.
|
| PennTreebankTokenizer | |
| PennTreebankTokenizer.EnglishTreebankStreamTokenizer |
A StreamTokenizer for PennTreebank trees.
|
| TokenizerAdapter |
This class adapts between a
java.io.StreamTokenizer and a edu.stanford.nlp.process.Tokenizer. |
| TreeFactory | |
| TreeNormalizer | |
| TreeReader |
This class implements the
TreeReader interface to read Penn Treebank-style files. |