grammarscope.io.read

Class PennTreebankTokenizer.EnglishTreebankStreamTokenizer

java.lang.Object

java.io.StreamTokenizer

grammarscope.io.read.PennTreebankTokenizer.EnglishTreebankStreamTokenizer

Enclosing class:

PennTreebankTokenizer

private static class PennTreebankTokenizer.EnglishTreebankStreamTokenizer
extends java.io.StreamTokenizer

A StreamTokenizer for PennTreebank trees.

Field Summary

Fields inherited from class java.io.StreamTokenizer

nval, sval, TT_EOF, TT_EOL, TT_NUMBER, TT_WORD, ttype

Constructor Summary

Constructors

Modifier	Constructor and Description
private	EnglishTreebankStreamTokenizer(java.io.Reader r) Create a StreamTokenizer for PennTreebank trees.

Method Summary

Methods inherited from class java.io.StreamTokenizer

commentChar, eolIsSignificant, lineno, lowerCaseMode, nextToken, ordinaryChar, ordinaryChars, parseNumbers, pushBack, quoteChar, resetSyntax, slashSlashComments, slashStarComments, toString, whitespaceChars, wordChars

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

EnglishTreebankStreamTokenizer

private EnglishTreebankStreamTokenizer(java.io.Reader r)

Create a StreamTokenizer for PennTreebank trees. This sets up all the character meanings for treebank trees

Parameters:

r - The reader steam