Inheritance diagram for grammarscope.io.read.PennTreebankTokenizer:

Collaboration diagram for grammarscope.io.read.PennTreebankTokenizer:

[legend]

Classes
class	EnglishTreebankStreamTokenizer

Public Member Functions
	PennTreebankTokenizer (final Reader r)

Public Member Functions inherited from grammarscope.io.read.TokenizerAdapter
	TokenizerAdapter (final StreamTokenizer st)

String	getNext ()

void	setEolString (final String eolString)

boolean	isEol (final String str)

Public Member Functions inherited from grammarscope.io.read.AbstractTokenizer< String >
T	next ()

boolean	hasNext ()

void	remove ()

T	peek ()

List< T >	tokenize ()

Additional Inherited Members
Protected Member Functions inherited from grammarscope.io.read.AbstractTokenizer< String >
abstract T	getNext ()

Protected Attributes inherited from grammarscope.io.read.TokenizerAdapter
final StreamTokenizer	st

String	eolString = "<EOL>"

Protected Attributes inherited from grammarscope.io.read.AbstractTokenizer< String >
T	nextToken

Constructor & Destructor Documentation

◆ PennTreebankTokenizer()

grammarscope.io.read.PennTreebankTokenizer.PennTreebankTokenizer ( final Reader r )

The documentation for this class was generated from the following file:

grammarscope/library/src/main/java/grammarscope/io/read/PennTreebankTokenizer.java