grammarscope.server.parser

Class Segmenter

java.lang.Object

grammarscope.server.parser.Segmenter

All Implemented Interfaces:

ISegmenter

public class Segmenter
extends java.lang.Object
implements ISegmenter

Segmenter

Author:

Bernard Bou

Field Summary

Fields

Modifier and Type	Field and Description
private edu.stanford.nlp.process.TokenizerFactory<edu.stanford.nlp.ling.CoreLabel>	tokenizerFactory

Constructor Summary

Constructors

Constructor and Description
Segmenter(Language language) Constructor

Method Summary

Modifier and Type	Method and Description
private static java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>>	getSentences(java.util.List<edu.stanford.nlp.ling.CoreLabel> words) Segment words into sentences
java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>>	getSentenceWords(java.lang.String text) Get sentences from text
java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>>	getSentenceWords(java.net.URL url) Get document from URL
java.lang.String	getTokenizeModel() Get tokenize model
private java.util.List<edu.stanford.nlp.ling.CoreLabel>	getTokens(java.lang.String text) Get list of tokens from text
private java.util.List<edu.stanford.nlp.ling.CoreLabel>	getTokens(java.net.URL url) Get list of tokens from URL

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tokenizerFactory

private final edu.stanford.nlp.process.TokenizerFactory<edu.stanford.nlp.ling.CoreLabel> tokenizerFactory

Constructor Detail

Segmenter

public Segmenter(Language language)

Constructor

Parameters:

language - language

Method Detail

getTokens

private java.util.List<edu.stanford.nlp.ling.CoreLabel> getTokens(java.lang.String text)

Get list of tokens from text

Parameters:

text - text

Returns:

list of tokens

getTokens

private java.util.List<edu.stanford.nlp.ling.CoreLabel> getTokens(java.net.URL url)
                                                           throws java.io.IOException

Get list of tokens from URL

Parameters:

url - document URL

Returns:

list of tokens

Throws:

java.io.IOException - exception

getSentences

private static java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>> getSentences(java.util.List<edu.stanford.nlp.ling.CoreLabel> words)

Segment words into sentences

Parameters:

words - words

Returns:

sentence-indexed words

getSentenceWords

public java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>> getSentenceWords(java.lang.String text)

Description copied from interface: ISegmenter

Get sentences from text

Specified by:

getSentenceWords in interface ISegmenter

Parameters:

text - text to segment

Returns:

list of sentences

getSentenceWords

public java.util.List<java.util.List<edu.stanford.nlp.ling.CoreLabel>> getSentenceWords(java.net.URL url)
                                                                                 throws java.io.IOException

Description copied from interface: ISegmenter

Get document from URL

Specified by:

getSentenceWords in interface ISegmenter

Parameters:

url - URL

Returns:

list of sentences

Throws:

java.io.IOException - exception

getTokenizeModel

public java.lang.String getTokenizeModel()

Description copied from interface: ISegmenter

Get tokenize model

Specified by:

getTokenizeModel in interface ISegmenter

Returns:

model