|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
ObjectTokenStream
FacetTokenizer
public class FacetTokenizer
Performs special tokenization for facet fields. Looks for the hierarchy marker "::" between hierarchy levels. For instance, the string "US::California::Alameda County::Berkeley" would be made into four tokens: US US::California US::California::Alameda County US::California::Alameda County::Berkeley
Field Summary | |
---|---|
(package private) Token |
nextToken
|
(package private) int |
pos
|
(package private) String |
str
|
Constructor Summary | |
---|---|
FacetTokenizer(String str)
Construct a token stream to remove accents from the input tokens. |
Method Summary | |
---|---|
Token |
next()
Retrieve the next token in the stream. |
Methods inherited from class TokenStream |
---|
close |
Methods inherited from class Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
String str
int pos
Token nextToken
Constructor Detail |
---|
public FacetTokenizer(String str)
str
- The string to tokenizeMethod Detail |
---|
public Token next() throws IOException
next
in class TokenStream
IOException
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |