org.cdlib.xtf.textIndexer
Class UnicodeNormalizingFilter

Object
  extended by TokenStream
      extended by TokenFilter
          extended by UnicodeNormalizingFilter

public class UnicodeNormalizingFilter
extends TokenFilter

Apply Unicode Normalization to the tokens.

Author:
Marcos Fragomeni
See Also:
Normalizer

Field Summary
private  FastCache<String,String> cache
          Keep a cache of lookups performed to-date
private static int CACHE_SIZE
          How many recent mappings to maintain
 
Fields inherited from class TokenFilter
input
 
Constructor Summary
UnicodeNormalizingFilter(TokenStream input)
           
 
Method Summary
 Token next()
           
 
Methods inherited from class TokenFilter
close
 
Methods inherited from class Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

CACHE_SIZE

private static final int CACHE_SIZE
How many recent mappings to maintain

See Also:
Constant Field Values

cache

private FastCache<String,String> cache
Keep a cache of lookups performed to-date

Constructor Detail

UnicodeNormalizingFilter

public UnicodeNormalizingFilter(TokenStream input)
Method Detail

next

public Token next()
           throws IOException
Specified by:
next in class TokenStream
Throws:
IOException