|
||||||||||
PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |
Class Summary | |
DefaultTokeniser | A composed tokeniser containing a SimpleTokeniser and a
Transducer . |
DFSMState | Implements a state of the deterministic finite state machine of the tokeniser. |
FSMState | A state of the finite state machine that is the kernel tokeniser |
SimpleTokeniser | Implementation of a Unicode rule based tokeniser. |
TestTokeniser | |
UnicodeType | Used as an object wrapper that holds an Unicode type (the byte value of the static member of java.lang.Character). |
Exception Summary | |
InvalidRuleException | Fired when an invalid tokeniser rule is found |
TokeniserException | The top level exception for all the exceptions fired by the tokeniser |
|
||||||||||
PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES |