Package gate.creole.tokeniser

Class Summary
DefaultTokeniser Implementation of a Unicode rule based tokeniser.
TestTokeniser  
 

Exception Summary
InvalidRuleException Fired when an invalid tokeniser rule is found
TokeniserException The top level exception for all the exceptions fired by the tokeniser