+1 Che Dong wrote: >>Attached StandardTokenizer.jj with Sigram Based east >>asia language support: >>tested under Windows and GNU/Linux >> >>Just treat different UnicodeBlock with different word >>segment method. >> >>Hope in the future released we can add more language >>support in StandardTokenizer.jj step by step and keep >>it fit for most i18n environment. >>Some common app, like Jive, can use it as default >>Analyser. >>Use localized Analyzier for advanced usage. >> >>Thank you. >> >>Che, Dong
-- To unsubscribe, e-mail: <mailto:[EMAIL PROTECTED]> For additional commands, e-mail: <mailto:[EMAIL PROTECTED]>
