It appears you are thinking of rules, quite different than the ones I am thinking of.
We will see in time ... Ruud > W dniu 2014-09-24 o 21:03, R.J. Baars pisze: >> Maybe we agree to disagree.. >> >> Having them as one token makes detecting patterns easy using regular >> expressions.. > > But writing suggestions becomes a nightmare, as you have to use groups > and it becomes complex very soon. > > Marcin > >> >> Ruud >> >> >>> For Polish, I actually want to have numbers tokenized. It makes writing >>> number format rules easier. For example, we use comma as a decimal >>> separator, not a dot. >>> >>> Best >>> Marcin >>> 24 wrz 2014 17:12 "Andriy Rysin" <ary...@gmail.com> napisaà â(a): >>> >>>> Hmm, so when you meet 1.001 in the document you would not know if it's >>>> a one 1001 or 1,001... >>>> In Ukrainian I have rule that require following noun to be in a proper >>>> form and it'll be different for whole and fractional number endings... >>>> >>>> And if many documents treat dot as comma would not it make sense to >>>> create a rule that catches that and proposes correct format? >>>> >>>> Andriy >>>> >>>> 2014-09-24 10:53 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>: >>>>> >>>>> Even when the locale would be nl, there are so many document using >>>>> the >>>>> English format, we would have to use both. >>>>> >>>>> But if . and , are treated the same when between digits, it would >>>>> work >>>>> anyway. >>>>> >>>>> Ruud >>>>> >>>>>> I did some code for Ukrainan that ignores decimal separator "," >>>> within >>>>>> numbers when tokenizing. I didn't address number group separator "." >>>>>> yet (looks like this will require srx file change), but . is not >>>>>> used >>>>>> widely so I didn't consider it as important. But it would be nice if >>>>>> this was handled at common level (taking to account locale of the >>>>>> language). >>>>>> >>>>>> Andriy >>>>>> >>>>>> >>>>>> 2014-09-24 8:03 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>: >>>>>>> Numbers like 1.234 or 1,000.00 are tokenized into several tokens, >>>> while >>>>>>> it >>>>>>> is one number. >>>>>>> >>>>>>> What do you think about changing the tokenizer to treat them as one >>>>>>> number? This would maybe affect all languages having rules >>>> concerning >>>>>>> numbers, so this is not the right time, but maybe after releasing >>>> 2.7? >>>>>>> >>>>>>> Ruud >>>>>>> >>>>>>> >>>>>>> >>>> ------------------------------------------------------------------------------ >>>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>>> Reports >>>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White >>>>>>> paper >>>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog >>>>>>> Analyzer >>>>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>>>>>> _______________________________________________ >>>>>>> Languagetool-devel mailing list >>>>>>> Languagetool-devel@lists.sourceforge.net >>>>>>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >>>>>> >>>>>> >>>> ------------------------------------------------------------------------------ >>>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>>> Reports >>>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>>>>> _______________________________________________ >>>>>> Languagetool-devel mailing list >>>>>> Languagetool-devel@lists.sourceforge.net >>>>>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >>>>>> >>>>> >>>>> >>>>> >>>>> >>>> ------------------------------------------------------------------------------ >>>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>>> Reports >>>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>>>> _______________________________________________ >>>>> Languagetool-devel mailing list >>>>> Languagetool-devel@lists.sourceforge.net >>>>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>>> Reports >>>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Languagetool-devel mailing list >>>> Languagetool-devel@lists.sourceforge.net >>>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >>>> >>>> >>> ------------------------------------------------------------------------------ >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >>> Reports >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >>> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk_______________________________________________ >>> Languagetool-devel mailing list >>> Languagetool-devel@lists.sourceforge.net >>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >>> >> >> >> >> ------------------------------------------------------------------------------ >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> _______________________________________________ >> Languagetool-devel mailing list >> Languagetool-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >> >> . >> > > > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk > _______________________________________________ > Languagetool-devel mailing list > Languagetool-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/languagetool-devel > ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ Languagetool-devel mailing list Languagetool-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/languagetool-devel
