Maybe we agree to disagree.. Having them as one token makes detecting patterns easy using regular expressions..
Ruud > For Polish, I actually want to have numbers tokenized. It makes writing > number format rules easier. For example, we use comma as a decimal > separator, not a dot. > > Best > Marcin > 24 wrz 2014 17:12 "Andriy Rysin" <ary...@gmail.com> napisaÅ(a): > >> Hmm, so when you meet 1.001 in the document you would not know if it's >> a one 1001 or 1,001... >> In Ukrainian I have rule that require following noun to be in a proper >> form and it'll be different for whole and fractional number endings... >> >> And if many documents treat dot as comma would not it make sense to >> create a rule that catches that and proposes correct format? >> >> Andriy >> >> 2014-09-24 10:53 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>: >> > >> > Even when the locale would be nl, there are so many document using the >> > English format, we would have to use both. >> > >> > But if . and , are treated the same when between digits, it would work >> > anyway. >> > >> > Ruud >> > >> >> I did some code for Ukrainan that ignores decimal separator "," >> within >> >> numbers when tokenizing. I didn't address number group separator "." >> >> yet (looks like this will require srx file change), but . is not used >> >> widely so I didn't consider it as important. But it would be nice if >> >> this was handled at common level (taking to account locale of the >> >> language). >> >> >> >> Andriy >> >> >> >> >> >> 2014-09-24 8:03 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>: >> >>> Numbers like 1.234 or 1,000.00 are tokenized into several tokens, >> while >> >>> it >> >>> is one number. >> >>> >> >>> What do you think about changing the tokenizer to treat them as one >> >>> number? This would maybe affect all languages having rules >> concerning >> >>> numbers, so this is not the right time, but maybe after releasing >> 2.7? >> >>> >> >>> Ruud >> >>> >> >>> >> >>> >> ------------------------------------------------------------------------------ >> >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >> Reports >> >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> >>> >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> >>> _______________________________________________ >> >>> Languagetool-devel mailing list >> >>> Languagetool-devel@lists.sourceforge.net >> >>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >> Reports >> >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> >> _______________________________________________ >> >> Languagetool-devel mailing list >> >> Languagetool-devel@lists.sourceforge.net >> >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >> >> >> > >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS >> Reports >> > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Languagetool-devel mailing list >> > Languagetool-devel@lists.sourceforge.net >> > https://lists.sourceforge.net/lists/listinfo/languagetool-devel >> >> >> ------------------------------------------------------------------------------ >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer >> >> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk >> _______________________________________________ >> Languagetool-devel mailing list >> Languagetool-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel >> >> > ------------------------------------------------------------------------------ > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer > http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk_______________________________________________ > Languagetool-devel mailing list > Languagetool-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/languagetool-devel > ------------------------------------------------------------------------------ Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk _______________________________________________ Languagetool-devel mailing list Languagetool-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/languagetool-devel
