Having a number split into multiple tokens will definitely make rules
around those numbers quite complicated.
On Sep 24, 2014 3:04 PM, "R.J. Baars" <r.j.ba...@xs4all.nl> wrote:
> Maybe we agree to disagree..
>
> Having them as one token makes detecting patterns easy using regular
> expressions..
>
> Ruud
>
>
> > For Polish, I actually want to have numbers tokenized. It makes writing
> > number format rules easier. For example, we use comma as a decimal
> > separator, not a dot.
> >
> > Best
> > Marcin
> > 24 wrz 2014 17:12 "Andriy Rysin" <ary...@gmail.com> napisał(a):
> >
> >> Hmm, so when you meet 1.001 in the document you would not know if it's
> >> a one 1001 or 1,001...
> >> In Ukrainian I have rule that require following noun to be in a proper
> >> form and it'll be different for whole and fractional number endings...
> >>
> >> And if many documents treat dot as comma would not it make sense to
> >> create a rule that catches that and proposes correct format?
> >>
> >> Andriy
> >>
> >> 2014-09-24 10:53 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>:
> >> >
> >> > Even when the locale would be nl, there are so many document using the
> >> > English format, we would have to use both.
> >> >
> >> > But if . and , are treated the same when between digits, it would work
> >> > anyway.
> >> >
> >> > Ruud
> >> >
> >> >> I did some code for Ukrainan that ignores decimal separator ","
> >> within
> >> >> numbers when tokenizing. I didn't address number group separator "."
> >> >> yet (looks like this will require srx file change), but . is not used
> >> >> widely so I didn't consider it as important. But it would be nice if
> >> >> this was handled at common level (taking to account locale of the
> >> >> language).
> >> >>
> >> >> Andriy
> >> >>
> >> >>
> >> >> 2014-09-24 8:03 GMT-04:00 R.J. Baars <r.j.ba...@xs4all.nl>:
> >> >>> Numbers like 1.234 or 1,000.00 are tokenized into several tokens,
> >> while
> >> >>> it
> >> >>> is one number.
> >> >>>
> >> >>> What do you think about changing the tokenizer to treat them as one
> >> >>> number? This would maybe affect all languages having rules
> >> concerning
> >> >>> numbers, so this is not the right time, but maybe after releasing
> >> 2.7?
> >> >>>
> >> >>> Ruud
> >> >>>
> >> >>>
> >> >>>
> >>
> ------------------------------------------------------------------------------
> >> >>> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> >> >>> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
> >> Reports
> >> >>> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> >> >>> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> >> >>>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> >> >>> _______________________________________________
> >> >>> Languagetool-devel mailing list
> >> >>> Languagetool-devel@lists.sourceforge.net
> >> >>> https://lists.sourceforge.net/lists/listinfo/languagetool-devel
> >> >>
> >> >>
> >>
> ------------------------------------------------------------------------------
> >> >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> >> >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
> >> Reports
> >> >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> >> >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> >> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> >> >> _______________________________________________
> >> >> Languagetool-devel mailing list
> >> >> Languagetool-devel@lists.sourceforge.net
> >> >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel
> >> >>
> >> >
> >> >
> >> >
> >> >
> >>
> ------------------------------------------------------------------------------
> >> > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> >> > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS
> >> Reports
> >> > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> >> > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> >> >
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> >> > _______________________________________________
> >> > Languagetool-devel mailing list
> >> > Languagetool-devel@lists.sourceforge.net
> >> > https://lists.sourceforge.net/lists/listinfo/languagetool-devel
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> >> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> >> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> >> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Languagetool-devel mailing list
> >> Languagetool-devel@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/languagetool-devel
> >>
> >>
> >
> ------------------------------------------------------------------------------
> > Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> > Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> > Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> > Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk_______________________________________________
> > Languagetool-devel mailing list
> > Languagetool-devel@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/languagetool-devel
> >
>
>
>
>
> ------------------------------------------------------------------------------
> Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
> Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
> Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
> Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
>
> http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
> _______________________________________________
> Languagetool-devel mailing list
> Languagetool-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/languagetool-devel
>
------------------------------------------------------------------------------
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://pubads.g.doubleclick.net/gampad/clk?id=154622311&iu=/4140/ostg.clktrk
_______________________________________________
Languagetool-devel mailing list
Languagetool-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/languagetool-devel
