Roger Oberholtzer <[email protected]> writes:
> On Tue, 2009-09-15 at 16:48 +0900, Olaf Meeuwissen wrote:
>
>> For that matter, splitting on whitespace sucks when you use Japanese
>> (which normally doesn't use whitespace). Splitting should be done on
>> word boundaries. Hopefully 0.12 will address at least some of these
>> issues for non-POSIX locales.
>
> I am not doubting the complexity. Perhaps it could be as simple as a
> configured parameter where one could add characters, beyond the ones
> defined by the locale, that are to be treated as whitespace/punctuation.
> If you are happy with the current result, just leave the list alone.
>
> ANSI C defines ispunct(), which is locale-sensitive. Perhaps that is of
> some use here. Is that also available in python?
That would be used on the server side so gets whatever locale the server
process was started under or configured to use. Any tickets that are
submitted may well be in a language that doesn't match that. Case in
point, our Trac instances use C/POSIX for their locale. Tickets are in
en_US/en_GB/ja_JP.
> As to Japanese, I am guessing that word boundaries at sentence
> punctuation is ok. It is just spaces that are the problem.
Are you talking about addressing the problem of splitting at word
boundaries here? Then yes, punctuation would indicate a boundary. But
that doesn't really help. Your two lines above would then result in a
whopping total of _three_ "words":
As to Japanese
I am guessing that word boundaries at sentence punctuation is ok
It is just spaces that are the problem
Mind you, end-of-line is _not_ necessarily a word boundary in Japanese.
Anyway, we're not using the ticket summary to harvest tags so it's not
that much of a problem. The area where it "hurts" most is in the wiki
where we don't get the links for free.
Hope this helps,
--
Olaf Meeuwissen, LPIC-2 FLOSS Engineer -- AVASYS Corporation
FSF Associate Member #1962 Help support software freedom
http://www.fsf.org/jf?referrer=1962
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Trac
Users" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/trac-users?hl=en
-~----------~----~----~----~------~----~------~--~---
