On Wed, 2009-09-16 at 08:25 +0900, Olaf Meeuwissen wrote: > > ANSI C defines ispunct(), which is locale-sensitive. Perhaps that is of > > some use here. Is that also available in python? > > That would be used on the server side so gets whatever locale the server > process was started under or configured to use. Any tickets that are > submitted may well be in a language that doesn't match that. Case in > point, our Trac instances use C/POSIX for their locale. Tickets are in > en_US/en_GB/ja_JP.
Definitely server side, as that is the only config Trac has. I am guessing the text is not identified as such in Trac? I am guessing it is limited to the encoding, like UTF-8. This surely makes it more complicated. Is your Trac translated into Japanese (menus and all)? Or is it mixed English / local? I am guessing the later. Indeed a general solution would help. > > As to Japanese, I am guessing that word boundaries at sentence > > punctuation is ok. It is just spaces that are the problem. > > Are you talking about addressing the problem of splitting at word > boundaries here? Then yes, punctuation would indicate a boundary. But > that doesn't really help. Your two lines above would then result in a > whopping total of _three_ "words": I was. I meant that my issue would not improve or make worse languages like Japanese. > > As to Japanese > I am guessing that word boundaries at sentence punctuation is ok > It is just spaces that are the problem > > Mind you, end-of-line is _not_ necessarily a word boundary in Japanese. That is just an example of <part1><whitespace><part1>. > Anyway, we're not using the ticket summary to harvest tags so it's not > that much of a problem. The area where it "hurts" most is in the wiki > where we don't get the links for free. > > Hope this helps, I guess. I was really hoping to generate a tag cloud based on all words used in selected fields, not just those in a safe keyword context. And we did not even discuss the uselessness of having words like 'a', 'the', 'it' and such included. It all stems from allowing searching other than the keyword part of things. I guess I will adapt. My python programming is not what it should be. Now, if this had been written in Tcl... --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Trac Users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/trac-users?hl=en -~----------~----~----~----~------~----~------~--~---
