On 9/16/2016 8:30 AM, Janusz S. Bien wrote:
Quote/Cytat - Eric Muller <[email protected]> (pią, 16 wrz 2016,
17:03:54):
On 9/16/2016 6:52 AM, Janusz S. Bień wrote:
(when working on a corpus of historical Polish we
noticed some cases where standard Unicode equivalence was not
convenient).
I'm very interested to know more about those cases.
For our search engine we were unable to use compatibility equivalence
"out of the box" for splitting the ligature because it also converted
long s to short s while we wanted to preserve the distinction.
I am interested in the problems with *canonical* equivalence. I thought
that you were talking about those before.
Compatibility equivalence is a completely different beast. It is, IMHO,
too coarse a tool and best forgotten. For any particular task, it's
typically doing too much (e.g. long/short s folding in your case) and
too little (not everything you need). There was an attempt at improving
the situation, by providing a whole bunch of fine grained, targeted
transformations (http://www.unicode.org/reports/tr30/), but that did not
pan out.
Eric.
Thanks,
Eric.
