Yonik Seeley wrote:
A related problem exists even if the prefix length vInt is changed to
represent the number of unicode chars (as opposed to number of java chars),
right? The prefix length is no longer the offset into the char[] to put the
suffix.
Yes, I suppose this is a problem too. Sigh.
Another approach might be to convert the target to a UTF-8 byte[]
and do all comparisons on byte[]. UTF-8 has some very nice properties,
including that the byte[] representation of UTF-8 strings compare the same
as UCS-4 would.
I was not aware of that, but I see you are correct:
o The byte-value lexicographic sorting order of UTF-8 strings is the
same as if ordered by character numbers.
(From http://www.faqs.org/rfcs/rfc3629.html)
That makes the byte representation much more palatable, since Lucene
orders terms lexicographically.
Doug
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
