The ZWJ and ZWNJ are normally ignored in collation (including matching and searching), you'll see that if you look at the data for the Unicode Collation Algorithm.
However, this raises any interesting issue. There are a few particular sequences where there is a semantic difference caused by the presence of this character. Those should be added to the collation sequences in the Unicode CLDR database (www.unicode.org/cldr/) âMark ----- Original Message ----- From: "Bob Eaton" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, November 04, 2004 04:58 Subject: Different Indic strings due to presence of ZWJ... > A question has come up recently about two similarly looking words that don't > match based on a 'string comparison' due to the presence of the ZWJ. > > For example, the two strings /àààààà/ and > /ààààâàà/ are identical to look at, but they differ in that > the latter has a ZWJ between the ààâ and the à. > > The problem is that certain âhalf-consonantâ + âfull-consonantâ > conjuncts require the ZWJ in order to prevent a âfull conjunctâ form > from occurring (e.g. /ààà/ as /ààâà/). The only way to prevent > the full conjunct form is to insert the ZWJ. > > But in the âààâ plus àâ case, there is no more conjunct form > than that. So both with and without the ZWJ gives the same presentation > result. > > This means that ultimately, the ZWJ is unnecessary in some cases of > âhalf-plus-fullâ conjuncts, but it is necessary in others (i.e. > /ààâà/). > > The keyboard I use has a key to press to get âhalf-consonantsâ (i.e. by > inserting both the halant and the ZWJ, since that is what is required in the > âharderâ case). The problem is I use it also to get the half-à even > though in that particular case, it isnât necessary. But a colleague is > using the 'halant-only' key, since the ZWJ is not technically necessary in > this case. The result is that the software thinks the two strings are > different. > > Having the software think this, is, first of all, a real hassle, since most > users canât tell the difference between the two and wonât know why the > software thinks they're different. > > So I have two questions: > 1) What does your keyboard do in this respect when typing "half+full" > consonant conjuncts? Do you only use the ZWJ where it is absolutely > necessary (i.e. /ààâà/, but not /ààà/). > > 2) If different, what do you think it ought to do? > > Thanks, > > Bob > > > >
