On Feb 19, 2012, at 2:44 PM, Brendan Eich wrote:
> Allen Wirfs-Brock wrote:
>> On Feb 19, 2012, at 2:15 PM, Brendan Eich wrote:
>>> I'm not a Unicode expert but I believe the latter is called "character".
>>
>> Me neither, but I believe the correct term is "code point" which refers to
>> the full 21-bit code while "Unicode character" is the logical entity
>> corresponding to that code point. That usage of "character" is difference
>> from the current usage within ECMAScript where "character" is what we call
>> the elements of the vector of 16-bit number that are used to represent a
>> String value. You can access then as string values of length 1 via [ ] or
>> as numeric values via the charCodeAt method.
>
> Thanks. We have a confusing transposition of terms between Unicode and
> ECMA-262, it seems. Should we fix?
The ES5.1 spec.is ok because it always uses (as defined in section 6) the term
"Unicode character" when it means exactly that and uses "character" when
talking about the elements of String values. It says that both "code unit" and
"character" refer to a 16-bit unsigned value.
Your proposal would change that equivalence. In one sense, the BSR would be a
switch that controls whether a ES "character" corresponds to "code unit" or a
"code point"
>
>>> JS must keep the "\uXXXX" notation for uint16 storage units, and one can
>>> create invalid Unicode strings already. This hazard does not go away, we
>>> keep compatibility, but the BRS adds no new hazards and in practice, if
>>> well-used, should reduce the incidence of invalid-Unicode-string bugs.
>>>
>>> The "\u{...}" notation is independent and should work whatever the BRS
>>> setting, IMHO. In "UCS-2" (default) setting, "\u{...}" can make pairs. In
>>> "UTF-16" setting, it makes only characters. And of course in the latter
>>> case indexing and length count characters.
>>
>> I think your names for the BRS modes are misleading.
>
> You got me, in fact I used "full Unicode" for the BRS-thrown setting
> elsewhere.
>
> My implementor's bias is showing, because I expect many engines would use
> UTF-16 internally and have non-O(1) indexing for strings with the
> contains-non-BMP-and-BRS-set-to-full-Unicode flag bit.
A fine implementation, but not observable. Another implementation approach
that would preserve O(1) indexing would be to simply have two or three
different internal string representations with 1, 2, or 4 byte internal
characters. (You can automatically pick the needed character size when the
string is created because string are immutable and created with their value).
A not-quite O(1) approach would segment strings into substring spans using such
an representation. Representation choice probably depends a lot on what you
think are the most common use cases. If it is string processing in JS then a
fast representations is probably what you want to choose. If it is just
passing text that is already UTF-8 or UTF-16 encoded from inputs to output
then a representation that minimizing transcoding would probably be a higher
priority.
Allen
_______________________________________________
es-discuss mailing list
[email protected]
https://mail.mozilla.org/listinfo/es-discuss
