On 5/22/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote:
> That's why Java and C++ use \u, so you would write L\u00F6wis
> as an identifier. ...
> I think you are really arguing for \u escapes in identifiers here.
Yes, that is effectively what I was suggesting.
> *This* is truly unambiguous. I claim that it is also useless.
It means users could see the usability benefits of PEP3131, but the
python internals could still work with ASCII only.
It simplifies checking for identifiers that *don't* stick to ASCII,
which reduces some of the concerns about confusable characters, and
which ones to allow.
Short list of judgment calls that we need to resolve if we go with
non-ASCII identifiers, but can largely ignore if we just use escaping:
Based only on UAX 31:
ID vs XID (unicode changed their mind on recommendations)
include stability extensions? (*Python* didn't allow those
letters previously.)
which of ID_CONTINUE should be left out. (We don't want "-", and
some of the punctuation and other marks may be closer to "-" than to
"_". Or they might not be, and I don't know how to judge that.)
layout and control charcters (At the top of section 2, tr31
recommends acting as though they weren't there ... but if we use a
normal (unicode) string, then they will still affect the hash. Down
in 2.2, they say not to permit them, except sometimes...)
Canonicalization
Combining Marks should be accepted (only as continuation chars),
but not if they're enclosing marks, because ... well, I'm not sure,
but I'll have to trust them.
Specific character Adjustments (sec 2.3) -- The example suggests
that we might have to tailor for our use of "_", though I didn't get
that from the table. They do suggest tailoring out certain
Decomposition Types.
Additional (non-letter?) characters which may occur in words (see
UAX29, but I don't claim to fully understand it)
Undefined code points, particularly those which might be defined later?
Should we exclude the letters that look like punctuation? A
proposed update (http://www.unicode.org/reports/tr31/tr31-8.html)
mentions U+02B9 (modifier letter prime) only because the visually
equivalent U+0374 (Greek Numeral Sign) shouldn't be an identifier, but
does fold to it under (some?) canonicalization. (They suggest
allowing both, instead of neither.)
Then TR 39 http://www.unicode.org/reports/tr39/ recommends excluding
(most, but not all of)
characters not in modern use;
characters only used in specialized fields, such as liturgical
characters, mathematical letter-like symbols, and certain phonetic
alphabetics;
and ideographic characters that are not part of a set of core CJK
ideographs consisting of the CJK Unified Ideographs block plus IICore
(the set of characters defined by the IRG as the minimal set of
required ideographs for East Asian use).
They summarize this in
http://www.unicode.org/reports/tr39/data/xidmodifications.txt; I
wouldn't add the hyphen-minus back in, but I don't know whether
katakana middle dot should be allowed.
Should mixed-script identifiers be allowed? According to TR 36
(http://www.unicode.org/reports/tr36/) ASCII only is the safest, and
that is followed by limits on mixed-script identifiers. Those limits
sound reasonable to me, but ... I'm not the one who would be mixing
them.
Note that even "highly restrictive" allows ASCII + Han + Hiragana +
Katakana, ASCII + Han + Bopomofo, and ASCII + Han + Hangul. (I think
we wanted at least the ASCII numbers with anything.)
-jJ
_______________________________________________
Python-3000 mailing list
Python-3000@python.org
http://mail.python.org/mailman/listinfo/python-3000
Unsubscribe:
http://mail.python.org/mailman/options/python-3000/archive%40mail-archive.com
