[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

[Christopher Barker]

On Sun, Nov 14, 2021 at 4:53 PM Steven D'Aprano <st...@pearwood.info> wrote:

> Out of all the approximately thousand bazillion ways to write obfuscated
> Python code, which may or may not be malicious, why are Unicode
> confusables worth this level of angst and concern?
>

I for one am not full of angst nor particularly concerned. Though ti's a
fine idea to inform folks about h this issues.

I am, however, surprised and disappointed by the NKFC normalization.

For example, in writing math we often use different scripts to mean
different things (e.g. TeX's
Blackboard Bold). So if I were to use some of the Unicode Mathematical
Alphanumeric Symbols, I wouldn't want them to get normalized.

Then there's the question of when this normalization happens (and when it
doesn't). If one is doing any kind of metaprogramming, even just using
getattr() and setattr(), things could get very confusing:

In [55]: class Junk:
    ...:     𝗵ｅ𝓵𝔩º = "hello"
    ...:

In [56]: setattr(Junk, "ᵖ𝖗𝐢𝘯𝓽", "print")

In [57]: dir(Junk)
Out[57]:
 '__weakref__',
<snip>
 'hello',
 'ᵖ𝖗𝐢𝘯𝓽']

In [58]: Junk.hello
Out[58]: 'hello'

In [59]: Junk.𝗵ｅ𝓵𝔩º
Out[59]: 'hello'

In [60]: Junk.print
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-60-f2a7d3de5d06> in <module>
----> 1 Junk.print

AttributeError: type object 'Junk' has no attribute 'print'

In [61]: Junk.ᵖ𝖗𝐢𝘯𝓽
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-61-004f4c8b2f07> in <module>
----> 1 Junk.ᵖ𝖗𝐢𝘯𝓽

AttributeError: type object 'Junk' has no attribute 'print'

In [62]: getattr(Junk, "ᵖ𝖗𝐢𝘯𝓽")
Out[62]: 'print'

Would a proposal to switch the normalization to NFC only have any hope of
being accepted?

and/or adding normaliztion to setattr() and maybe other places where names
are set in code?

-CHB

-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z2AIS6Y6NVNF5QSD7GMTB76NSP6NAIKV/
Code of Conduct: http://python.org/psf/codeofconduct/