[Python-Dev] Re: [Python-checkins] bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (GH-22575)

[Walter Dörwald]

On 7 Oct 2020, at 1:27, Victor Stinner wrote:


Hi Walter,


https://github.com/python/cpython/commit/a8bf44d04915f7366d9f8dfbf84822ac37a4bab3


Le mar. 6 oct. 2020 à 17:02, Walter Dörwald  
a écrit :
It would be even simpler to use unicodedata.lookup() which returns 
the unicode character when passed the name of the character


That was my first idea as well when I reviewed the change, but the
function contains this comment:

def checkletter(self, name, code):
# Helper that put all \N escapes inside eval'd raw strings,
# to make sure this script runs even if the compiler
# chokes on \N escapes

test_named_sequences_full() checks that unicodedata.lookup() works,


OK, that change would then have checked unicodedata.lookup() twice.

However I'm puzzled by the fact that the "\N{}" escape sequence is 
supposed to raise a SyntaxError. And indeed it does in some cases:


Python 3.8.5 (default, Jul 21 2020, 10:48:26)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

import unicodedata
unicodedata.lookup("DIGIT ZERO")

'0'

"\N{DIGIT ZERO}"

'0'

"\N{EURO SIGN}"

'€'

unicodedata.lookup("EURO SIGN")

'€'

unicodedata.lookup("KEYCAP NUMBER SIGN")

'#️⃣'

"\N{KEYCAP NUMBER SIGN}"

  File "", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in 
position 0-21: unknown Unicode character name

unicodedata.lookup("LATIN CAPITAL LETTER A WITH MACRON AND GRAVE")

'Ā̀'

"\N{LATIN CAPITAL LETTER A WITH MACRON AND GRAVE}"

  File "", line 1
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in 
position 0-47: unknown Unicode character name


It seems that unicodedata.lookup() honors "Code point sequences", but 
\N{} does not.


Indeed 
https://docs.python.org/3/library/unicodedata.html#unicodedata.lookup

mentions that fact:

   Changed in version 3.3: Support for name aliases and named sequences 
has been added.


https://docs.python.org/3/reference/lexical_analysis.html#string-and-bytes-literals

doesn't mention anything. It simply states

Escape Sequence: \N{name}
Meaning: Character named name in the Unicode database

with the footnote "Changed in version 3.3: Support for name aliases has 
been added.".


Which leads to the question:

Should \N{} be updated to support "Code point sequences"?

Furthermore it states: "Unlike Standard C, all unrecognized escape 
sequences are left in the string unchanged", which could be interpreted 
as meaning that "\N{BAD}" results in "\\N{BAD}".



but that checkletter() raises a SyntaxError. Look at the code ;-)


That would have helped. ;)


Victor


Servus,
   Walter
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RNZUBXZ3WGIQ57CONGFEVEPM4NFS5CWW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Resignation from Stefan Krah

[Antoine Pitrou]

Hello,

Apparently, Stefan Krah (core developer and author of the C _decimal
module) was silently banned or moderated from posting to python.org
mailing-lists.  He asked me to forward the following message:


==
Hello,

Today I have left the Python organization.  It wasn't an easy decision,
after all there are so many amazing people here.

My vision of how development should be handled differs from many people
who are currently active.  Other projects are more aligned with my
preferences, so I prefer to focus my energies elsewhere.

Having a shared understanding of what constitutes politeness is
important and eliminates all sources of friction that sometimes result
in losing one's patience.

All the best,

Stefan Krah



Best regards

Antoine.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZIAN7ERZNF4ZE2B2SLYNRPVNERNACG5A/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Resignation from Stefan Krah

[Hasan Diwan]

Sorry to see you go, Stefan. You will be missed. -- H

On Wed, 7 Oct 2020 at 14:50, Antoine Pitrou  wrote:

>
> Hello,
>
> Apparently, Stefan Krah (core developer and author of the C _decimal
> module) was silently banned or moderated from posting to python.org
> mailing-lists.  He asked me to forward the following message:
>
>
>
> ==
> Hello,
>
> Today I have left the Python organization.  It wasn't an easy decision,
> after all there are so many amazing people here.
>
> My vision of how development should be handled differs from many people
> who are currently active.  Other projects are more aligned with my
> preferences, so I prefer to focus my energies elsewhere.
>
> Having a shared understanding of what constitutes politeness is
> important and eliminates all sources of friction that sometimes result
> in losing one's patience.
>
> All the best,
>
> Stefan Krah
>
> 
>
>
> Best regards
>
> Antoine.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/ZIAN7ERZNF4ZE2B2SLYNRPVNERNACG5A/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
OpenPGP:
https://sks-keyservers.net/pks/lookup?op=get&search=0xFEBAD7FFD041BBA1
If you wish to request my time, please do so using
*bit.ly/hd1AppointmentRequest
*.
Si vous voudrais faire connnaisance, allez a *bit.ly/hd1AppointmentRequest
*.

Sent
from my mobile device
Envoye de mon portable
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6XYZV2MYBUFR5CQG6VQQTMLRUIF6E32N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 617 -- New PEG parser for CPython

[Terry Reedy]

On 10/6/2020 2:02 PM, Guido van Rossum wrote:
That's appreciated, but I think what's needed more is someone who 
actually wants to undertake this project. It's not just a matter of 
running a small script for hours -- someone will have to come up with a 
way to fuzz that is actually useful for this particular situation 
(inserting random characters in files isn't going to be very effective). 


Changes should be by token or broader grammatical construct.  However, 
the real difficulty in auto testing is the lack of a decision mechanism 
for correct output (code object versus SyntaxError) other than the tests 
being either designed or checked by parser experts.


Classical fuzzing looks for some clearly wrong -- a crash -- rather than 
an answer *or* an Exception.  So yes, random fuzzing that does not pass 
known limits could be done to look for crashes.  But this is different 
from raising SyntaxError to reject wrong programs.


Consider unary prefix operators:

*a is a SyntaxError, because the grammar circumscribes the use of '*' as 
prefix.


-'' is not, which might surprise some, but I presume the error not being 
caught until runtime, as a TypeError, is correct for the grammar as 
written.  Or did I just discover a parser bug?  Or a possible grammar 
improvement?
(In other words, if I, even with my experience, tried grammar/parser 
fuzzing, I might be as much a nuisance as a help.)


It would not necessarily be a regression if the grammar and parser were 
changed so that an obvious error like "- " were to be 
caught as a SyntaxError.




"(One area we have not explored extensively is rejection of all
wrong programs. 


I consider false rejection to be a bigger sin than false acceptance. 
Wrong programs, like "-''", are likely to fail at runtime anyway.  So 
one could test acceptance of randomly generated correct but not 
ridiculously big programs.  But I guess compiling the stdlib and other 
packages already pretty well covered this.



 We have unit tests that check for a certain number
of explicit rejections, but more work could be done, e.g. by using a
fuzzer that inserts random subtle bugs into existing code. We're
open to help in this area.)"
https://www.python.org/dev/peps/pep-0617/#validation



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FNMPQUDPZTX7E4CAGDENNFU6AQJJMW34/
Code of Conduct: http://python.org/psf/codeofconduct/