Re: [Python-Dev] PEP 572 and assert

On Wed, Jul 18, 2018 at 03:41:08AM +1000, Steven D'Aprano wrote: > Besides, there is a legitimate use for assignment expressions in > assertions: > > assert (spam := something) > 2, 'not enough spam' > assert sequence[foo] == 999, 'sequence item isn't 999' Ah, obviously the index in

Re: [Python-Dev] Fuzzing the Python standard library

In many languages numeric types can't hold arbitrarily large values, and I for one hadn't really previously recognized that if you read in a numeric value with an exponent that it would be represented *exactly* in memory (and thus one object with a very compact representation can take up huge

Re: [Python-Dev] Fuzzing the Python standard library

On Tue, Jul 17, 2018 at 9:44 AM, Jussi Judin wrote: > * Exceptions that are something else than the documented ones. These usually > indicate an internal implementation issue. For example one would not expect > an UnicodeDecodeError from netrc.netrc() function when the documentation[3] >

Re: [Python-Dev] Fuzzing the Python standard library

On Tue, Jul 17, 2018 at 4:57 PM Jussi Judin wrote: > Quick answer: undocumented billion laughs/exponential entity expansion > type of an attack that is accessible through web through any library that > uses fractions module to parse user input (that are actually available on > Github). > Are

Re: [Python-Dev] Fuzzing the Python standard library

Quick answer: undocumented billion laughs/exponential entity expansion type of an attack that is accessible through web through any library that uses fractions module to parse user input (that are actually available on Github). Could be mitigated by explicitly mentioning this in documentation,

Re: [Python-Dev] Question about PEP 484

Hi Sebastian, Both Teddy (cc'd) and I would like to volunteer to help. We're excited about the prospect of an informational pyi PEP. Best, Rebecca On Tue, Jul 17, 2018 at 10:42 AM 'Adam Cataldo' via pytype < pyt...@googlegroups.com> wrote: > Hi Sebastian, > > Of course, we'd be happy to work

Re: [Python-Dev] 2to3 for python annotations

On Tue, Jul 17, 2018 at 1:17 PM, Jelle Zijlstra wrote: > > > 2018-07-17 12:37 GMT-07:00 Philippe Fremy : > >> Hi, >> >> While contributing to pyannotate, I became familiar enough with 2to3 >> fixers to be able to convert Python 2 style annotations to Python 3. >> >> Is it something that would be

Re: [Python-Dev] 2to3 for python annotations

2018-07-17 12:37 GMT-07:00 Philippe Fremy : > Hi, > > While contributing to pyannotate, I became familiar enough with 2to3 > fixers to be able to convert Python 2 style annotations to Python 3. > > Is it something that would be interesting to put into python 2to3 ? If > so I would propose a PR

[Python-Dev] 2to3 for python annotations

Hi, While contributing to pyannotate, I became familiar enough with 2to3 fixers to be able to convert Python 2 style annotations to Python 3. Is it something that would be interesting to put into python 2to3 ? If so I would propose a PR for this. Cheers, Philippe

Re: [Python-Dev] Const access to CPython objects outside of GIL?

[Radim Řehůřek ] > Thanks Tim. That's one unambiguous answer. I'm glad that part came across ;-) > I did consider posting to python-list, but this seemed somewhat > python-devish. > I agree it's on the margin - I don't mind. Any appetite for documenting which foundational functions are

Re: [Python-Dev] PEP 572 and assert

On Jul 17, 2018, at 12:18, MRAB wrote: > Why use len(...) == 0 instead of not(...)? > >assert not(subdirs := list(path.iterdir())), subdirs Off-topic, but it’s a style thing. Some style guides (and my personal preference) require to be more explicit when checking for empty sequences.

Re: [Python-Dev] PEP 572 and assert

On 2018-07-17 19:34, Tim Peters wrote: [Barry Warsaw] Thanks!  I thought it was cute.  It was just something that occurred to me as I was reviewing some existing code.  The intent wasn’t to use `subdirs` outside of the assert statement, but I’m warm to it because it means I

Re: [Python-Dev] PEP 572 and assert

On Jul 17, 2018, at 11:34, Tim Peters wrote: > Assuming the result of list(path.iterdir()) can change over time (seems very > likely), > >assert len(list(path.iterdir())) == 0, list(path.iterdir()) > > _could_ end up both triggering and displaying an empty list in the exception >

Re: [Python-Dev] PEP 572 and assert

[Barry Warsaw] > Thanks! I thought it was cute. It was just something that occurred to me > as I was reviewing some existing code. The intent wasn’t to use `subdirs` > outside of the assert statement, but I’m warm to it because it means I > don’t have to do wasted work outside of the assert

Re: [Python-Dev] PEP 572 and assert

On Jul 17, 2018, at 07:59, Guido van Rossum wrote: > > Personally I like Barry's example just fine -- assuming `subdirs` is not used > later, this feels like a good use case. Thanks! I thought it was cute. It was just something that occurred to me as I was reviewing some existing code. The

Re: [Python-Dev] Question about PEP 484

Great Jelle! We look forward to working with you to. On Tue, Jul 17, 2018 at 10:41 AM Jelle Zijlstra wrote: > > > 2018-07-17 9:55 GMT-07:00 Sebastian Rittau : > >> On 17.07.2018 17:05, Guido van Rossum wrote: >> >>> This is a good point. I presume specifying this unambiguously would be a >>>

Re: [Python-Dev] Question about PEP 484

Hi Sebastian, Of course, we'd be happy to work with you on this! We just need to figure out which of us will drive this on our end (most likely Rebecca or Teddy). I'll huddle with the team and get back to you with an answer on who later today. On Tue, Jul 17, 2018 at 9:58 AM Sebastian Rittau

Re: [Python-Dev] PEP 572 and assert

On Tue, Jul 17, 2018 at 07:24:12PM +0300, Serhiy Storchaka wrote: > 17.07.18 18:48, Guido van Rossum пише: > >On Tue, Jul 17, 2018 at 8:28 AM, Serhiy Storchaka >> wrote: > >Should not the assert statement introduce an implicit lexical scope > >for preventing

Re: [Python-Dev] Question about PEP 484

2018-07-17 9:55 GMT-07:00 Sebastian Rittau : > On 17.07.2018 17:05, Guido van Rossum wrote: > >> This is a good point. I presume specifying this unambiguously would be a >> huge amount of work, and it would mostly codify mypy's current behavior. I >> don't think that's within the scope of PEP

Re: [Python-Dev] Fuzzing the Python standard library

I'm not a core Python Dev, but quick question, why would you expect " fractions.Fraction("1.64E664644")" not to take 100s of megabytes and hours to run? Simply evaluating: 164 * 10**66464 will take hundreds of megabytes by definition. Regards Damian On Tue, Jul 17, 2018, 12:54 Jussi

Re: [Python-Dev] PEP 572 and assert

On Tue, Jul 17, 2018 at 9:24 AM, Serhiy Storchaka wrote: > 17.07.18 18:48, Guido van Rossum пише: > >> On Tue, Jul 17, 2018 at 8:28 AM, Serhiy Storchaka > > wrote: >> Should not the assert statement introduce an implicit lexical scope >> for preventing leaking

Re: [Python-Dev] Question about PEP 484

On 17.07.2018 17:05, Guido van Rossum wrote: This is a good point. I presume specifying this unambiguously would be a huge amount of work, and it would mostly codify mypy's current behavior. I don't think that's within the scope of PEP 484, but it could well be done as a separate PEP (perhaps

[Python-Dev] Fuzzing the Python standard library

Hi, I have been fuzzing[1] various parts of Python standard library for Python 3.7 with python-afl[2] to find out internal implementation issues that exist in the library. What I have been looking for are mainly following: * Exceptions that are something else than the documented ones. These

Re: [Python-Dev] Const access to CPython objects outside of GIL?

Thanks Tim. That's one unambiguous answer. I did consider posting to python-list, but this seemed somewhat python-devish. Any appetite for documenting which foundational functions are const-safe in the sense I described? Perhaps we could help. What kind of analysis, demand or momentum is needed

Re: [Python-Dev] Const access to CPython objects outside of GIL?

[Tim] > > Same as the docs, I use "Python object" to mean a pointer to PyObject. > In > that sense, a Py_buffer is no more a "Python object" than, e.g,, is a > > Py_ssize_t. > [Antoine > Yes, and the payload of an int object is not a "Python object". > The OP mentioned PyInt_AS_LONG as an

Re: [Python-Dev] PEP 572 and assert

On Wed, Jul 18, 2018 at 2:24 AM, Serhiy Storchaka wrote: > 17.07.18 18:48, Guido van Rossum пише: >> >> On Tue, Jul 17, 2018 at 8:28 AM, Serhiy Storchaka > > wrote: >> Should not the assert statement introduce an implicit lexical scope >> for preventing leaking

Re: [Python-Dev] PEP 572 and assert

17.07.18 19:25, Tim Peters пише: It is a win (to my eyes) if the code it replaced was     if __debug__:         subdirs = list(path.iterdir())         assert len(subdirs) == 0, subdirs in which case the semantics are the same in either spelling, with or without -O, but the spelling at the

Re: [Python-Dev] PEP 572 and assert

[Serhiy Storchaka] > Recently Barry shown an example: > > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > It looks awful to me. It looks even worse than using asserts for > validating the user input. The assert has a side effect, and it depends > on the interpreter option (-O).

Re: [Python-Dev] PEP 572 and assert

17.07.18 18:48, Guido van Rossum пише: On Tue, Jul 17, 2018 at 8:28 AM, Serhiy Storchaka > wrote: Should not the assert statement introduce an implicit lexical scope for preventing leaking variables? I don't see why. As Chris said, side effects in asserts are

Re: [Python-Dev] PEP 572 and assert

> > I don't see why. As Chris said, side effects in asserts are nothing new > and > Indeed -- this new feature makes it easier to affect the local scope in all sorts of new places. It was decided that the additional complexity is worth it to make the language more expressive, and it was also

Re: [Python-Dev] Const access to CPython objects outside of GIL?

On Tue, 17 Jul 2018 10:59:02 -0500 Tim Peters wrote: > > Same as the docs, I use "Python object" to mean a pointer to PyObject. In > that sense, a Py_buffer is no more a "Python object" than, e.g,, is a > Py_ssize_t. Yes, and the payload of an int object is not a "Python object". The OP

Re: [Python-Dev] Const access to CPython objects outside of GIL?

On Tue, Jul 17, 2018 at 4:34 AM, Victor Stinner wrote: > > IMHO you need a different approach to implement optimizations. For > example, use your objects which don't rely on the GIL to be > consistent. Sadly, I have no experience with that and so cannot > provide any example. > I DO NOT

Re: [Python-Dev] Const access to CPython objects outside of GIL?

[Tim]> There is no intention to support GIL-free access to any Python objects. So > > that's the contract: "All warranties are null & void if you do just > about > > anything while not holding the GIL". > [Antoine] > Actually, accessing a Py_buffer that's been obtained in the regular way

Re: [Python-Dev] PEP 572 and assert

On Tue, Jul 17, 2018 at 8:28 AM, Serhiy Storchaka wrote: > 17.07.18 17:59, Guido van Rossum пише: > >> The PEP has no specific opinion except it is not forbidden. >> >> Personally I like Barry's example just fine -- assuming `subdirs` is not >> used later, this feels like a good use case. >> > >

Re: [Python-Dev] PEP 572 and assert

17.07.18 17:59, Guido van Rossum пише: The PEP has no specific opinion except it is not forbidden. Personally I like Barry's example just fine -- assuming `subdirs` is not used later, this feels like a good use case. Shouldn't this problem be solved in the same way as for comprehensions?

Re: [Python-Dev] Const access to CPython objects outside of GIL?

On 17.07.2018 7:18, Radim Řehůřek wrote: Hi all, one of our Python projects calls for pretty heavy, low-level optimizations. We went down the rabbit hole and determined that having access to PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict, unicode) on Python objects

Re: [Python-Dev] Question about PEP 484

This is a good point. I presume specifying this unambiguously would be a huge amount of work, and it would mostly codify mypy's current behavior. I don't think that's within the scope of PEP 484, but it could well be done as a separate PEP (perhaps an informational one?). I hope you understand

Re: [Python-Dev] PEP 572 and assert

On Tue, Jul 17, 2018 at 1:50 AM, Serhiy Storchaka wrote: > Recently Barry shown an example: > > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > It looks awful to me. It looks even worse than using asserts for > validating the user input. The assert has a side effect, and it

Re: [Python-Dev] Const access to CPython objects outside of GIL?

2018-07-17 6:18 GMT+02:00 Radim Řehůřek : > one of our Python projects calls for pretty heavy, low-level optimizations. > > We went down the rabbit hole and determined that having access to > PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict, unicode) > on Python objects **outside

Re: [Python-Dev] PEP 572 and assert

2018-07-17 10:50 GMT+02:00 Serhiy Storchaka : > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > Does PEP 572 encourages writing such code, discourages this, or completely > forbids? If I understood correctly Guido, Python the language must not prevent developers to experiment

Re: [Python-Dev] Question about PEP 484

On 16.07.2018 19:21, Adam Cataldo via Python-Dev wrote: * * One thing we care about in particular, given the implementation of pytype, is the detailed definition of what goes in a .pyi file. Do folks think this makes sense to include as part of PEP 484, or would this be better

Re: [Python-Dev] PEP 572 and assert

On Tue, 17 Jul 2018 18:56:36 +1000 Chris Angelico wrote: > On Tue, Jul 17, 2018 at 6:50 PM, Serhiy Storchaka wrote: > > Recently Barry shown an example: > > > > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > > > It looks awful to me. It looks even worse than using asserts for

Re: [Python-Dev] PEP 572 and assert

On Tue, Jul 17, 2018 at 6:50 PM, Serhiy Storchaka wrote: > Recently Barry shown an example: > > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > It looks awful to me. It looks even worse than using asserts for validating > the user input. The assert has a side effect, and it

[Python-Dev] PEP 572 and assert

Recently Barry shown an example: assert len(subdirs := list(path.iterdir())) == 0, subdirs It looks awful to me. It looks even worse than using asserts for validating the user input. The assert has a side effect, and it depends on the interpreter option (-O). Even if subdirs is not used

Re: [Python-Dev] Const access to CPython objects outside of GIL?

On Tue, 17 Jul 2018 00:15:04 -0500 Tim Peters wrote: > > More generally, what is the CPython 2.7/3.5 contract regarding (lack of) > > object mutation, and the need for reference counting and synchronization > > via GIL? > > There is no intention to support GIL-free access to any Python objects.