Re: [Python-Dev] Naming comprehension syntax [was Re: Informal educator feedback on PEP 572 ...]
On Mon, Jul 9, 2018 at 3:18 PM, Guido van Rossum wrote: > Definitely docs first. And we should keep references to generator > expressions too, if only to explain that they've been renamed. > > Perhaps someone should try it out in a 3rd party tutorial to see how it > goes? > I'm not sure what "trying it out in a tutorial" would look like. I try to be pretty clear about terminology when I teach newbies -- so I don't want to tell anyone this new thing is called a "generator comprehension" if they aren't going to see that term anywhere else. -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR(206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception chris.bar...@noaa.gov ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
On Fri, Jul 13, 2018 at 5:03 AM André Malo wrote: > > * INADA Naoki wrote: > > > Is there any real application which marshal.dumps() performance is > > critical? > > I'm using it for spooling big chunks of data on disk, exactly for the reason > that it's faster than pickle. > > Cheers, Does your data contains repetition of same object (not same value)? If yes, this change will affects you. If no, you can use older version which doesn't have overhead of checking object identity. >>> x = [0]*100 >>> y = [0]*100 >>> data = [x,y,x] >>> import marshal >>> len(marshal.dumps(data)) # x is marshaled once 1020 >>> d[0] is d[2] True >>> d[0] is d[1] False >>> import json >>> len(json.dumps(data)) # x is marshaled twice 906 >>> d = marshal.loads(marshal.dumps(data, 2)) # x is marshaled twice >>> len(d) 1520 >>> d[0] is d[2] False -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On 07/12/2018 03:03 PM, Abdur-Rahmaan Janhangeer wrote: btw smileys in this thread should have been :=) lol! -- ~Ethan~ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
ok then was a point i wanted to clear btw smileys in this thread should have been :=) Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
*D in BDFL stands for Dictator. * The B diminishes that ex sugar not same as salty sugar Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
On Thu, 12 Jul 2018 22:03:30 +0200 André Malo wrote: > * INADA Naoki wrote: > > > Is there any real application which marshal.dumps() performance is > > critical? > > I'm using it for spooling big chunks of data on disk, exactly for the reason > that it's faster than pickle. Which kind of data is that? Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
* INADA Naoki wrote: > Is there any real application which marshal.dumps() performance is > critical? I'm using it for spooling big chunks of data on disk, exactly for the reason that it's faster than pickle. Cheers, -- "Das Verhalten von Gates hatte mir bewiesen, dass ich auf ihn und seine beiden Gefährten nicht zu zählen brauchte" -- Karl May, "Winnetou III" ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On Thu, Jul 12, 2018 at 6:21 PM, Barry Warsaw wrote: > [...] > > I was -1 as well, but I’d say I’m a firm +0 now[*]. I like how many of > the problematic syntactic and semantic issues have been narrowed and > prohibited, and I can see myself using this sparingly. > [...] I think experience will show that's how it's best used - only for measurable wins. But then Python is the kind of language where "if it's simpler, use that" is a part of the philosophy, thanks to the Zen's popularity. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On Thu, Jul 12, 2018, 1:24 PM Barry Warsaw wrote: > It’s not the first time I’ve found myself in this position with a new > Python feature, and it’s one of the reasons I deeply trust Guido’s > intuition and sensibilities. > Sure... Except for the terrible choice to drop the '<>' inequality operator, and why all my scripts start with barry_as_FLUFL. :-) > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On Jul 12, 2018, at 09:23, INADA Naoki wrote: >> >> Yes, the PEP has improved significantly since that time. My guess is the >> same poll taken now could give an opposite result. >> > > I still -0 on PEP 572. But strong -1 on restart discussion about changing it. > We should polish and implement it for now, not change. I think that’s likely true. While extremely painful for so many of us, I think the end result is a much better PEP, and a much better feature. I was -1 as well, but I’d say I’m a firm +0 now[*]. I like how many of the problematic syntactic and semantic issues have been narrowed and prohibited, and I can see myself using this sparingly. It’s not the first time I’ve found myself in this position with a new Python feature, and it’s one of the reasons I deeply trust Guido’s intuition and sensibilities. Cheers, -Barry [*] Not that it matters; the PEP is accepted - time to move on! The world won’t end. :) signature.asc Description: Message signed with OpenPGP ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On Fri, Jul 13, 2018 at 12:48 AM Ivan Levkivskyi wrote: > > On 12 July 2018 at 16:41, Victor Stinner wrote: >> >> 2018-07-12 17:14 GMT+02:00 Abdur-Rahmaan Janhangeer : >> > sorry for reviving the dead but community acceptance, a fundamental pep >> > principle has not been respected for 572 >> > >> > also 29 core devs dislike vs 3 like >> >> [...] *as the PEP evolved* in the meanwhile. > > > Yes, the PEP has improved significantly since that time. My guess is the same > poll taken now could give an opposite result. > I still -0 on PEP 572. But strong -1 on restart discussion about changing it. We should polish and implement it for now, not change. -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
On 12 July 2018 at 16:41, Victor Stinner wrote: > 2018-07-12 17:14 GMT+02:00 Abdur-Rahmaan Janhangeer >: > > sorry for reviving the dead but community acceptance, a fundamental pep > > principle has not been respected for 572 > > > > also 29 core devs dislike vs 3 like > > [...] *as the PEP evolved* in the meanwhile. Yes, the PEP has improved significantly since that time. My guess is the same poll taken now could give an opposite result. -- Ivan ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 572, Assignment Expressions
Hooray! I could be a happy python oneliner now! Greetings. ZHUO QL (KDr2, http://kdr2.com) On Thursday, July 12, 2018, 8:12:54 AM GMT+8, Guido van Rossum wrote: As anticippated, after a final round of feedback I am hereby accepting PEP 572, Assignment Expressions: https://www.python.org/dev/ peps/pep-0572/ Thanks to everyone who participated in the discussion or sent a PR. Below is a list of changes since the last post (https://mail.python.org/ pipermail/python-dev/2018- July/154557.html) -- they are mostly cosmetic so I won't post the doc again, but if you want to go over them in detail, here's the history of the file on GitHub: https://github.com/python/ peps/commits/master/pep-0572. rst, and here's a diff since the last posting: https://github.com/python/ peps/compare/26e6f61f...master (sadly it's repo-wide -- you can click on Files changed and then navigate to pep-0572.rst). - Tweaked the example at line 95-100 to use result = ... rather than return ... so as to make a different rewrite less feasible - Replaced the weak "2-arg iter" example with Giampaolo Roloda's while chunk := file.read(8192): process(chunk) - Added prohibition of unparenthesized assignment expressions in annotations and lambdas - Clarified that TargetScopeError is a new subclass of SyntaxError - Clarified the text forbidding assignment to comprehension loop control variables - Clarified that the prohibition on := with annotation applies to inline annotation (i.e. they cannot be syntactically combined in the same expression) - Added conditional expressions to the things := binds less tightly than - Dropped section "This could be used to create ugly code" - Clarified the example in Appendix C Now on to the implementation work! (Maybe I'll sprint on this at the core-dev sprint in September.) -- --Guido van Rossum (python.org/~guido)___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/zhuoql%40yahoo.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
2018-07-12 17:14 GMT+02:00 Abdur-Rahmaan Janhangeer : > sorry for reviving the dead but community acceptance, a fundamental pep > principle has not been respected for 572 > > also 29 core devs dislike vs 3 like You are referring to a *poll* that I ran in May. I don't see any community issue, the PEP process was always the same: Guido van Rossum takes the final decision. IMHO since the poll has be done, some core devs changed their mind, *as the PEP evolved* in the meanwhile. For example, I'm more on the +1 side now (I was on the strong -1 side when I ran the poll). Anyway, it no longer matters since the PEP has been approved. It's now time to celebrate! > maybe there are special cases where the BDFL can pin issues D in BDFL stands for Dictator. Guido has super power :-) I'm fine with that. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?
sorry for reviving the dead but community acceptance, a fundamental pep principle has not been respected for 572 also 29 core devs dislike vs 3 like maybe there are special cases where the BDFL can pin issues also, maybe there are two aspects, one disliking := and one the actual expression assignment as for me i don't like the := symbol Abdur-Rahmaan Janhangeer https://github.com/Abdur-rahmaanJ > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
On Thu, 12 Jul 2018 09:21:55 +0300 Serhiy Storchaka wrote: > > > Is there any real application which marshal.dumps() performance is > > critical? > EVE Online is a well known example. > > What if write a script which loads .pyc files and stabilize them? This > could solve the problem for applications which need stable .pyc files, > with zero impact on common use. Should python-dev maintain that script? If yes, it sounds better to make marshal itself deterministic. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
Eve is indeed based on stackless 2, and are well capable of ignoring changes they don't think they need (or were when I was working with them). At one point I seem to remember they optimised their interpreter to use singleton floating-point values, saving large quantities of memory by having only one floating-point zero. Steve Holden On Thu, Jul 12, 2018 at 9:55 AM, Alex Walters wrote: > > > > -Original Message- > > From: Python-Dev > list=sdamon@python.org> On Behalf Of Victor Stinner > > Sent: Thursday, July 12, 2018 4:01 AM > > To: Serhiy Storchaka > > Cc: python-dev > > Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler? > > > > 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka : > > >> Is there any real application which marshal.dumps() performance is > > >> critical? > > > > > > EVE Online is a well known example. > > > > EVE Online has been created in 2003. I guess that it still uses Python > 2.7. > > > > I'm not sure that a video game would pick marshal in 2018. > > > > EVE doesn't use stock CPython, IIRC. They use a version of stackless 2, > with their own patches. If a company is willing to patch python itself, I > don't think their practices should be cited without more context about what > they actually modified. > > > Victor > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium- > > list%40sdamon.com > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > steve%40holdenweb.com > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 572, Assignment Expressions
(status := "Accepted") and "Congratulations!" ;-) (hope I did that right, but I can't try it yet!) Thanks for hanging in there, Guido, and for your patience with everyone during the discussions. I'm glad you're still with us! --Chris On Wed, Jul 11, 2018 at 5:10 PM, Guido van Rossum wrote: > As anticippated, after a final round of feedback I am hereby accepting PEP > 572, Assignment Expressions: https://www.python.org/dev/peps/pep-0572/ > > Thanks to everyone who participated in the discussion or sent a PR. > > Below is a list of changes since the last post ( > https://mail.python.org/pipermail/python-dev/2018-July/154557.html) -- > they are mostly cosmetic so I won't post the doc again, but if you want to > go over them in detail, here's the history of the file on GitHub: > https://github.com/python/peps/commits/master/pep-0572.rst, and here's a > diff since the last posting: https://github.com/python/peps > /compare/26e6f61f...master (sadly it's repo-wide -- you can click on > Files changed and then navigate to pep-0572.rst). > >- Tweaked the example at line 95-100 to use result = ... rather than return >... so as to make a different rewrite less feasible >- Replaced the weak "2-arg iter" example with Giampaolo Roloda's while >chunk := file.read(8192): process(chunk) >- *Added prohibition of unparenthesized assignment expressions in >annotations and lambdas* >- Clarified that TargetScopeError is a *new* subclass of SyntaxError >- Clarified the text forbidding assignment to comprehension loop >control variables >- Clarified that the prohibition on := with annotation applies to >*inline* annotation (i.e. they cannot be syntactically combined in the >same expression) >- Added conditional expressions to the things := binds less tightly >than >- Dropped section "This could be used to create ugly code" >- Clarified the example in Appendix C > > Now on to the implementation work! (Maybe I'll sprint on this at the > core-dev sprint in September.) > > -- > --Guido van Rossum (python.org/~guido) > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > chris.jerdonek%40gmail.com > > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
> -Original Message- > From: Python-Dev list=sdamon@python.org> On Behalf Of Victor Stinner > Sent: Thursday, July 12, 2018 4:01 AM > To: Serhiy Storchaka > Cc: python-dev > Subject: Re: [Python-Dev] Can I make marshal.dumps() slower but stabler? > > 2018-07-12 8:21 GMT+02:00 Serhiy Storchaka : > >> Is there any real application which marshal.dumps() performance is > >> critical? > > > > EVE Online is a well known example. > > EVE Online has been created in 2003. I guess that it still uses Python 2.7. > > I'm not sure that a video game would pick marshal in 2018. > EVE doesn't use stock CPython, IIRC. They use a version of stackless 2, with their own patches. If a company is willing to patch python itself, I don't think their practices should be cited without more context about what they actually modified. > Victor > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/tritium- > list%40sdamon.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
On Thu, Jul 12, 2018 at 3:22 PM Serhiy Storchaka wrote: > > 12.07.18 08:43, INADA Naoki пише: > > I'm working on making pyc stable, via stablizing marshal.dumps() > > https://bugs.python.org/issue34093 > > This is not enough for making pyc stable. The order in frozesets still > is arbitrary. But we can use PYTHONHASHSEED to make pyc stable. Currently, refcnt is the only known issue for reproducible pyc build. > > > Sadly, it makes marshal.dumps() 40% slower. > > Luckily, this overhead is small (only 4%) for dumps(compile(source)) case. > > What about the memory consumption? No overhead, because we already used same hashtable for w_ref. I just make it two-pass, instead of one-pass. > > > So my question is: May I remove unstable but faster code? > > > > Or should I make this optional and we maintain two complex code? > > If so, should this option enabled by default or not? > > My concern is that even if not make it optional, this will complicate > the code. When it's not optional, it makes almost duplicate of w_object for reference counting in object tree. https://github.com/python/cpython/pull/8226/commits/e170116e80dfd27f923c88fc11e42f0d6f687a00 > > > For example, xmlrpc uses marshal. But xmlrpc has significant overhead > > other than marshaling, like dumps(compile(source)) case. So I expect > > marshal.dumps() performance is not critical for it too. > > xmlrpc doesn't use the marshal module. It uses terms marshalling and > unmarshalling, but in different meaning. > Oh, I just grepped and misunderstood. > > Is there any real application which marshal.dumps() performance is critical? > EVE Online is a well known example. > Do they use version>=3? In version 3, FLAG_REF is introduced and it made significant runtime overhead already. If marshaling speed is very important, version<2 should be used. > What if write a script which loads .pyc files and stabilize them? This > could solve the problem for applications which need stable .pyc files, > with zero impact on common use. > Hmm, do you mean which?: * Adding marshal.dump_stable_pyc() and use it like `marshal.dump_stable_pyc(marshal.loads(code))` * Implementing pure Python marshal.dumps in distutils -- INADA Naoki ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
2018-07-12 8:21 GMT+02:00 Serhiy Storchaka : >> Is there any real application which marshal.dumps() performance is >> critical? > > EVE Online is a well known example. EVE Online has been created in 2003. I guess that it still uses Python 2.7. I'm not sure that a video game would pick marshal in 2018. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?
12.07.18 08:43, INADA Naoki пише: I'm working on making pyc stable, via stablizing marshal.dumps() https://bugs.python.org/issue34093 This is not enough for making pyc stable. The order in frozesets still is arbitrary. Sadly, it makes marshal.dumps() 40% slower. Luckily, this overhead is small (only 4%) for dumps(compile(source)) case. What about the memory consumption? So my question is: May I remove unstable but faster code? Or should I make this optional and we maintain two complex code? If so, should this option enabled by default or not? My concern is that even if not make it optional, this will complicate the code. For example, xmlrpc uses marshal. But xmlrpc has significant overhead other than marshaling, like dumps(compile(source)) case. So I expect marshal.dumps() performance is not critical for it too. xmlrpc doesn't use the marshal module. It uses terms marshalling and unmarshalling, but in different meaning. Is there any real application which marshal.dumps() performance is critical? EVE Online is a well known example. What if write a script which loads .pyc files and stabilize them? This could solve the problem for applications which need stable .pyc files, with zero impact on common use. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
