Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
2016-09-13 18:36 GMT+02:00 Victor Stinner : > Ok, it's start listing regressions/major issues :-) > > * Bug in _PyDict_Pop() on a splitted table: > http://bugs.python.org/issue28120 -- bug in the new compact dict > implementation A new one: a crash in os.execve() and os.spawnve(): http://bugs.python.org/issue28114 It was reported by Windows users but also Twisted (test suite) on Linux. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Python parser performance optimizations
OK, but if nobody responds within a week we should close it. IMO there's no value in keeping things around that nobody is going to apply. I don't expect that a year from now we'll suddenly a surge of interest in this patch, sorry. On Fri, Sep 16, 2016 at 4:25 AM, Artyom Skrobov wrote: > Thank you very much for your comments, > > I appreciate that we're all volunteers, and that if nobody fancies > reviewing a big invasive patch, then it won't get reviewed. > > Still, I want to note that the suggested optimization has a noticeable > positive effect on many benchmarks -- even though the effect may only > become of practical value in such uncommon use cases as parsing huge data > tables. > > As I found out later, JSON wasn't a viable option for storing dozens of > megabytes of deeply-nested data, either. To get acceptable deserialization > performance, I eventually had to resort to pickled files. > > > -Original Message- > From: gvanros...@gmail.com [mailto:gvanros...@gmail.com] On Behalf Of > Guido van Rossum > Sent: 15 September 2016 17:01 > To: Artyom Skrobov > Cc: python-dev@python.org; br...@python.org; jimjjew...@gmail.com; nd > Subject: Re: [Python-Dev] Python parser performance optimizations > > I wonder if this patch could just be rejected instead of lingering > forever? It clearly has no champion among the current core devs and > therefore it won't be included in Python 3.6 (we're all volunteers so > that's how it goes). > > The use case for the patch is also debatable: Python's parser wasn't > designed to *efficiently* parse huge data tables like that, and if you > have that much data, using JSON is the right answer. So this doesn't > really scratch anyone's itch except of the patch author (Artyom). > > From a quick look it seems the patch is very disruptive in terms of > what it changes, so it's not easy to review. > > I recommend giving up, closing the issue as "won't fix", recommending > to use JSON, and moving on. Sometimes a change is just not worth the > effort. > > --Guido > > On Tue, Aug 9, 2016 at 1:59 AM, Artyom Skrobov > wrote: > > Hello, > > > > > > > > This is a monthly ping to get a review on http://bugs.python.org/ > issue26415 > > -- “Excessive peak memory consumption by the Python parser”. > > > > > > > > Following the comments from July, the patches now include updating > Misc/NEWS > > and compiler.rst to describe the change. > > > > > > > > The code change itself is still the same as a month ago. > > > > > > > > > > > > From: Artyom Skrobov > > Sent: 07 July 2016 15:44 > > To: python-dev@python.org; st...@pearwood.info; > mafagafogiga...@gmail.com; > > greg.ew...@canterbury.ac.nz > > Cc: nd > > Subject: RE: Python parser performance optimizations > > > > > > > > Hello, > > > > > > > > This is a monthly ping to get a review on http://bugs.python.org/ > issue26415 > > -- “Excessive peak memory consumption by the Python parser”. > > > > The first patch of the series (an NFC refactoring) was successfully > > committed earlier in June, so the next step is to get the second patch, > “the > > payload”, reviewed and committed. > > > > > > > > To address the concerns raised by the commenters back in May: the patch > > doesn’t lead to negative memory consumption, of course. The base for > > calculating percentages is the smaller number of the two; this is the > same > > style of reporting that perf.py uses. In other words, “200% less memory > > usage” is a threefold shrink. > > > > > > > > The absolute values, and the way they were produced, are all reported > under > > the ticket. > > > > > > > > > > > > From: Artyom Skrobov > > Sent: 26 May 2016 11:19 > > To: 'python-dev@python.org' > > Subject: Python parser performance optimizations > > > > > > > > Hello, > > > > > > > > Back in March, I’ve posted a patch at http://bugs.python.org/issue26526 > -- > > “In parsermodule.c, replace over 2KLOC of hand-crafted validation code, > with > > a DFA”. > > > > > > > > The motivation for this patch was to enable a memory footprint > optimization, > > discussed at http://bugs.python.org/issue26415 > > > > My proposed optimization reduces the memory footprint by up to 30% on the > > standard benchmarks, and by 200% on a degenerate case which sparked the > > discussion. > > > > The run time stays unaffected by this optimization. > > > > > > > > Python Developer’s Guide says: “If you don’t get a response within a few > > days after pinging the issue, then you can try emailing > > python-dev@python.org asking for someone to review your patch.” > > > > > > > > So, here I am. > > > > > > ___ > > Python-Dev mailing list > > Python-Dev@python.org > > https://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > https://mail.python.org/mailman/options/python-dev/guido%40python.org > > > > > > -- > --Guido van Rossum (python.org/~guido) > > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev ma
Re: [Python-Dev] Python parser performance optimizations
Thank you very much for your comments, I appreciate that we're all volunteers, and that if nobody fancies reviewing a big invasive patch, then it won't get reviewed. Still, I want to note that the suggested optimization has a noticeable positive effect on many benchmarks -- even though the effect may only become of practical value in such uncommon use cases as parsing huge data tables. As I found out later, JSON wasn't a viable option for storing dozens of megabytes of deeply-nested data, either. To get acceptable deserialization performance, I eventually had to resort to pickled files. -Original Message- From: gvanros...@gmail.com [mailto:gvanros...@gmail.com] On Behalf Of Guido van Rossum Sent: 15 September 2016 17:01 To: Artyom Skrobov Cc: python-dev@python.org; br...@python.org; jimjjew...@gmail.com; nd Subject: Re: [Python-Dev] Python parser performance optimizations I wonder if this patch could just be rejected instead of lingering forever? It clearly has no champion among the current core devs and therefore it won't be included in Python 3.6 (we're all volunteers so that's how it goes). The use case for the patch is also debatable: Python's parser wasn't designed to *efficiently* parse huge data tables like that, and if you have that much data, using JSON is the right answer. So this doesn't really scratch anyone's itch except of the patch author (Artyom). From a quick look it seems the patch is very disruptive in terms of what it changes, so it's not easy to review. I recommend giving up, closing the issue as "won't fix", recommending to use JSON, and moving on. Sometimes a change is just not worth the effort. --Guido On Tue, Aug 9, 2016 at 1:59 AM, Artyom Skrobov wrote: > Hello, > > > > This is a monthly ping to get a review on http://bugs.python.org/issue26415 > -- “Excessive peak memory consumption by the Python parser”. > > > > Following the comments from July, the patches now include updating Misc/NEWS > and compiler.rst to describe the change. > > > > The code change itself is still the same as a month ago. > > > > > > From: Artyom Skrobov > Sent: 07 July 2016 15:44 > To: python-dev@python.org; st...@pearwood.info; mafagafogiga...@gmail.com; > greg.ew...@canterbury.ac.nz > Cc: nd > Subject: RE: Python parser performance optimizations > > > > Hello, > > > > This is a monthly ping to get a review on http://bugs.python.org/issue26415 > -- “Excessive peak memory consumption by the Python parser”. > > The first patch of the series (an NFC refactoring) was successfully > committed earlier in June, so the next step is to get the second patch, “the > payload”, reviewed and committed. > > > > To address the concerns raised by the commenters back in May: the patch > doesn’t lead to negative memory consumption, of course. The base for > calculating percentages is the smaller number of the two; this is the same > style of reporting that perf.py uses. In other words, “200% less memory > usage” is a threefold shrink. > > > > The absolute values, and the way they were produced, are all reported under > the ticket. > > > > > > From: Artyom Skrobov > Sent: 26 May 2016 11:19 > To: 'python-dev@python.org' > Subject: Python parser performance optimizations > > > > Hello, > > > > Back in March, I’ve posted a patch at http://bugs.python.org/issue26526 -- > “In parsermodule.c, replace over 2KLOC of hand-crafted validation code, with > a DFA”. > > > > The motivation for this patch was to enable a memory footprint optimization, > discussed at http://bugs.python.org/issue26415 > > My proposed optimization reduces the memory footprint by up to 30% on the > standard benchmarks, and by 200% on a degenerate case which sparked the > discussion. > > The run time stays unaffected by this optimization. > > > > Python Developer’s Guide says: “If you don’t get a response within a few > days after pinging the issue, then you can try emailing > python-dev@python.org asking for someone to review your patch.” > > > > So, here I am. > > > ___ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2016-09-09 - 2016-09-16) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open5507 (-61) closed 34432 (+199) total 39939 (+138) Open issues with patches: 2367 Issues opened (82) == #15369: pybench and test.pystone poorly documented http://bugs.python.org/issue15369 reopened by haypo #21009: Potential deadlock in concurrent futures when garbage collecti http://bugs.python.org/issue21009 reopened by davin #22493: Deprecate the use of flags not at the start of regular express http://bugs.python.org/issue22493 reopened by serhiy.storchaka #24254: Make class definition namespace ordered by default http://bugs.python.org/issue24254 reopened by ncoghlan #25573: FrameSummary repr() does not support previously working uses o http://bugs.python.org/issue25573 reopened by r.david.murray #26502: traceback.extract_tb breaks compatibility by returning FrameSu http://bugs.python.org/issue26502 reopened by r.david.murray #28027: Remove Lib/plat-*/* files http://bugs.python.org/issue28027 reopened by doko #28046: Remove the concept of platform-specific directories http://bugs.python.org/issue28046 opened by zach.ware #28050: test_traceback is broken by new CALL_FUNCTION* opcodes http://bugs.python.org/issue28050 opened by haypo #28052: clarify concurrent.futures docs to not refer to async Futures http://bugs.python.org/issue28052 opened by davin #28053: parameterize what serialization is used in multiprocessing http://bugs.python.org/issue28053 opened by davin #28054: Diff for visually comparing actual with expected in mock.asser http://bugs.python.org/issue28054 opened by Eli Rose #28055: pyhash's siphash24 assumes alignment of the data pointer http://bugs.python.org/issue28055 opened by doko #28058: [Patch] Don't use st_uid and st_gid on CloudABI http://bugs.python.org/issue28058 opened by EdSchouten #28060: Clean up division fast paths in Objects/longobject.c http://bugs.python.org/issue28060 opened by mark.dickinson #28062: Streamline repr(partial object) http://bugs.python.org/issue28062 opened by ebarry #28068: Error in freeze.py due to unguarded sys.abiflags usage under W http://bugs.python.org/issue28068 opened by gevorg #28069: signalmodule.c does "is" comparisons for SIG_IGN and SIG_DFL http://bugs.python.org/issue28069 opened by mark.dickinson #28074: Add Configuration file parser action http://bugs.python.org/issue28074 opened by Chris Nyland #28075: os.stat fails when access is denied http://bugs.python.org/issue28075 opened by ramson #28080: Allow reading member names with bogus encodings in zipfile http://bugs.python.org/issue28080 opened by sjt #28082: re: convert re flags to (much friendlier) IntFlag constants http://bugs.python.org/issue28082 opened by ethan.furman #28083: socket: finish constant to Enum/Flag conversion http://bugs.python.org/issue28083 opened by ethan.furman #28085: SSL: Add client and server protocols for SSLContext http://bugs.python.org/issue28085 opened by christian.heimes #28086: test.test_getargs2.TupleSubclass test failure http://bugs.python.org/issue28086 opened by rbcollins #28087: macOS 12 poll syscall returns prematurely http://bugs.python.org/issue28087 opened by MicroTransactionsMatterToo #28088: Document Transport.set_protocol and get_protocol http://bugs.python.org/issue28088 opened by yselivanov #28089: Document TCP_NODELAY by default http://bugs.python.org/issue28089 opened by yselivanov #28090: Document PEP 530 http://bugs.python.org/issue28090 opened by yselivanov #28091: Document PEP 525 http://bugs.python.org/issue28091 opened by yselivanov #28092: Build failure for 3.6 on Centos 5.11 http://bugs.python.org/issue28092 opened by steven.daprano #28095: test_startup_imports of test_site fails on OS X due to new imp http://bugs.python.org/issue28095 opened by ned.deily #28097: IDLE: document all key bindings, add menu items for more. http://bugs.python.org/issue28097 opened by terry.reedy #28099: Drop Mac OS X Tiger support in Python 3.6 http://bugs.python.org/issue28099 opened by haypo #28100: Refactor error messages in symtable.c http://bugs.python.org/issue28100 opened by levkivskyi #28107: Update typing module dicumentation for NamedTuple http://bugs.python.org/issue28107 opened by levkivskyi #28108: Python configure fails to detect tzname on platforms that have http://bugs.python.org/issue28108 opened by belopolsky #28110: launcher.msi has different product codes between 32 and 64-bit http://bugs.python.org/issue28110 opened by steve.dower #28111: geometric_mean can raise OverflowError when checking for inf http://bugs.python.org/issue28111 opened by steven.daprano #28113: Remove Py_CreateSymbolicLinkW http://bugs.python.org/issue28113 opened by eryksun #28115: Use argparse for the zipfile module http://bugs.pyt
Re: [Python-Dev] [python-committers] [RELEASE] Python 3.6.0b1 is now available
On 15.09.16 09:48, Berker Peksağ wrote: Fixed, it should redirect to https://www.python.org/blogs/ now. Thanks for noticing this! Thanks Berker! ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
