[Python-Dev] Should standard library modules optimize for CPython?
I think I know the answer to this, but I'm going to ask it anyway... I know that there is a general policy of trying to write code in the standard library that does not disadvantage other implementations. How far does that go the other way? Should the standard library accept slower code because it will be much faster in other implementations? Briefly, I have a choice of algorithm for the median function in the statistics module. If I target CPython, I will use a naive but simple O(N log N) implementation based on sorting the list and returning the middle item. (That's what the module currently does.) But if I target PyPy, I will use an O(N) algorithm which knocks the socks off the naive version even for smaller lists. In CPython that's typically 2-5 times slower; in PyPy it's typically 3-8 times faster, and the bigger the data set the more the advantage. For the specific details, see http://bugs.python.org/issue21592 My feeling is that the CPython standard library should be written for CPython, that is, it should stick to the current naive implementation of median, and if PyPy wants to speed the function up, they can provide their own version of the module. I should *not* complicate the implementation by trying to detect which Python the code is running under and changing algorithms accordingly. However, I should put a comment in the module pointing at the tracker issue. Does this sound right to others? Thanks, -- Steve ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should standard library modules optimize for CPython?
Steven D'Aprano, 01.06.2014 10:11: > Briefly, I have a choice of algorithm for the median function in the > statistics module. If I target CPython, I will use a naive but simple > O(N log N) implementation based on sorting the list and returning the > middle item. (That's what the module currently does.) But if I target > PyPy, I will use an O(N) algorithm which knocks the socks off the naive > version even for smaller lists. In CPython that's typically 2-5 times > slower; in PyPy it's typically 3-8 times faster, and the bigger the data > set the more the advantage. > > For the specific details, see http://bugs.python.org/issue21592 > > My feeling is that the CPython standard library should be written for > CPython, that is, it should stick to the current naive implementation of > median, and if PyPy wants to speed the function up, they can provide > their own version of the module. Note that if you compile the module with Cython, CPython heavily benefits from the new implementation, too, by a factor of 2-5x. So there isn't really a reason to choose between two implementations because of the two runtimes, just use the new one for both and compile it for CPython. I added the necessary bits to the ticket. Stefan ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should standard library modules optimize for CPython?
On 1 Jun 2014 18:13, "Steven D'Aprano" wrote: > > My feeling is that the CPython standard library should be written for > CPython, that is, it should stick to the current naive implementation of > median, and if PyPy wants to speed the function up, they can provide > their own version of the module. I should *not* complicate the > implementation by trying to detect which Python the code is running > under and changing algorithms accordingly. However, I should put a > comment in the module pointing at the tracker issue. Does this sound > right to others? One option is to set the pure Python module up to be paired with an accelerator module (and update the test suite accordingly), even if we *don't provide* an accelerator in CPython. That just inverts the more common case (where we have an accelerator written in C, but another implementation either doesn't need one, or just doesn't have one yet). Cheers, Nick. > > > Thanks, > > > > -- > Steve > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should standard library modules optimize for CPython?
Le 01/06/2014 10:11, Steven D'Aprano a écrit : My feeling is that the CPython standard library should be written for CPython, that is, it should stick to the current naive implementation of median, and if PyPy wants to speed the function up, they can provide their own version of the module. I should *not* complicate the implementation by trying to detect which Python the code is running under and changing algorithms accordingly. However, I should put a comment in the module pointing at the tracker issue. Does this sound right to others? It sounds ok to me. Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] [RELEASE] Python 2.7.7
I'm happy to announce the immediate availability of Python 2.7.7. Python 2.7.7 is a regularly scheduled bugfix release for the Python 2.7 series. This release includes months of accumulated bugfixes. All the changes in Python 2.7.7 are described in detail in the Misc/NEWS file of the source tarball. You can view it online at http://hg.python.org/cpython/raw-file/f89216059edf/Misc/NEWS The 2.7.7 release also contains fixes for two severe, if arcane, potential security vulnerabilities. The first was the possibility of reading arbitrary process memory using JSONDecoder.raw_decode. [1] (No other json APIs are affected.) The second security issue is an integer overflow in the strop module. [2] (You actually have no reason whatsoever to use the strop module.) Another security note for 2.7.7 is that the release includes a backport from Python 3 of hmac.compare_digest. This begins the implementation of PEP 466, Network Security Enhancements for Python 2.7.x. Downloads are at https://python.org/download/releases/2.7.7/ This is a production release. As always, please report bugs to http://bugs.python.org/ Build great things, Benjamin Peterson 2.7 Release Manager (on behalf of all of Python's contributors) [1] http://bugs.python.org/issue21529 [2] http://bugs.python.org/issue21530 ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updating turtle.py
On May 30, 2014, at 8:32 PM, Terry Reedy wrote: > B. Lets assuming that turtle.py is, at least to some degree, fair game for > fixes and enhancements. PSF Python PyLadies (Jessica Keller, Lynn Root) are > participating in the 2014 GNOME Outreach Program for Women (OPW) > https://wiki.python.org/moin/OPW/2014 . One of the projects (bottem of that > page) is Graphical Python, in particular Turtle. > > A few days ago, Jessica posted > http://bugs.python.org/issue21573 Clean up turtle.py code formatting > "Lib/turtle.py has some code formatting issues. Let's clean them up to make > the module easier to read as interns start working on it this summer." She > want to follow cleanup with code examination, fixes, and enhancements. If these modules are going to change (and Gregor gives us the go-ahead), I suggest we do real clean-ups, not shallow pep8/pylint micro-changes. I use these modules as part of a program to teach adults how to teach programming to children. I've have good success but think the code for several of the modules needs to be simplified. At some point, kids wrote some of this code but along the way it got "adultified", making it less useful for teaching younger kids. I would like to be involved in helping to improve these modules in a substantive way and would be happy to coach anyone who wants to undertake the effort and bring a useful patch to fruition. One thing I would not like to see happen is telling interns that their time is being well spent by pep-8 checking code in the standard library. It sends that wrong message about what constitutes an actual contribution to the core. There are plenty of useful things to do instead (we have an "easy" tag on tracker to highlight a few of them). Another thought is that there are tons of python projects that could use real help and those would likely be a better place to start than trying to patch mature standard library code (where the chance of regression, code churn, or rejection is much higher). Over the past few years, I've taught Python to over three thousand programmers and have gotten a number of them started in open source (a number of them are now active contributors to OpenStack for example), but I almost never direct them to take their baby steps in the Python core (unless they've found an actual defect or room for improvement). It's a bummer, but in mature code, almost every idea that occurs to a beginner is something that makes the code worse in some way -- that isn't always true but it happens often enough to be discouraging. Raymond___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Should standard library modules optimize for CPython?
On Jun 1, 2014, at 9:17 AM, Antoine Pitrou wrote: > Le 01/06/2014 10:11, Steven D'Aprano a écrit : >> >> My feeling is that the CPython standard library should be written for >> CPython, that is, it should stick to the current naive implementation of >> median, and if PyPy wants to speed the function up, they can provide >> their own version of the module. I should *not* complicate the >> implementation by trying to detect which Python the code is running >> under and changing algorithms accordingly. However, I should put a >> comment in the module pointing at the tracker issue. Does this sound >> right to others? > > It sounds ok to me. That makes sense. Raymond___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Updating turtle.py
Raymond Hettinger writes: > One thing I would not like to see happen is telling interns that > their time is being well spent by pep-8 checking code in the > standard library. It sends that wrong message about what > constitutes an actual contribution to the core. There are plenty > of useful things to do instead (we have an "easy" tag on tracker to > highlight a few of them). I have to ask for a qualification here, at least in the case of GSoC interns. Of course the intern should contribute to the code, but they are also supposed to become developers in the community. Spending a few hours checking code for PEP-8-correctness is useful training in writing good core code going forward. I agree with you that if they don't move on after a day or so, they should be told to do so. OTOH, I haven't yet met an intern who was willing and able to write in good PEP 8 style to start with, let alone one who was willing to "waste his time" doing style-checking on existing code -- is this really a problem? I agree that they should be told that this is an investment in *their* skills, and at best of marginal value to *Python*, of course. As you point out, directing them away from core code to other projects requiring PEP 8 in their style guides is usually a good idea, too. > It's a bummer, but in mature code, almost every idea that occurs to > a beginner is something that makes the code worse in some way -- > that isn't always true but it happens often enough to be > discouraging. This is precisely why style-checking in the core may be a good idea for interns: assume the code is *good* code (it probably is), don't mess with the algorithms, but make the code "look right" according to project standards. The risk you cite is still there, but much less. It shows them what Pythonicity looks like at a deeper level than the relatively superficial[1] guidelines in PEP 8. Footnotes: [1] Not deprecatory. Consistent good looks are important. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
