On Wed, May 14, 2008 at 4:45 AM, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Jesse Noller wrote: > > I am looking for any questions, concerns or benchmarks python-dev has > > regarding the possible inclusion of the pyprocessing module to the > > standard library - preferably in the 2.6 timeline. > > I think for inclusion in 2.6 it's to late. For 3.0, it's definitely > too late - the PEP acceptance deadline was a year ago (IIRC). >
That's a fair point - however, as this is a proposed inclusion of a standalone library that includes tests, documentation and benchmarks - I was hoping it would be possible to attempt to get it in. A target for 3.1 or 2.7 is also possible. > > > As I am trying to finish up the PEP, I want to see if I can address > > any questions or include any other useful data (including benchmarks) > > in the PEP prior to publishing it. I am also intending to include > > basic benchmarks for both the processing module against the threading > > module as a comparison. > > I'm worried whether it's stable, what user base it has, whether users > (other than the authors) are lobbying for inclusion. Statistically, > it seems to be not ready yet: it is not even a year old, and has not > reached version 1.0 yet. Just as a point of clarification: I am not the module author. I am simply a relatively passionate user using it to solve real problems. I am working on the PEP after speaking with the author about it. One of the outstanding issues I have listed is the "lack" of a 1.0 version - however given the fairly random way that most open source projects pick numbering schemes - I don't see this as much more than an administrative issue. As for user base - at PyCon this year, I was involved in several open space discussion about the concurrency/parallelism "problem" as it relates to Python, as well as giving a lightning talk on this precise subject including mentioning this module and alternatives. The sheer number of people working on "big" problems who approached me to talk about this was astounding. The presence online of the same like-minded people is also quite large. Overwhelmingly, many of the python programmers I spoke to are looking for "a solution" that does not require the alteration of a known programming paradigm (i.e: threads) to allow them to take advantage of systems which are not getting "faster" - instead, they are getting wider. Simply put due to the GIL - pure python applications can not take advantage of these machines which are now more common than not without switching to an alternative interpreter - which for many - myself included - is not an attractive option. Guido, and others called this module out specifically as a "work-around" to the GIL for concurrency within cPython applications. The API matches a known one and in most (if not all) cases, it is an easy drop in to allow programmers to achieve what they may not be able to do with the standard library as it is today. Obviously, there are other solutions that aim to solve this problem domain - Twisted, ParallelPython, etc all "work around" the GIL through various methods - but the fundamental problem with those is that they ask a user to "re-learn" how to construct their applications. While the concept of shared memory and "threads" may not be the "final answer" in the concurrent programming world - fundamentally, it's wrong for us to say that in order for people to really use python to solve concurrent problems they have to: 1: Re-architect their applications 2: Re-learn how to program (i.e: Async programming, Actor models, shared-nothing) I really do feel that inclusion of this library offers us the best of both worlds - it gives us (as a community) an easy answer to those people who would dismiss python due to the GIL and it also allows users to easily implement their applications. The GIL is currently advantageous to the cPython code base for simple ease of implementation and maintenance. Barring the one-day adoption of the bulk of Adam Olsen's safe-threading work or a massive trend in programming to "rid" ourselves of the threaded mindset - this gives cPython a leg-up in the concurrency world and gives users a tool to solve hard problems. -Jesse _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
