Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD

2010-12-29 Thread Brian Quinlan
On Dec 29, 2010, at 2:55 PM, Victor Stinner wrote: Le mercredi 29 décembre 2010 à 21:49 +0100, Martin v. Löwis a écrit : Of course, one may wonder why test_first_completed manages to create 41 SemLock objects, when all it tries to do is two future calls. More numbers (on Linux): - Queue: 3

Re: [Python-Dev] Issue #10348: concurrent.futures doesn't work on BSD

2010-12-29 Thread Brian Quinlan
On Dec 29, 2010, at 12:49 PM, Martin v. Löwis wrote: If the functionality is not supported then users get an import error (within multiprocessing). However, RDM's understanding is correct, and the test is creating more than supported. Hmm. The tests do the absolute minimum stuff that

Re: [Python-Dev] futures API

2010-12-11 Thread Brian Quinlan
On Dec 11, 2010, at 6:44 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: I have a process running for a long time, and which

Re: [Python-Dev] futures API

2010-12-11 Thread Brian Quinlan
On Dec 11, 2010, at 6:33 PM, Nick Coghlan wrote: On Sun, Dec 12, 2010 at 6:53 AM, Brian Quinlan br...@sweetapp.com wrote: Is it still unclear why it is there? Maybe you could propose some additional documentation. Did you get my question the other day as to whether a weakref.WeakKeySet

Re: [Python-Dev] futures API

2010-12-10 Thread Brian Quinlan
Oops. I accidentally replied off-list: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: --- El jue, 9/12/10, Brian Quinlan escribió: On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote: I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib

Re: [Python-Dev] futures API

2010-12-10 Thread Brian Quinlan
On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create

Re: [Python-Dev] futures API

2010-12-10 Thread Brian Quinlan
On Dec 10, 2010, at 11:39 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Thomas Nagy escribió: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: I have a process running for a long time, and which may use futures of different max_workers count. I

Re: [Python-Dev] futures API

2010-12-09 Thread Brian Quinlan
On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote: Hello, I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 ) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions

Re: [Python-Dev] futures API

2010-12-09 Thread Brian Quinlan
On Dec 9, 2010, at 2:39 PM, Raymond Hettinger wrote: On Dec 9, 2010, at 9:02 AM, Brian Quinlan wrote: On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote: Hello, I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86

Re: [Python-Dev] PEP 3148 ready for pronouncement [ACCEPTED]

2010-07-12 Thread Brian Quinlan
On 13 Jul 2010, at 00:59, Titus von der Malsburg wrote: On Tue, Jul 13, 2010 at 12:48:35AM +1000, Nick Coghlan wrote: On Tue, Jul 13, 2010 at 12:19 AM, Titus von der Malsburg That's what actually happens, so you can code it either way That's great! None of the examples I found used the

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-28 Thread Brian Quinlan
On May 28, 2010, at 1:39 PM, Scott Dial wrote: On 5/27/2010 4:13 AM, Brian Quinlan wrote: On 27 May 2010, at 17:53, Floris Bruynooghe wrote: On Thu, May 27, 2010 at 01:46:07PM +1200, Greg Ewing wrote: On 27/05/10 00:31, Brian Quinlan wrote: You have two semantic choices here: 1. let

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-27 Thread Brian Quinlan
On May 27, 2010, at 1:21 PM, Greg Ewing wrote: On 27/05/10 12:04, Jesse Noller wrote: Namespaces are only a honking great idea if you actually let them do the job they're designed for. concurrent.* is the namespace, futures is the package within the namespace - concurrent.futures is highly

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-27 Thread Brian Quinlan
On 27 May 2010, at 17:53, Floris Bruynooghe wrote: On Thu, May 27, 2010 at 01:46:07PM +1200, Greg Ewing wrote: On 27/05/10 00:31, Brian Quinlan wrote: You have two semantic choices here: 1. let the interpreter exit with the future still running 2. wait until the future finishes

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-27 Thread Brian Quinlan
On 28 May 2010, at 09:18, Greg Ewing wrote: Brian Quinlan wrote: I think that the Executor suffix is a good indicator of the interface being provided. It's not usually considered necessary for the name of a type to indicate its interface. We don't have 'listsequence' and 'dictmapping

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-27 Thread Brian Quinlan
On May 28, 2010, at 11:57 AM, Reid Kleckner wrote: On Thu, May 27, 2010 at 4:13 AM, Brian Quinlan br...@sweetapp.com wrote: Keep in mind that this library magic is consistent with the library magic that the threading module does - unless the user sets Thread.daemon to True

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On 26 May 2010, at 18:44, Stephen J. Turnbull wrote: Nick Coghlan writes: On 26/05/10 13:51, Stephen J. Turnbull wrote: People have been asking what's special about this module, to violate the BCP principle? There's nothing special about the fact that several people would use a robust

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On 26 May 2010, at 18:09, Glyph Lefkowitz wrote: On May 24, 2010, at 5:36 AM, Brian Quinlan wrote: On May 24, 2010, at 5:16 AM, Glyph Lefkowitz wrote: On May 23, 2010, at 2:37 AM, Brian Quinlan wrote: On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote: ProcessPoolExecutor has the same

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On May 26, 2010, at 8:57 PM, Greg Ewing wrote: Having read through the PEP again, here are my thoughts. * I'm bothered by the term future. To my mind, it's too long on cleverness and too short on explanativeness. I think that the standard library is no place for cuteness of naming. The name

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On 26 May 2010, at 22:06, Floris Bruynooghe wrote: Hi On Sun, May 23, 2010 at 10:47:08AM +1000, Brian Quinlan wrote: Jesse, the designated pronouncer for this PEP, has decided to keep discussion open for a few more days. So fire away! In thread.py the module automatically registers

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On 26 May 2010, at 22:42, Nick Coghlan wrote: On 26/05/10 20:57, Greg Ewing wrote: Having read through the PEP again, here are my thoughts. * It seems unnecessarily verbose to tack Executor onto the end of every Executor subclass. They could simply be called ThreadPool and ProcessPool without

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-26 Thread Brian Quinlan
On 26 May 2010, at 22:50, Antoine Pitrou wrote: On Wed, 26 May 2010 22:32:33 +1000 Nick Coghlan ncogh...@gmail.com wrote: Ha, I'm a bit surprised. Isn't it what futures already provides? (except that for some reason it insists on the SomeExecutor naming scheme)

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-24 Thread Brian Quinlan
On May 24, 2010, at 5:16 AM, Glyph Lefkowitz wrote: On May 23, 2010, at 2:37 AM, Brian Quinlan wrote: On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote: On May 22, 2010, at 8:47 PM, Brian Quinlan wrote: Jesse, the designated pronouncer for this PEP, has decided to keep discussion

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote: On May 22, 2010, at 8:47 PM, Brian Quinlan wrote: Jesse, the designated pronouncer for this PEP, has decided to keep discussion open for a few more days. So fire away! As you wish! I retract my request ;-) The PEP should

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On May 23, 2010, at 7:15 PM, geremy condra wrote: On Sun, May 23, 2010 at 2:37 AM, Brian Quinlan br...@sweetapp.com wrote: snip Finally, why isn't this just a module on PyPI? It doesn't seem like there's any particular benefit to making this a stdlib module and going through the whole

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On May 23, 2010, at 7:54 PM, Lennart Regebro wrote: On Sun, May 23, 2010 at 11:39, Brian Quinlan br...@sweetapp.com wrote: This package eliminates the need to construct the boilerplate present in many Python applications i.e. a thread or process pool, a work queue and result queue

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On May 23, 2010, at 8:43 PM, Robert Collins wrote: On Sun, May 23, 2010 at 10:15 PM, Brian Quinlan br...@sweetapp.com wrote: Also, you can't fix bugs except by releasing new versions of Python. Therefore the API must be completely stable, and the product virtually bugfree before

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On May 23, 2010, at 8:43 PM, Dirkjan Ochtman wrote: On Sun, May 23, 2010 at 12:15, Brian Quinlan br...@sweetapp.com wrote: I doubt it. Simple modules are unlikely to develop a following because it is too easy to partially replicate their functionality. urlparse and os.path are very useful

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-23 Thread Brian Quinlan
On 23 May 2010, at 21:17, Lennart Regebro wrote: On Sun, May 23, 2010 at 12:15, Brian Quinlan br...@sweetapp.com wrote: You could make the same argument about any module in the stdlib. Yeah, and that's exactly what I did. I doubt it. Simple modules are unlikely to develop a following

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-22 Thread Brian Quinlan
On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote: On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan br...@sweetapp.com wrote: Except that now isn't the time for that discussion. This PEP has discussed on-and-off for several months on both stdlib-sig and python-dev. I think any time till

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-22 Thread Brian Quinlan
On 22 May 2010, at 23:59, R. David Murray wrote: On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan br...@sweetapp.com wrote: On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote: On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan br...@sweetapp.com wrote: Except that now isn't the time

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-22 Thread Brian Quinlan
Hey all, Jesse, the designated pronouncer for this PEP, has decided to keep discussion open for a few more days. So fire away! Cheers, Brian ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-22 Thread Brian Quinlan
On May 23, 2010, at 9:43 AM, Jeffrey Yasskin wrote: I think the PEP's overall API is good to go. On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan br...@sweetapp.com wrote: On 22 May 2010, at 23:59, R. David Murray wrote: If there is still discussion then perhaps the PEP isn't ready

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-22 Thread Brian Quinlan
On May 23, 2010, at 10:06 AM, Jeffrey Yasskin wrote: On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan br...@sweetapp.com wrote: Rename executor = executer -1 for consistency with Java. -1 pending an explanation of why executer is better Rename submit to apply apply focuses attention

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-21 Thread Brian Quinlan
Hey Mark, This really isn't the time to propose changes. The PEP has been discussed extensively on stdlib-sig and python-dev. On May 21, 2010, at 9:29 PM, Mark Summerfield wrote: On 2010-05-21, Brian Quinlan wrote: The PEP is here: http://www.python.org/dev/peps/pep-3148/ [snip] Hi Brian

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-21 Thread Brian Quinlan
On May 21, 2010, at 9:44 PM, John Arbash Meinel wrote: Brian Quinlan wrote: The PEP is here: http://www.python.org/dev/peps/pep-3148/ I think the PEP is ready for pronouncement, and the code is pretty much ready for submission into py3k (I will have to make some minor changes

Re: [Python-Dev] PEP 3148 ready for pronouncement

2010-05-21 Thread Brian Quinlan
On May 21, 2010, at 9:47 PM, John Arbash Meinel wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brian Quinlan wrote: The PEP is here: http://www.python.org/dev/peps/pep-3148/ I think the PEP is ready for pronouncement, and the code is pretty much ready for submission into py3k (I

[Python-Dev] PEP 3148 ready for pronouncement

2010-05-20 Thread Brian Quinlan
The PEP is here: http://www.python.org/dev/peps/pep-3148/ I think the PEP is ready for pronouncement, and the code is pretty much ready for submission into py3k (I will have to make some minor changes in the patch like changing the copyright assignment):

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-05-02 Thread Brian Quinlan
I've updated the PEP to include: - completion callbacks (for interoperability with Twisted Deferreds) - a pointer to the discussion on stdlig-sig See: http://svn.python.org/view/peps/trunk/pep-3148.txt?r1=78618r2=80679 Rejected ideas: - Having a registration system for executors Not yet

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-16 Thread Brian Quinlan
Nick Coghlan wrote: You may want to consider providing global thread and process executors in the futures module itself. Code which just wants to say do this in the background without having to manage the lifecycle of its own executor instance is then free to do so. I've had a lot of

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-11 Thread Brian Quinlan
On 10 Mar 2010, at 23:32, Nick Coghlan wrote: Brian Quinlan wrote: Getting rid of the process-global state like this simplifies testing (both testing of the executors themselves and of application code which uses them). It also eliminates the unpleasant interpreter shutdown/module globals

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-10 Thread Brian Quinlan
On 10 Mar 2010, at 08:32, Dj Gilcrease wrote: On Mon, Mar 8, 2010 at 2:11 PM, exar...@twistedmatrix.com wrote: Getting rid of the process-global state like this simplifies testing (both testing of the executors themselves and of application code which uses them). It also eliminates the

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-10 Thread Brian Quinlan
On 9 Mar 2010, at 08:39, Greg Ewing wrote: Terry Reedy wrote: Looking more close, I gather that the prime results will be printed 'in order' (waiting on each even if others are done) while the url results will be printed 'as available'. Seems to me that if you care about the order of the

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-10 Thread Brian Quinlan
On 9 Mar 2010, at 08:11, exar...@twistedmatrix.com wrote: On 08:56 pm, digitalx...@gmail.com wrote: On Mon, Mar 8, 2010 at 12:04 PM, Dj Gilcrease digitalx...@gmail.com wrote: A style I have used in my own code in the past is a Singleton class with register and create methods, where the

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-10 Thread Brian Quinlan
On 9 Mar 2010, at 03:21, Terry Reedy wrote: On 3/6/2010 4:20 AM, Brian Quinlan wrote: On 6 Mar 2010, at 03:21, Daniel Stutzbach wrote: On Fri, Mar 5, 2010 at 12:03 AM, Brian Quinlan br...@sweetapp.com mailto:br...@sweetapp.com wrote: import futures +1 on the idea, -1 on the name

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-06 Thread Brian Quinlan
, return_when=ALL_COMPLETED)``? Hi Brett, That recommendation was designed to make it easy to change the API without breaking code. I'd don't think that recommendation makes sense anymore any I'll update the PEP. Cheers, Brian On Thu, Mar 4, 2010 at 22:03, Brian Quinlan br...@sweetapp.com

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-06 Thread Brian Quinlan
On 6 Mar 2010, at 08:42, Jesse Noller wrote: If people agree with this; do you feel the proposal of said namespace should be a separate PEP, or piggy back on this? I don't want to piggy back on Brian's hard work. It doesn't really matter to me. We can either update this PEP to propose the

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-06 Thread Brian Quinlan
On 6 Mar 2010, at 09:54, Antoine Pitrou wrote: Le Fri, 5 Mar 2010 17:03:02 +1100, Brian Quinlan br...@sweetapp.com a écrit : The PEP lives here: http://python.org/dev/peps/pep-3148/ Ok, here is my take on it: cancel() Attempt to cancel the call. If the call is currently being executed

Re: [Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-06 Thread Brian Quinlan
On 7 Mar 2010, at 03:04, Phillip J. Eby wrote: At 05:32 AM 3/6/2010, Brian Quinlan wrote: Using twisted (or any other asynchronous I/O framework) forces you to rewrite your I/O code. Futures do not. Twisted's Deferred API has nothing to do with I/O. I see, you just mean the API

[Python-Dev] [PEP 3148] futures - execute computations asynchronously

2010-03-04 Thread Brian Quinlan
Hi all, I recently submitted a daft PEP for a package designed to make it easier to execute Python functions asynchronously using threads and processes. It lets the user focus on their computational problem without having to build explicit thread/process pools and work queues. The

Re: [Python-Dev] Possible py3k io wierdness

2009-04-12 Thread Brian Quinlan
I've added a new proposed patch to: http://bugs.python.org/issue5700 The idea is: - only IOBase implements close() (though a subclass can override close without causing problems so long as it calls super().close() or calls .flush() and ._close() directly) - change IOBase.close to call

Re: [Python-Dev] Possible py3k io wierdness

2009-04-06 Thread Brian Quinlan
Nick Coghlan wrote: Brian Quinlan wrote: - you need the cooperation of your subclasses i.e. they must call super().flush() in .flush() to get correct close behavior (and this represents a backwards-incompatible semantic change) Are you sure about that? Going by the current _pyio semantics

Re: [Python-Dev] Possible py3k io wierdness

2009-04-05 Thread Brian Quinlan
Hey Antoine, Thanks for the clarification! I see that the C implementation matches the Python implementation but I don't see how the semantics of either are useful in this case. If a subclass implements flush then, as you say, it must also implement close and call flush itself before

Re: [Python-Dev] Possible py3k io wierdness

2009-04-05 Thread Brian Quinlan
Antoine Pitrou wrote: Brian Quinlan brian at sweetapp.com writes: I don't see why this is helpful. Could you explain why _RawIOBase.close() calling self.flush() is useful? I could not explain it for sure since I didn't write the Python version. I suppose it's so that people who only override

Re: [Python-Dev] dict(keys, values)

2007-02-01 Thread Brian Quinlan
George Sakkis wrote: Perhaps this has been brought up in the past but I couldn't find it in the archives: far too often I use the idiom dict(zip(keys,values)), or the same with izip. How does letting dict take two positional arguments sound ? Pros: - Pretty obvious semantics, no mental

Re: [Python-Dev] dict(keys, values)

2007-02-01 Thread Brian Quinlan
George Sakkis wrote: Um, you do realize that dict(keys=keys, values=values) is already valid and quite different from dict(zip(keys, values)), don't you ? :) Sorry, minor misreading on my part. Like that time in Sunday school when I missed the not in Though shall not kill. That was a rough

Re: [Python-Dev] Any reason that any()/all() do not take a predicateargument?

2006-04-15 Thread Brian Quinlan
seq = [1,2,3,4,5] if any(seq, lambda x: x==5): ... which is clearly more readable than reduce(seq, lambda x,y: x or y==5, False) How about this? if any(x==5 for x in seq): Aren't all of these equivalent to: if 5 in seq: ... ? Cheers, Brian