Re: [Python-Dev] Ask a question for a script about re.findall Modlue

[Hagen Fürstenau]

> Your problem is easily explained however: the second argument to
> p.findall() should be an offset, not a flag set. (You are confusing
> re.findall() and p.findall().)

I filed a doc bug for this:

http://bugs.python.org/issue8785

Cheers,
Hagen



signature.asc
Description: OpenPGP digital signature
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:

On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan   
wrote:
Except that now isn't the time for that discussion. This PEP has  
discussed

on-and-off for several months on both stdlib-sig and python-dev.


I think any time till the PEP is accepted is a good time to discuss
changes to the API


I disagree. If a PEP is being updated continuously then there is  
nothing stable to pronounce on.




Issues with the PEP:
1) Examples as written fail on windows. Patch to fix @
http://code.google.com/p/pythonfutures/issues/detail?id=5


Updated, thanks!


Issues with Implementation:
1) Globals are used for tracking running threads (but not processes)
and shutdown state, but are not accessed as a globals every where they
are modified so it could be inconsistent.





2) The atexit handle randomly throws an exception on exit on windows
when running the tests or examples.

Error in atexit._run_exitfuncs:
TypeError: print_exception(): Exception expected for value, str found


Lets take this off-list.

Cheers,
Brian


Issues 1 & 2 would be solved by moving thread tracking back into the
executor responsible for the threads, or making a singleton that
tracked threads / processes for all executors.
http://code.google.com/p/pythonfutures/issues/detail?id=6 is one such
implementation
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[R. David Murray]

On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan  wrote:
> On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:
> > On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan   
> > wrote:
> >> Except that now isn't the time for that discussion. This PEP has  
> >> discussed
> >> on-and-off for several months on both stdlib-sig and python-dev.
> >
> > I think any time till the PEP is accepted is a good time to discuss
> > changes to the API
> 
> I disagree. If a PEP is being updated continuously then there is  
> nothing stable to pronounce on.

Well, you've been making updates as a result of this round of
discussion.

If there is still discussion then perhaps the PEP isn't ready for
pronouncement yet.  At some point someone can decide it is all
bikeshedding and ask for pronouncement on that basis, but I don't
think it is appropriate to cut off discussion by saying "it's ready for
pronouncement" unless you want increase the chances of its getting
rejected.

The usual way of doing this (at least so far as I have observed, which
granted hasn't been too many cases) is to say something like "I think
this PEP is ready for pronouncement" and then wait for feedback on that
assertion or for the pronouncement.  It's especially good if you can answer
any concerns that are raised with "that was discussed already and we
concluded X".  Bonus points for finding a thread reference and adding
it to the PEP :)

--
R. David Murray  www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Jesse Noller]

On Sat, May 22, 2010 at 9:59 AM, R. David Murray  wrote:
> On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan  wrote:
>> On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:
>> > On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan 
>> > wrote:
>> >> Except that now isn't the time for that discussion. This PEP has
>> >> discussed
>> >> on-and-off for several months on both stdlib-sig and python-dev.
>> >
>> > I think any time till the PEP is accepted is a good time to discuss
>> > changes to the API
>>
>> I disagree. If a PEP is being updated continuously then there is
>> nothing stable to pronounce on.
>
> Well, you've been making updates as a result of this round of
> discussion.
>
> If there is still discussion then perhaps the PEP isn't ready for
> pronouncement yet.  At some point someone can decide it is all
> bikeshedding and ask for pronouncement on that basis, but I don't
> think it is appropriate to cut off discussion by saying "it's ready for
> pronouncement" unless you want increase the chances of its getting
> rejected.

I commiserate with Brian here - he's been very patient, and has been
working on things, taking in input, etc for awhile now on this. In his
mind, it is done (or at least incredibly close to done) and opening
the door in the conversation for more API nit picking and debate about
the exact verbiage on method names means we're never going to be done
splashing paint.

> The usual way of doing this (at least so far as I have observed, which
> granted hasn't been too many cases) is to say something like "I think
> this PEP is ready for pronouncement" and then wait for feedback on that
> assertion or for the pronouncement.  It's especially good if you can answer
> any concerns that are raised with "that was discussed already and we
> concluded X".  Bonus points for finding a thread reference and adding
> it to the PEP :)

While sure, this is true - I'd actually back Brian up on trying to
avoid more "why didn't you call it a banana" style discussions. At
some point the constant back and forth has to stop, and to his credit,
Brian has made a lot of changes, listened to a lot of feedback, etc.
It's fair for him to just ask that a decision be made.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Guido van Rossum]

On Sat, May 22, 2010 at 7:09 AM, Jesse Noller  wrote:
> On Sat, May 22, 2010 at 9:59 AM, R. David Murray  
> wrote:
>> On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan  wrote:
>>> On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:
>>> > On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan 
>>> > wrote:
>>> >> Except that now isn't the time for that discussion. This PEP has
>>> >> discussed
>>> >> on-and-off for several months on both stdlib-sig and python-dev.
>>> >
>>> > I think any time till the PEP is accepted is a good time to discuss
>>> > changes to the API
>>>
>>> I disagree. If a PEP is being updated continuously then there is
>>> nothing stable to pronounce on.
>>
>> Well, you've been making updates as a result of this round of
>> discussion.
>>
>> If there is still discussion then perhaps the PEP isn't ready for
>> pronouncement yet.  At some point someone can decide it is all
>> bikeshedding and ask for pronouncement on that basis, but I don't
>> think it is appropriate to cut off discussion by saying "it's ready for
>> pronouncement" unless you want increase the chances of its getting
>> rejected.
>
> I commiserate with Brian here - he's been very patient, and has been
> working on things, taking in input, etc for awhile now on this. In his
> mind, it is done (or at least incredibly close to done) and opening
> the door in the conversation for more API nit picking and debate about
> the exact verbiage on method names means we're never going to be done
> splashing paint.
>
>> The usual way of doing this (at least so far as I have observed, which
>> granted hasn't been too many cases) is to say something like "I think
>> this PEP is ready for pronouncement" and then wait for feedback on that
>> assertion or for the pronouncement.  It's especially good if you can answer
>> any concerns that are raised with "that was discussed already and we
>> concluded X".  Bonus points for finding a thread reference and adding
>> it to the PEP :)
>
> While sure, this is true - I'd actually back Brian up on trying to
> avoid more "why didn't you call it a banana" style discussions. At
> some point the constant back and forth has to stop, and to his credit,
> Brian has made a lot of changes, listened to a lot of feedback, etc.
> It's fair for him to just ask that a decision be made.

Great points Jesse! Since I really don't have the time or expertise to
make a judgment on this PEP, I hereby appoint you chair of the
approval process for this PEP. That basically means that when you
think it's ready to be approved, you say so, and it's a done deal. The
remaining feedback cycle is up to you now -- it sounds like you're
ready for closure, which sounds good to me (again, without having read
the PEP or tried to write something using the proposed code). You can
do it however you like: you can declare it approved now, or read it
over once more yourself and suggest some final changes, or set a
period (e.g. 48 hours) during which final comments have to be
received, which you then will judge by merit or by your whim, or you
can flip a coin or say a prayer... (I've tried most of those myself in
the past and haven't done too badly if I say so myself. :-) You're the
boss now. I know you will do the right thing for this PEP.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[R. David Murray]

On Sat, 22 May 2010 10:09:37 -0400, Jesse Noller  wrote:
> On Sat, May 22, 2010 at 9:59 AM, R. David Murray  wr=
> ote:
> > On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan  wr=
> ote:
> >> On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:
> >> > On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan 
> >> > wrote:
> >> >> Except that now isn't the time for that discussion. This PEP has
> >> >> discussed
> >> >> on-and-off for several months on both stdlib-sig and python-dev.
> >> >
> >> > I think any time till the PEP is accepted is a good time to discuss
> >> > changes to the API
> >>
> >> I disagree. If a PEP is being updated continuously then there is
> >> nothing stable to pronounce on.
> >
> > Well, you've been making updates as a result of this round of
> > discussion.
> >
> > If there is still discussion then perhaps the PEP isn't ready for
> > pronouncement yet. =A0At some point someone can decide it is all
> > bikeshedding and ask for pronouncement on that basis, but I don't
> > think it is appropriate to cut off discussion by saying "it's ready for
> > pronouncement" unless you want increase the chances of its getting
> > rejected.
> 
> I commiserate with Brian here - he's been very patient, and has been
> working on things, taking in input, etc for awhile now on this. In his
> mind, it is done (or at least incredibly close to done) and opening
> the door in the conversation for more API nit picking and debate about
> the exact verbiage on method names means we're never going to be done
> splashing paint.

OK, so you are saying that the comments in question are bikeshedding.
I can accept that easily.  What I was trying to point out was that saying
"discussion is closed" is not the best way to nurture community consensus.
Saying "we've reached the bikesheedding point, let's pronounce" is a very
different thing to my mind, even if it is only a matter of tone.

Have fun pronouncing ;)

--
R. David Murray  www.bitdance.com
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On 22 May 2010, at 23:59, R. David Murray wrote:

On Sat, 22 May 2010 19:12:05 +1000, Brian Quinlan  
 wrote:

On May 22, 2010, at 5:30 AM, Dj Gilcrease wrote:

On Fri, May 21, 2010 at 8:26 AM, Brian Quinlan 
wrote:

Except that now isn't the time for that discussion. This PEP has
discussed
on-and-off for several months on both stdlib-sig and python-dev.


I think any time till the PEP is accepted is a good time to discuss
changes to the API


I disagree. If a PEP is being updated continuously then there is
nothing stable to pronounce on.


Well, you've been making updates as a result of this round of
discussion.


Yes, I've been making documentation and PEP updates to clarify points  
that people found confusing and will continue to do so.



If there is still discussion then perhaps the PEP isn't ready for
pronouncement yet.  At some point someone can decide it is all
bikeshedding and ask for pronouncement on that basis, but I don't
think it is appropriate to cut off discussion by saying "it's ready  
for

pronouncement" unless you want increase the chances of its getting
rejected.


Here are the new proposed non-documentation changes that I've  
collected (let me know if I've missed any):


Rename "executor" => "executer"
Rename "submit" to "apply"
Rename "done" to "finished"
Rename "not_finished" to "pending"
Rename "FIRST_COMPLETED" to "ONE_COMPLETED"

We can discuss naming for all eternity and never reach a point where  
even half of the participants are satisfied.


Since naming has been discussed extensively here and in stdlib-sig, I  
think that we have to decide that it is good enough and move on. Or  
decide that it isn't good enough and reject the PEP.


Cheers,
Brian


The usual way of doing this (at least so far as I have observed, which
granted hasn't been too many cases) is to say something like "I think
this PEP is ready for pronouncement" and then wait for feedback on  
that
assertion or for the pronouncement.  It's especially good if you can  
answer

any concerns that are raised with "that was discussed already and we
concluded X".  Bonus points for finding a thread reference and adding
it to the PEP :)

--
R. David Murray  www.bitdance.com


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Jeffrey Yasskin]

I think the PEP's overall API is good to go.

On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan  wrote:
>
> On 22 May 2010, at 23:59, R. David Murray wrote:
>> If there is still discussion then perhaps the PEP isn't ready for
>> pronouncement yet.  At some point someone can decide it is all
>> bikeshedding and ask for pronouncement on that basis, but I don't
>> think it is appropriate to cut off discussion by saying "it's ready for
>> pronouncement" unless you want increase the chances of its getting
>> rejected.
>
> Here are the new proposed non-documentation changes that I've collected (let
> me know if I've missed any):
>
> ...

I propose to rename the Future.result method to Future.get. "get" is
what Java 
(http://java.sun.com/javase/7/docs/api/java/util/concurrent/Future.html)
and C++ (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3092.pdf
section 30.6.6 para 12) use, and the word "result" doesn't seem
particularly better or worse than "get" for our purposes, which
inclines me to stay consistent.

> We can discuss naming for all eternity and never reach a point where even
> half of the participants are satisfied.

Agreed. To reduce the length of the discussion, I'm not going to reply
to counter-arguments to my proposal, but I think it'll be useful to
Jesse if people who agree or disagree speak up briefly. I'll reply the
other naming proposals in another message.

Jeffrey
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Jeffrey Yasskin]

On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan  wrote:
> Rename "executor" => "executer"

-1 for consistency with Java.

> Rename "submit" to "apply"

"apply" focuses attention on the function object, while "submit"
focuses attention, properly I think, on the fact that you're handing
something to the executor to run. So -1.

> Rename "done" to "finished"

"done" is nice and short, and I don't think "finished" or "completed"
will be any less prone to people thinking the task actually ran. So
-1.

> Rename "not_finished" to "pending"

+0.5. Doesn't matter that much, but pending is used elsewhere in the
proposal for this concept. On the other hand, "pending" could be
thought to refer to the state before "running". Possibly "finished"
should be renamed to "done" here, since it's described as '"finished",
contains the futures that completed (finished or were cancelled)',
which uses "finished" for two different concepts.

> Rename "FIRST_COMPLETED" to "ONE_COMPLETED"

"ONE_COMPLETED" could imply that the first result set must contain
exactly one element, but in fact, if multiple tasks finish before the
waiting thread has a chance to wake up, multiple futures could be
returned as done. So -1.


And like my other post, I won't argue about these, leaving the actual
decision up to Brian and Jesse.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

Hey all,

Jesse, the designated pronouncer for this PEP, has decided to keep  
discussion open for a few more days.


So fire away!

Cheers,
Brian

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On May 23, 2010, at 9:43 AM, Jeffrey Yasskin wrote:


I think the PEP's overall API is good to go.

On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan   
wrote:


On 22 May 2010, at 23:59, R. David Murray wrote:

If there is still discussion then perhaps the PEP isn't ready for
pronouncement yet.  At some point someone can decide it is all
bikeshedding and ask for pronouncement on that basis, but I don't
think it is appropriate to cut off discussion by saying "it's  
ready for

pronouncement" unless you want increase the chances of its getting
rejected.


Here are the new proposed non-documentation changes that I've  
collected (let

me know if I've missed any):

...


I propose to rename the Future.result method to Future.get. "get" is
what Java (http://java.sun.com/javase/7/docs/api/java/util/concurrent/Future.html 
)

and C++ (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3092.pdf
section 30.6.6 para 12) use, and the word "result" doesn't seem
particularly better or worse than "get" for our purposes, which
inclines me to stay consistent.


In C++ and Java, there is only one result-retrieving method so "get"  
seems like a reasonable name.


My implementation has a second method .exception(), which returns the  
exception raised by the submitted function (or None if no exception  
was raised). I thought that having multiple getter methods, where one  
is called .get() would be a bit confusing.


But I don't really care so I'm -0.

Cheers,
Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Jesse Noller]

On Sat, May 22, 2010 at 8:47 PM, Brian Quinlan  wrote:
> Hey all,
>
> Jesse, the designated pronouncer for this PEP, has decided to keep
> discussion open for a few more days.
>
> So fire away!

Man, everyone's faster on the email thing lately than me :)

Yes, I spoke to Brian, and since we're not in a rush - please do bring
up serious issues you might have, assume for the moment that the
names, unless they're "def lol_python" are going to stay pretty
consistent unless everyone breaks out the pitchforks, and also assume
the API won't see much changing.

So please, do bring up issues. I'm obviously biased towards accepting
it - however, nothing is ever set in stone.

jesse
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On May 23, 2010, at 10:06 AM, Jeffrey Yasskin wrote:

On Sat, May 22, 2010 at 4:12 PM, Brian Quinlan   
wrote:

Rename "executor" => "executer"


-1 for consistency with Java.



-1 pending an explanation of why "executer" is better


Rename "submit" to "apply"


"apply" focuses attention on the function object, while "submit"
focuses attention, properly I think, on the fact that you're handing
something to the executor to run. So -1.


-1


Rename "done" to "finished"


"done" is nice and short, and I don't think "finished" or "completed"
will be any less prone to people thinking the task actually ran. So
-1.


-0


Rename "not_finished" to "pending"


+0.5. Doesn't matter that much, but pending is used elsewhere in the
proposal for this concept. On the other hand, "pending" could be
thought to refer to the state before "running". Possibly "finished"
should be renamed to "done" here, since it's described as '"finished",
contains the futures that completed (finished or were cancelled)',
which uses "finished" for two different concepts.


I think that using "finished" is bad terminology here.

So +1 to "finished" => "done".
I don't have a preference for "not_done" vs. "pending".



Rename "FIRST_COMPLETED" to "ONE_COMPLETED"


"ONE_COMPLETED" could imply that the first result set must contain
exactly one element, but in fact, if multiple tasks finish before the
waiting thread has a chance to wake up, multiple futures could be
returned as done. So -1.


A logician would probably call it "SOME_COMPLETED".
What about "ANY_COMPLETED"? Though I think that "FIRST_COMPLETED"  
still reads better.


Cheers,
Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Glyph Lefkowitz]

On May 22, 2010, at 8:47 PM, Brian Quinlan wrote:

> Jesse, the designated pronouncer for this PEP, has decided to keep discussion 
> open for a few more days.
> 
> So fire away!

As you wish!

The PEP should be consistent in its usage of terminology about callables.  It 
alternately calls them "callables", "functions", and "functions or methods".  
It would be nice to clean this up and be consistent about what can be called 
where.  I personally like "callables".

The execution context of callable code is not made clear.  Implicitly, submit() 
or map() would run the code in threads or processes as defined by the executor, 
but that's not spelled out clearly.

More relevant to my own interests, the execution context of the callables 
passed to add_done_callback and remove_done_callback is left almost completely 
to the imagination.  If I'm reading the sample implementation correctly, 
,
 it looks like in the multiprocessing implementation, the done callbacks are 
invoked in a random local thread.  The fact that they are passed the future 
itself *sort* of implies that this is the case, but the multiprocessing module 
plays fast and loose with object identity all over the place, so it would be 
good to be explicit and say that it's *not* a pickled copy of the future 
sitting in some arbitrary process (or even on some arbitrary machine).

This is really minor, I know, but why does it say "NOTE: This method can be 
used to create adapters from Futures to Twisted Deferreds"?  First of all, 
what's the deal with "NOTE"; it's the only "NOTE" in the whole PEP, and it 
doesn't seem to add anything.  This sentence would read exactly the same if 
that word were deleted.  Without more clarity on the required execution context 
of the callbacks, this claim might not actually be true anyway; Deferred 
callbacks can only be invoked in the main reactor thread in Twisted.  But even 
if it is perfectly possible, why leave so much of the adapter implementation up 
to the imagination?  If it's important enough to mention, why not have a 
reference to such an adapter in the reference Futures implementation, since it 
*should* be fairly trivial to write?

The fact that add_done_callback is implemented using a set is weird, since it 
means you can't add the same callback more than once.  The set implementation 
also means that the callbacks get called in a semi-random order, potentially 
creating even _more_ hard-to-debug order of execution issues than you'd 
normally have with futures.  And I think that this documentation will be 
unclear to a lot of novice developers: many people have trouble with the idea 
that "a = Foo(); b = Foo(); a.bar_method != b.bar_method", but "import 
foo_module; foo_module.bar_function == foo_module.bar_function".

It's also weird that you can remove callbacks - what's the use case?  Deferreds 
have no callback-removal mechanism and nobody has ever complained of the need 
for one, as far as I know.  (But lots of people do add the same callback 
multiple times.)

I suggest having have add_done_callback, implementing it with a list so that 
callbacks are always invoked in the order that they're added, and getting rid 
of remove_done_callback.

futures._base.Executor isn't exposed publicly, but it needs to be.  The PEP 
kinda makes it sound like it is ("Executor is an abstract class...").  Plus, A 
third party library wanting to implement an executor of its own shouldn't have 
to copy and paste the implementation of Executor.map.

One minor suggestion on the "internal future methods" bit - something I wish 
we'd done with Deferreds was to put 'callback()' and 'addCallbacks()' on 
separate objects, so that it was very explicit whether you were on the emitting 
side of a Deferred or the consuming side.  That seems to be the case with these 
internal methods - they are not so much "internal" as they are for the producer 
of the Future (whether a unit test or executor) so you might want to put them 
on a different object that it's easy for the thing creating a Future() to get 
at but hard for any subsequent application code to fiddle with by accident.  
Off the top of my head, I suggest naming it "Invoker()".  A good way to do this 
would be to have an Invoker class which can't be instantiated (raises an 
exception from __init__ or somesuch), then a Future.create() method which 
returns an Invoker, which itself has a '.future' attribute.

Finally, why isn't this just a module on PyPI?  It doesn't seem like there's 
any particular benefit to making this a stdlib module and going through the 
whole PEP process - except maybe to prompt feedback like this :).  Issues like 
the ones I'm bringing up could be fixed pretty straightforwardly if it were 
just a matter of filing a bug on a small package, but fixing a stdlib module is 
a major undertaking.

___
Python-Dev

Re: [Python-Dev] PEP 3148 ready for pronouncement

[Brian Quinlan]

On May 23, 2010, at 2:44 PM, Glyph Lefkowitz wrote:



On May 22, 2010, at 8:47 PM, Brian Quinlan wrote:

Jesse, the designated pronouncer for this PEP, has decided to keep  
discussion open for a few more days.


So fire away!


As you wish!


I retract my request ;-)

The PEP should be consistent in its usage of terminology about  
callables.  It alternately calls them "callables", "functions", and  
"functions or methods".  It would be nice to clean this up and be  
consistent about what can be called where.  I personally like  
"callables".


Did you find the terminology confusing? If not then I propose not  
changing it.


But changing it in the user docs is probably a good idea. I like  
"callables" too.




The execution context of callable code is not made clear.   
Implicitly, submit() or map() would run the code in threads or  
processes as defined by the executor, but that's not spelled out  
clearly.


More relevant to my own interests, the execution context of the  
callables passed to add_done_callback and remove_done_callback is  
left almost completely to the imagination.  If I'm reading the  
sample implementation correctly, , it looks like in the multiprocessing implementation, the done  
callbacks are invoked in a random local thread.  The fact that they  
are passed the future itself *sort* of implies that this is the  
case, but the multiprocessing module plays fast and loose with  
object identity all over the place, so it would be good to be  
explicit and say that it's *not* a pickled copy of the future  
sitting in some arbitrary process (or even on some arbitrary machine).


The callbacks will always be called in a thread other than the main  
thread in the process that created the executor. Is that a strong  
enough contract?


This is really minor, I know, but why does it say "NOTE: This method  
can be used to create adapters from Futures to Twisted Deferreds"?   
First of all, what's the deal with "NOTE"; it's the only "NOTE" in  
the whole PEP, and it doesn't seem to add anything.  This sentence  
would read exactly the same if that word were deleted.  Without more  
clarity on the required execution context of the callbacks, this  
claim might not actually be true anyway; Deferred callbacks can only  
be invoked in the main reactor thread in Twisted.  But even if it is  
perfectly possible, why leave so much of the adapter implementation  
up to the imagination?  If it's important enough to mention, why not  
have a reference to such an adapter in the reference Futures  
implementation, since it *should* be fairly trivial to write?


I'm a bit surprised that this doesn't allow for better  
interoperability with Deferreds given this discussion:


At 02:36 PM 3/16/2010 -0700, Brian Quinlan wrote:

"""
From P.J Eby:

On Mar 7, 2010, at 11:56 AM, P.J. Eby wrote:


At 10:59 AM 3/7/2010 -0800, Jeffrey Yasskin wrote:
Given a way to register "on-done" callbacks with the future, it  
would be straightforward to wait for a future without blocking, too.


Yes, and with a few more additions besides that one, you might be  
on the way to an actual competitor for Deferreds.  For example:  
retry support, chaining, logging, API for transparent result  
processing, coroutine support, co-ordination tools like locks,  
sempaphores and queues, etc.


OK, but lets just think about making the APIs compatible e.g. you  
have some code that uses Futures and now you want to integrate it  
with some code that uses Deferreds.


I think Jeff's suggestion of having a completion callback on Futures  
would make it possible to write a Future-to-Deferred adapter. Is  
that correct?


As long as the callback signature included a way to pass in an error,  
then yes, that'd probably be sufficient.

"""

If add_done_callback doesn't help with twisted interoperability then  
I'd suggest removing it to allow for something that may be more useful  
to be added later.




The fact that add_done_callback is implemented using a set is weird,  
since it means you can't add the same callback more than once.  The  
set implementation also means that the callbacks get called in a  
semi-random order, potentially creating even _more_ hard-to-debug  
order of execution issues than you'd normally have with futures.   
And I think that this documentation will be unclear to a lot of  
novice developers: many people have trouble with the idea that "a =  
Foo(); b = Foo(); a.bar_method != b.bar_method", but "import  
foo_module; foo_module.bar_function == foo_module.bar_function".


It's also weird that you can remove callbacks - what's the use  
case?  Deferreds have no callback-removal mechanism and nobody has  
ever complained of the need for one, as far as I know.  (But lots of  
people do add the same callback multiple times.)


I suggest having have add_done_callback, implementing it with a list  
so that callbacks