I am very happy with those changes. I think deadlock should be addressed before the first release as it changes the detailed semantics of some of the operations, but you've promised to do that, so cool. :)
I think it's fine to leave the embedding of Deferred-like things into futures and the embedding of futures into Deferred-like things until a later release. I expect it to be requested, but I don't think it'll be hard to add later. On Thu, Nov 12, 2009 at 6:38 PM, Brian Quinlan <br...@sweetapp.com> wrote: > Hey all, > > I compiled a summary of people's feedback (about technical issues - I agree > that the docs could be better but agreeing on the API seems like the first > step) and have some API change proposals. > > Here is a summary of the feedback: > - Use Twisted Deferreds rather than Futures > - The API too complex > - Make Future a callable and drop the .result()/.exception() methods > - Remove .wait() from Executor > - Make it easy to process results in the order of completion rather than in > the order that the futures were generated > - Executor context managers should wait until their workers complete before > exiting > - Extract Executor.map, etc. into separate functions/modules > - FutureList has too many methods or is not necessary > - Executor should have an easy way to produce a single future > - Should be able to wait on an arbitrary list of futures > - Should have a way of avoiding deadlock (will follow-up on this separately) > > Here is what I suggest as far as API changes (the docs suck, I'll polish > them when we reach consensus): > > FutureList is eliminated completely. > > Future remains unchanged - I disagree that Deferreds would be better, that > .exception() is not useful, and that .result() should be renamed .get() or > .__call__(). But I am easily persuadable :-) > > The Executor ABC is simplified to only contain a single method: > > def Executor.submit(self, fn, *args, **kwargs) : > > Submits a call for execution and returns a Future representing the pending > results of fn(*args, **kwargs) > > map becomes a utility function: > > def map(executor, *iterables, timeout=None) > > Equivalent to map(func, *iterables) but executed asynchronously and possibly > out-of-order. The returned iterator raises a TimeoutError if __next__() is > called and the result isn’t available after timeout seconds from the > original call to run_to_results(). If timeout is not specified or None then > there is no limit to the wait time. If a call raises an exception then that > exception will be raised when its value is retrieved from the iterator. > > wait becomes a utility function that can wait on any iterable of Futures: > > def wait(futures, return_when=ALL_COMPLETED) > > Wait until the given condition is met for the given futures. This method > should always be called using keyword arguments, which are: > > timeout can be used to control the maximum number of seconds to wait before > returning. If timeout is not specified or None then there is no limit to the > wait time. > > return_when indicates when the method should return. It must be one of the > following constants: > > NEXT_COMPLETED > NEXT_EXCEPTION > ALL_COMPLETED > > a new utility function is added that iterates over the given Futures and > returns the as they are completed: > > def itercompleted(futures, timeout=None): > > Returns an iterator that returns a completed Future from the given list when > __next__() is called. If no Futures are completed then __next__() is called > then __next__() waits until one does complete. Raises a TimeoutError if > __next__() is called and no completed future is available after timeout > seconds from the original call. > > The URL loading example becomes: > > import functools > import urllib.request > import futures > > URLS = ['http://www.foxnews.com/', > 'http://www.cnn.com/', > 'http://europe.wsj.com/', > 'http://www.bbc.co.uk/', > 'http://some-made-up-domain.com/'] > > def load_url(url, timeout): > return urllib.request.urlopen(url, timeout=timeout).read() > > with futures.ThreadPoolExecutor(50) as executor: > fs = [executor.submit(load_url, url, timeout=30) for url in URLS] > > for future in futures.itercompleted(fs): > if future.exception() is not None: > print('%r generated an exception: %s' % (url, future.exception())) > else: > print('%r page is %d bytes' % (url, len(future.result()))) > > What do you think? Are we moving in the right direction? > > Cheers, > Brian > > _______________________________________________ > stdlib-sig mailing list > stdlib-sig@python.org > http://mail.python.org/mailman/listinfo/stdlib-sig > -- Namasté, Jeffrey Yasskin http://jeffrey.yasskin.info/ _______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
