On 17 Jan 2010, at 01:44, Anh Hai Trinh wrote:
2. Do several different operations on different data e.g.
parallelizing code
like this:
db = setup_database(host, port)
data = parse_big_xml_file(request.body)
save_data_in_db(data, db)
I'm trying to get a handle on how streams accommodates the second
case. With
futures, I would write something like this:
db_future = executor.submit(setup_database, host, port)
data_future = executor.submit(parse_big_xml_file, data)
# Maybe do something else here.
wait(
[db_future, data_future],
timeout=10,
# If either function raises then we can't complete the operation
so
# there is no reason to make the user wait.
return_when=FIRST_EXCEPTION)
db = db_future.result(timeout=0)
data = data.result(timeout=0)
save_data_in_db(data, db)
For this kind of scenario, I feel `futures` and friends are not
needed. My solution is to explicit use different threads for different
operations then use join() thread to wait for a particular operation.
Threading concurrency means memory is shared and thread.join() can be
used to synchronize events.
It is definitely true that you can roll your own implementation using
threads but the purpose of the futures library is to make that
unnecessary.
Generally, I would be doubtful about any library that support
parallelization of code that "do several different operations on
different data". One could have put it as "write concurrent programs",
to which the answer must be a complete concurrency model: threading,
multiprocessing, Erlang, Goroutines and CSP channels, etc.
I don't understand your doubts. To me the example that I gave is
simple and useful.
Cheers,
Brian
_______________________________________________
stdlib-sig mailing list
stdlib-sig@python.org
http://mail.python.org/mailman/listinfo/stdlib-sig
