> 2. Do several different operations on different data e.g. parallelizing code > like this: > > db = setup_database(host, port) > data = parse_big_xml_file(request.body) > save_data_in_db(data, db) > > I'm trying to get a handle on how streams accommodates the second case. With > futures, I would write something like this: > > db_future = executor.submit(setup_database, host, port) > data_future = executor.submit(parse_big_xml_file, data) > # Maybe do something else here. > wait( > [db_future, data_future], > timeout=10, > # If either function raises then we can't complete the operation so > # there is no reason to make the user wait. > return_when=FIRST_EXCEPTION) > > db = db_future.result(timeout=0) > data = data.result(timeout=0) > save_data_in_db(data, db)
For this kind of scenario, I feel `futures` and friends are not needed. My solution is to explicit use different threads for different operations then use join() thread to wait for a particular operation. Threading concurrency means memory is shared and thread.join() can be used to synchronize events. Generally, I would be doubtful about any library that support parallelization of code that "do several different operations on different data". One could have put it as "write concurrent programs", to which the answer must be a complete concurrency model: threading, multiprocessing, Erlang, Goroutines and CSP channels, etc. Cheers, -- // aht http://blog.onideas.ws _______________________________________________ stdlib-sig mailing list stdlib-sig@python.org http://mail.python.org/mailman/listinfo/stdlib-sig
