Jesse Noller <jnoller <at> gmail.com> writes: > We already have an implementation that spawns a > subprocess and then pushes the required state to the child. The > fundamental need for things to be pickleable *all the time* kinda > makes it annoying to work with. >
just a lurker here... but this topic hits home with me so thought I'd chime in. I'm a windows user and I would *love* to use multiprocessing a lot more because *in theory* it solves a lot of the problems I deal with very nicely (lot sof financial data number crunching). However, the pickling requirement makes it very very difficult to actually get any reasonably complex code to work properly with it. A lot of the time the functions I want to call in the spawned processes are actually fairly self contained and don't need most of the environment of the parent process shoved into it, so it's annoying that it fails because some data I don't even need in the child process can't be pickled. What about having an option to skip all the parent environment data pickling and require the user to manually invoke any imports that are needed in the target functions as the first step inside their target function? for example... def target_function(object_from_module_xyz): import xyz return object_from_module_xyz.do_something() and if I forgot to import all the stuff necessary for the arguments being passed into my function to work, then it's my own problem. Although maybe there is some obvious problem with this that I am not seeing. Anyway, just food for thought. - Matt _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com