On Fri, Dec 10, 2010 at 4:42 AM, Astan Chee <astan.c...@gmail.com> wrote:
> On Fri, Dec 10, 2010 at 1:37 AM, Peter Otten <__pete...@web.de> wrote:
>>
>> I can't replicate the crash. However, your problem looks like there is a
>> ready-to-use solution:
>>
>>
>> http://docs.python.org/library/multiprocessing.html#using-a-pool-of-workers
>>
>> --
>> http://mail.python.org/mailman/listinfo/python-list
>
>
> Pool.map doesn't seem to be able to support multiple argument functions
> which is what I'm trying to do here. Any other suggestions?
> Thanks again
1. Post the real code you're using, and
2. Put the arguments you want in a tuple and pass that. As an example,
let's say I have the following function:
def my_func(x, y, z):
return x + y * z
you could rewrite this as:
def my_func(*args):
return args[0] + args[1] * args[2]
Here's a worked-out example:
#! /usr/bin/env python3
import multiprocessing
def my_func(x, y, z):
return x + y * z
def my_func_wrapper(t):
return my_func(*t)
# assume we can get an iterable over each argument
xs = [1, 2, 3, 4]
ys = [5, 6, 7, 8]
zs = [9, 1, 2, 3]
# set up the pool to match the number of CPUs
num_processes = multiprocessing.cpu_count()
pool = multiprocessing.Pool(num_processes)
#will call my_func(1, 5, 9), my_func(2, 6, 1), etc.
results = pool.map(my_func_wrapper, zip(xs, ys, zs))
print(results)
Interesting factoid: you can't seem to use lambda or a decorator to do
this, which would have been my first instinct. Pickle apparently
chokes, although marshall wouldn't.
Geremy Condra
--
http://mail.python.org/mailman/listinfo/python-list
