[issue16307] multiprocess.pool.map_async callables not working

2012-10-27 Thread Hynek Schlawack 

Hynek Schlawack added the comment:

LGTM.

Presuming you want to submit more patches in future, please take the time to 
sign a Python contributor agreement: http://www.python.org/psf/contrib/ . 
You'll get a pretty star next to your name in the bug tracker in return. ;)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-27 Thread Roundup Robot 

Roundup Robot added the comment:

New changeset 1bccec4ff980 by Hynek Schlawack in branch '3.3':
#16307: Fix multiprocessing.Pool.map_async not calling its callbacks
http://hg.python.org/cpython/rev/1bccec4ff980

--
nosy: +python-dev

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-27 Thread Hynek Schlawack 

Hynek Schlawack added the comment:

Applied. Thank you for your contribution!

--
resolution:  - fixed
stage: patch review - committed/rejected
status: open - closed

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-26 Thread Hynek Schlawack 

Hynek Schlawack added the comment:

Thanks for taking the time! I remember my frustrations when trying to grok how 
the mp test suite works. :)

A small nit-pick first: you have a lot of extra white space in your patches. 
Just run 'make patchcheck' first, that should warn you about that.

Not sure, but the tests look rather complex to me. I’d humbly propose the the 
simplified test attached, which also ensures that error_callback get only 
called on errors and callback on success.

Opinions?

--
Added file: http://bugs.python.org/file27724/map-async-fix-with-tests.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-26 Thread Janne Karila 

Janne Karila added the comment:

Otherwise I agree, but what if one of the callbacks is not called? The test 
case would fail on assertEqual(2, len(call_args)) but you wouldn't know which 
callback is the problem.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-26 Thread Hynek Schlawack 

Hynek Schlawack added the comment:

True, it makes sense to push this assert to the end.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-26 Thread Janne Karila 

Janne Karila added the comment:

The length assertion protects the test from breaking with IndexError. How about 
this? Though, it looks like two test cases to me.

+self.pool.map_async(int, ['1'],
+callback=call_args.append,
+error_callback=call_args.append).wait()
+self.assertEqual(1, len(call_args))
+self.assertEqual([1], call_args[0])
+self.pool.map_async(int, ['a'],
+callback=call_args.append,
+error_callback=call_args.append).wait()
+self.assertEqual(2, len(call_args))
+self.assertIsInstance(call_args[1], ValueError)

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-25 Thread Janne Karila 

Janne Karila added the comment:

I'm working on the test. It keeps failing...

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-25 Thread Janne Karila 

Janne Karila added the comment:

Three test cases added. 

The tricky part in writing the tests was to realize that when the tests are run 
with Manager, the callback goes through a proxy object.

--
Added file: http://bugs.python.org/file27711/test_callback.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Bbb 

New submission from Bbb:

When using map_async() my callbacks don't get called back.  When using 
apply_async() they do get called.

CODE:
from multiprocessing import Pool,TimeoutError
from time import sleep

servers=[s1,s2,s3,s4,s5,s6]
blah = no callback

def f(x):
print(start f( + x + ))
sleep(5)
print(end   f( + x + ))
return did  + x

def mycallback(x):
global blah
blah = called back
print(My callback  + str(x))



def myerrorcallback(r):
print(My errorcallback  + str(r))


if __name__ == '__main__':
pool = Pool(processes=7)
results = pool.map_async(f, servers,  callback=mycallback, 
error_callback=myerrorcallback)
print(results.get(timeout=11))
pool.close()
pool.join()
print(blah)

OUTPUT:
D:\pythonf.py
start f(s1)
start f(s2)
start f(s3)
start f(s4)
start f(s6)
start f(s5)
end   f(s1)
end   f(s2)
end   f(s3)
end   f(s4)
end   f(s5)
end   f(s6)
['did s1', 'did s2', 'did s3', 'did s4', 'did s5', 'did s6']
no callback

...whereas replacing the code with this:
   with Pool(processes=7) as pool: # start 4 worker processes
for server in servers:
r = pool.apply_async(f, (server,),  callback=mycallback, 
error_callback=myerrorcallback)
pool.close()
pool.join()
print (blah)


GIVES:
D:\python\f2.py
start f(s1)
start f(s2)
start f(s3)
start f(s4)
start f(s5)
start f(s6)
end   f(s2)
end   f(s1)
My callback did s2
My callback did s1
end   f(s4)
My callback did s4
end   f(s3)
My callback did s3
end   f(s5)
My callback did s5
end   f(s6)
My callback did s6
called back


This is v3.3 on Windows XP.

--
components: Library (Lib)
messages: 173653
nosy: Bbb
priority: normal
severity: normal
status: open
title: multiprocess.pool.map_async callables not working
type: behavior
versions: Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Janne Karila 

Janne Karila added the comment:

Indeed, the current implementation map_async simply ignores the callback 
arguments:

def map_async(self, func, iterable, chunksize=None, callback=None,
error_callback=None):
'''
Asynchronous version of `map()` method.
'''
return self._map_async(func, iterable, mapstar, chunksize)

--
nosy: +Janne.Karila

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Janne Karila 

Janne Karila added the comment:

Here's a patch that adds the missing arguments to the call to _map_async.
The bug seems to originate from enhancement #12708 that added starmap_async to 
3.3.

--
keywords: +patch
Added file: http://bugs.python.org/file27690/callback.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Antoine Pitrou 

Changes by Antoine Pitrou pit...@free.fr:


--
nosy: +hynek
stage:  - patch review
versions: +Python 3.4

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Hynek Schlawack 

Hynek Schlawack added the comment:

Could you add a test please? Thanks!

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16307] multiprocess.pool.map_async callables not working

2012-10-24 Thread Richard Oudkerk 

Changes by Richard Oudkerk shibt...@gmail.com:


--
nosy: +sbt

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue16307
___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com