New issue 2830: multiprocessing fails with "expected integer, got NoneType 
object"
https://bitbucket.org/pypy/pypy/issues/2830/multiprocessing-fails-with-expected

Mike Auty:

Hi there,

I haven't been able to minimize the test condition, and I can only provide the 
rough outline of the code, but hopefully the location of the error will help 
identify it.  The exception traceback is:

```
#!python

Traceback (most recent call last):
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/pool.py", line 119, in 
worker
    result = (True, func(*args, **kwds))
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/pool.py", line 44, in 
mapstar
    return list(map(*args))
  File "/usr/lib64/pypy3/lib_pypy/_functools.py", line 80, in __call__
    return self._func(*(self._args + fargs), **fkeywords)
  File "/path/to/scan_test.py", line 226, in _scan_chunk
    progress.value = chunk_end
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/managers.py", line 1034, 
in set
    return self._callmethod('set', (value,))
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/managers.py", line 716, 
in _callmethod
    conn.send((self._id, methodname, args, kwds))
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 206, 
in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 404, 
in _send_bytes
    self._send(header + buf)
  File "/usr/lib64/pypy3/lib-python/3/multiprocessing/connection.py", line 368, 
in _send
    n = write(self._handle, buf)
TypeError: expected integer, got NoneType object

```

In this instance, progress.value is `multiprocessing.Manager().Value("Q", 0)` 
and the multiprocessing code looks like:

```
#!python

                with multiprocessing.Pool() as pool:
                    result = pool.map_async(scan_chunk, scan_iterator())
                    while not result.ready():
                        result.wait(0.1)
                    for value in result.get():
                        yield from value
```

Some information that might help is that if I insert a print statement in the 
scan_chunk method, it ends up running fine, suggesting it's a timing condition 
of some kind.  It looks as though the handle's been closed, but I don't know 
enough about the code to figure out why.  Please let me know if you need more 
information and I'd be happy to provide to provide it if I'm allowed.  I'm also 
happy to test patches against 6.0.0 if that'd help...

Pypy --version:
Python 3.5.3 (fdd60ed87e941677e8ea11acf9f1819466521bf2, Apr 27 2018, 14:54:57)
[PyPy 6.0.0 with GCC 7.3.0]


_______________________________________________
pypy-issue mailing list
pypy-issue@python.org
https://mail.python.org/mailman/listinfo/pypy-issue

Reply via email to