[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Fix false positives in circular import detection with 
from-imports

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[doublex]

doublex  added the comment:

Example code (fails):


import os, concurrent.futures

def parallel_callback( arg ):
return os.getpid()

def parallel( *args ):
def thread_callback( param ):
with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor:
future = executor.submit( parallel_callback, param )
pid = future.result()
print( 'pid:', pid )
return pid
with concurrent.futures.ThreadPoolExecutor(max_workers=len(args)) as 
executor:
future = executor.map( thread_callback, args )
results = list(future)
print( 'DONE' )

parallel( 1, 2, 3 )

--
nosy: +doublex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +aeros

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Cebtenzzre]

Change by Cebtenzzre :


--
nosy: +cebtenzzre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Atsuo Ishimoto]

Atsuo Ishimoto  added the comment:

Some my observation at 
https://discuss.python.org/t/differences-between-3-8-and-3-9-in-importing-module/5520
 .

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Atsuo Ishimoto]

Atsuo Ishimoto  added the comment:

multiprocessing module used to work multithread environment until 
https://bugs.python.org/issue35943 merged.

I guess we can make multiprocessing thread-safe again if we move local imports 
to global.
Does it make sense?

--
nosy: +ishimoto

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Irit Katriel]

Irit Katriel  added the comment:

I think this is no a bug, on the basis that multiprocessing.Pool is not 
thread-safe.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Irit Katriel]

Irit Katriel  added the comment:

I see the same in Python 3.10 on windows 10.

If I change the relative imports to absolute imports in a couple of functions 
in multiprocessing.context as below, the attached (pool_error_on_3.9.py) script 
not longer raises the exception.

def SimpleQueue(self):
'''Returns a queue object'''
from multiprocessing.queues import SimpleQueue
return SimpleQueue(ctx=self.get_context())

def Pool(self, processes=None, initializer=None, initargs=(),
 maxtasksperchild=None):
'''Returns a process pool object'''
from multiprocessing.pool import Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
context=self.get_context())

--
nosy: +iritkatriel
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41567] multiprocessing.Pool from concurrent threads failure on 3.9.0rc1

[Carl Drougge]

New submission from Carl Drougge :

If several threads try to start a multiprocessing.Pool at the same time when no 
pool has been started before this often fails with an exception like this (the 
exact import varies):

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/tmp/py3.9.0rc1/lib/python3.9/threading.py", line 950, in 
_bootstrap_inner
self.run()
  File "/tmp/py3.9.0rc1/lib/python3.9/threading.py", line 888, in run
self._target(*self._args, **self._kwargs)
  File "/tmp/py3.9.0rc1/lib/python3.9/multiprocessing/context.py", line 118, in 
Pool
from .pool import Pool
ImportError: cannot import name 'Pool' from partially initialized module 
'multiprocessing.pool' (most likely due to a circular import) 
(/tmp/py3.9.0rc1/lib/python3.9/multiprocessing/pool.py)

This happens even if Pool was imported before starting the threads and is new 
in 3.9. It's easy to work around by starting a pool in the main thread before 
starting the other threads.

I have attached a minimal example that triggers it. Tested on Debian stable and 
FreeBSD 11.3.

--
components: Library (Lib)
files: pool_error_on_3.9.py
messages: 375542
nosy: drougge
priority: normal
severity: normal
status: open
title: multiprocessing.Pool from concurrent threads failure on 3.9.0rc1
type: behavior
versions: Python 3.9
Added file: https://bugs.python.org/file49401/pool_error_on_3.9.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com