[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[Josh Rosenberg]

Josh Rosenberg  added the comment:

Correct me if I'm wrong, but this isn't actually an issue for CPython, right? 
The GIL ensures that when a thread writes to _shutdown, nothing else is reading 
it until the GIL is released and acquired by a new thread (which synchronizes 
_shutdown).

It might conceivably be a problem on non-CPython interpreters if they have no 
other form of inter-thread synchronization involved; that said, the locking 
involved in the self.work_queue.get(block=True) call likely synchronizes by 
side-effect even there.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[Mark Lawrence]

Change by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[miles]

miles added the comment:

The attachment includes the patch file

--
keywords: +patch
nosy: +milesli
Added file: http://bugs.python.org/file38274/thread.py.patch

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23382
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[Mark Lawrence]

Mark Lawrence added the comment:

@Miles could you provide the code changes as a unified diff file, and if needed 
any changes to the test code as well, thanks.

--
components: +Library (Lib) -2to3 (2.x to 3.x conversion tool)
nosy: +BreamoreBoy
type:  - behavior
versions: +Python 2.7, Python 3.4, Python 3.5 -Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23382
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[miles]

New submission from miles:

Maybe can not shutdown ThreadPoolExecutor when call the method shutdown.

Though the variable of _shutdown is set to true in the method of shutdown, it 
may also reads the variable of _shutdown from cpu cache in the method of 
_worker, and the worst case is that it could see an out-of-date value of 
_shutdown forever. so need to acquire lock before reading the variable of 
_shutdown to make sure see an up-to-date value.


the following is the new code:



def _worker(executor_reference, work_queue):
try:
while True:
work_item = work_queue.get(block=True)
if work_item is not None:
work_item.run()
continue
executor = executor_reference()

shutdown = False
with executor._shutdown_lock.acquire():
shutdown = executor._shutdown

# Exit if:
#   - The interpreter is shutting down OR
#   - The executor that owns the worker has been collected OR
#   - The executor that owns the worker has been shutdown.
if _shutdown or executor is None or shutdown:
# Notice other workers
work_queue.put(None)
return
del executor
except BaseException:
_base.LOGGER.critical('Exception in worker', exc_info=True)




def shutdown(self, wait=True):
with self._shutdown_lock:
self._shutdown = True
self._work_queue.put(None)
if wait:
for t in self._threads:
t.join()

--
components: 2to3 (2.x to 3.x conversion tool)
messages: 235319
nosy: miles
priority: normal
severity: normal
status: open
title: Maybe can not shutdown ThreadPoolExecutor when call the method of 
shutdown
versions: Python 3.2

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23382
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[miles]

miles added the comment:

the attachment includes the new code

--
Added file: http://bugs.python.org/file38002/thread.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23382
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23382] Maybe can not shutdown ThreadPoolExecutor when call the method of shutdown

[miles]

miles added the comment:

The attachment includes the new code

--
Added file: http://bugs.python.org/file37997/thread.py

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue23382
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com