[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Tin Tvrtković]

Tin Tvrtković  added the comment:

I was trying to instrument one of our executors to expose some metrics, so I 
tried using the value of the semaphore as the number of idle threads and I 
noticed it was way too large. If this was fixed getting these metrics would be 
easier.

--
nosy: +tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Antoine Pitrou]

Antoine Pitrou  added the comment:

To be clear, this probably doesn't have any actual consequence, since the right 
number of threads is launched anyway (at least in the example). But it's 
probably worth making the implementation less quirky (also, the semaphore's 
internal counter *might* overflow at some point?).

--
components: +Library (Lib) -asyncio
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Antoine Pitrou]

Change by Antoine Pitrou :


--
versions: +Python 3.11 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Bennie Swart]

Change by Bennie Swart :


--
keywords: +patch
pull_requests: +24869
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/26265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44188] ThreadPoolExecutor unbalanced semaphore count

[Bennie Swart]

New submission from Bennie Swart :

The concurrent.futures.ThreadPoolExecutor class, which is the default asyncio 
executor, introduced the _idle_semaphore field in version 3.8 in order to track 
idle threads so they can be reused before increasing the pool size 
unnecessarily.
This semaphore counter becomes unbalanced when the thread pool is 
over-saturated, as can be seen in the file provided. This is due to workers 
always incrementing the count after finishing a job, whereas the executor only 
decrements the count if it is already greater than 0.
This seems to be a logic bug unrelated to the running environment and 
introduced since python 3.8.

--
components: asyncio
files: bug.py
messages: 394017
nosy: asvetlov, bennieswart, yselivanov
priority: normal
severity: normal
status: open
title: ThreadPoolExecutor unbalanced semaphore count
type: behavior
versions: Python 3.8
Added file: https://bugs.python.org/file50054/bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com