New submission from Erick <[email protected]>:
Currently the only way to use a class other than `threading.Thread` with
`concurrent.futures.ThreadPoolExecutor` is to extend `ThreadPoolExecutor` and
override the private method `_adjust_thread_count()`.
For example, suppose I have a class that applies some custom logic when running
code in a new thread:
```
class MyThread(threading.Thread):
def run(self):
with some_important_context():
super().run()
```
Using this class with `ThreadPoolExecutor` requires me to write the following
code:
```
class MyThreadPoolExecutor(ThreadPoolExecutor):
def _adjust_thread_count(self):
# if idle threads are available, don't spin new threads
if self._idle_semaphore.acquire(timeout=0):
return
# When the executor gets lost, the weakref callback will wake up
# the worker threads.
def weakref_cb(_, q=self._work_queue):
q.put(None)
num_threads = len(self._threads)
if num_threads < self._max_workers:
thread_name = '%s_%d' % (self._thread_name_prefix or self,
num_threads)
t = MyThread(name=thread_name, target=_worker,
args=(weakref.ref(self, weakref_cb),
self._work_queue,
self._initializer,
self._initargs))
t.start()
self._threads.add(t)
_threads_queues[t] = self._work_queue
with MyThreadPoolExecutor(max_workers=1) as executor:
future = executor.submit(pow, 323, 1235)
print(future.result())
```
That's a bummer, because now I have to maintain this method if there are
upstream fixes/changes. I also can't count on it existing in the future, since
it's a private method.
In other words, `ThreadPoolExecutor` is not composable, and extending it to use
a custom `Thread` class is neither safe nor maintainable.
Here's what I'd like to be able to do instead:
```
with ThreadPoolExecutor(max_workers=1, thread_class=MyThread) as executor:
future = executor.submit(pow, 323, 1235)
print(future.result())
```
----------
components: Library (Lib)
messages: 403007
nosy: erickpeirson
priority: normal
pull_requests: 27040
severity: normal
status: open
title: concurrent.future.ThreadPoolExecutor should parameterize class used for
threads
versions: Python 3.11
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue45339>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
