jenkins-bot has submitted this change. (
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1221451?usp=email )
Change subject: IMPR: use RLock instead of Queue to signal async_manager
activity
......................................................................
IMPR: use RLock instead of Queue to signal async_manager activity
Use an RLock instead of an auxiliary Queue to signal whether
async_manager is currently processing a request.
With Python 3.14, threading.RLock gained a public locked() method.
This method is backported for older Python versions and available in
Pywikibot since 10.2 via the backports module (and since 6.2 via the
tools module).
The lock is held while a request is being executed and released
afterwards. _flush() can reliably determine whether the worker thread
is still busy by checking _page_put_queue_busy.locked() in addition to
page_put_queue.empty().
Changes:
- Replace page_put_queue_busy with an internal RLock
(_page_put_queue_busy)
- Use the lock to signal active request processing
- Remove the undocumented page_put_queue_busy Queue
This preserves the original synchronization semantics while reducing
overhead and avoiding the need for a second Queue.
Bug: T147178
Change-Id: I7f25e34cc807ccf61987431db691858d910dfd21
---
M ROADMAP.rst
M pywikibot/__init__.py
2 files changed, 16 insertions(+), 10 deletions(-)
Approvals:
jenkins-bot: Verified
Xqt: Looks good to me, approved
diff --git a/ROADMAP.rst b/ROADMAP.rst
index 0905cf3..4ed43d8 100644
--- a/ROADMAP.rst
+++ b/ROADMAP.rst
@@ -3,6 +3,7 @@
**Improvements**
+* Use :class:`backports.RLock` instead of Queue to signal async_manager
activity (:phab:`T147178`)
* Add :meth:`User.is_partial_blocked()<pywikibot.User.is_partial_blocked>` and
methods
:meth:`APISite.is_partial_blocked()<pywikibot.site._apisite.APISite.is_partial_blocked>`
to detect
partial blocks (:phab:`T412613`)
@@ -28,6 +29,7 @@
**Code cleanups**
+* The undocumented ``page_put_queue_busy`` was removed without deprecation
period.
* Dysfunctional :meth:`APISite.alllinks()
<pywikibot.site._generators.GeneratorsMixin.alllinks>` was removed.
(:phab:`T359427`, :phab:`T407708`)
diff --git a/pywikibot/__init__.py b/pywikibot/__init__.py
index 7dc246a..abdd4a1 100644
--- a/pywikibot/__init__.py
+++ b/pywikibot/__init__.py
@@ -32,6 +32,7 @@
WbTime,
WbUnknown,
)
+from pywikibot.backports import RLock
from pywikibot.bot import (
Bot,
CurrentPageBot,
@@ -353,8 +354,8 @@
exit_queue = None
if _putthread is not threading.current_thread():
- while _putthread.is_alive() and not (page_put_queue.empty()
- and page_put_queue_busy.empty()):
+ while _putthread.is_alive() and (not page_put_queue.empty()
+ or _page_put_queue_busy.locked()):
try:
_putthread.join(1)
except KeyboardInterrupt:
@@ -393,13 +394,14 @@
while True:
if not block and page_put_queue.empty():
break
+
(request, args, kwargs) = page_put_queue.get(block)
- page_put_queue_busy.put(None)
- if request is None:
- break
- request(*args, **kwargs)
- page_put_queue.task_done()
- page_put_queue_busy.get()
+
+ with _page_put_queue_busy:
+ if request is None:
+ break
+ request(*args, **kwargs)
+ page_put_queue.task_done()
def async_request(request: Callable, *args: Any, **kwargs: Any) -> None:
@@ -408,14 +410,16 @@
# ignore RuntimeError if start() is called more than once
with page_put_queue.mutex, suppress(RuntimeError):
_putthread.start()
+
page_put_queue.put((request, args, kwargs))
#: Queue to hold pending requests
page_put_queue: Queue = Queue(_config.max_queue_size)
-# queue to signal that async_manager is working on a request. See T147178.
-page_put_queue_busy: Queue = Queue(_config.max_queue_size)
+# RLock to signal that async_manager is working on a request. See T147178.
+_page_put_queue_busy: RLock = RLock()
+
# set up the background thread
_putthread = threading.Thread(target=async_manager,
name='Put-Thread', # for debugging purposes
--
To view, visit
https://gerrit.wikimedia.org/r/c/pywikibot/core/+/1221451?usp=email
To unsubscribe, or for help writing mail filters, visit
https://gerrit.wikimedia.org/r/settings?usp=email
Gerrit-MessageType: merged
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Change-Id: I7f25e34cc807ccf61987431db691858d910dfd21
Gerrit-Change-Number: 1221451
Gerrit-PatchSet: 5
Gerrit-Owner: Xqt <[email protected]>
Gerrit-Reviewer: Mpaa <[email protected]>
Gerrit-Reviewer: Xqt <[email protected]>
Gerrit-Reviewer: jenkins-bot
_______________________________________________
Pywikibot-commits mailing list -- [email protected]
To unsubscribe send an email to [email protected]
