Xqt created this task.
Xqt added a project: Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION
**Feature summary**:
Process wait cycles in a queue.
**Use case**:
API requests may lead to a timeout and the same request is made after a
delay. The wait cycles are determined by the following config settings:
# Maximum number of times to retry an API request before quitting.
max_retries = 15
# Minimum time to wait before resubmitting a failed API request.
retry_wait = 5
# Maximum time to wait before resubmitting a failed API request.
retry_max = 120
The delay time start wit 5 seconds and doubles for each retry but no longer
than `retry_max` seconds. For 15 retries the scripts may wait for 22 minutes in
total until it terminates (or continue after the request is successfull). In
meantime the script does just idle or in case of `-async` option it processes
additional pages and puts them to the page_put_queue until the queue is full.
**Proposal**
Instead of waiting for a successfull request, remove the current request from
the queue and add it to delayed queue with a timestamp for retrying (or use a
priority queue to handle such an issue).
**Benefits**:
Requests should not block further requests if they are independent. The
script can run faster and blocking request are handled at the end.
TASK DETAIL
https://phabricator.wikimedia.org/T342147
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Xqt
Cc: Aklapper, Xqt, pywikibot-bugs-list, PotsdamLamb, Jyoo1011, JohnsonLee01,
SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS, Mdupont,
JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, jayvdb, Masti, Alchimista
_______________________________________________
pywikibot-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
