Andrawaag created this task.
Andrawaag added a project: Wikidata.
Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION
When we run bots we sometime run into api requests to backup for a few
seconds. Our bots respect that request and will pause and resume as the
suggested seconds. However, often the issue persists which leads to many
repeated attempts. See for example snippet from a recent run:
2020-02-11 22:29:03.462709: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:09.111319: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:14.735034: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:20.354678: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:25.998749: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:31.629414: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:37.255963: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:42.869407: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:48.503712: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:54.129682: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:29:59.730619: maxlag. sleeping for 5.466666666666667 seconds
2020-02-11 22:30:05.355369: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:11.371665: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:17.402557: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:23.387781: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:29.375381: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:35.377794: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:41.355254: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:47.319589: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:53.293700: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:30:59.300540: maxlag. sleeping for 5.816666666666666 seconds
2020-02-11 22:31:05.272313: maxlag. sleeping for 6.233333333333333 seconds
2020-02-11 22:31:11.697159: maxlag. sleeping for 6.233333333333333 seconds
2020-02-11 22:31:18.098883: maxlag. sleeping for 6.233333333333333 seconds
2020-02-11 22:31:24.493465: maxlag. sleeping for 6.233333333333333 seconds
We have currently have a limit of 25 attempts afterwhich an exception is sent
and the bot is terminated.
I am a bit surprised that the request to backoff does not become more
aggressive with incremental steps, i.e. staying around 5-6 seconds.
However the termination based on this approach disrupts the overall workflow,
so we are looking into different approaches.
One would be to increase the value in the number of max attempts from 25 to
100 or even 1000.
Or to exponentailly increase the waiting time per each iteration.
But I am wondering if the API benefits from these repetative attempts to
connect to the API.
Another approach would be to do not repeat, but to have some "Now is not a
good time", which our bots could consult before commensing with a bot run. This
way, we do not send unneccesary attempts.
Is this possible?
TASK DETAIL
https://phabricator.wikimedia.org/T245138
WORKBOARD
https://phabricator.wikimedia.org/project/board/71/
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Andrawaag
Cc: Aklapper, Andrawaag, darthmon_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic,
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude,
Mbch331
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
