[Pywikipedia-bugs] [Maniphest] [Created] T144023: Maxlag lag incorrectly causing bot to crash with pywikibot.data.api.APIError

[Multichill]

Multichill created this task. Multichill added projects: Pywikibot-core, MediaWiki-API. Herald added subscribers: pywikibot-bugs-list, Aklapper.

TASK DESCRIPTION

Pywikibot checks if maxlag is returned ( https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/pywikibot/data/api.py;31c0234c3e0dd5f0740aa7d69d94651fbcb8bea8$2075 )

lagpattern = re.compile(r"Waiting for [\d.]+: (?P<lag>\d+) seconds? lagged")
....

if code == "maxlag":
    lag = lagpattern.search(info)
    if lag:
        pywikibot.log(
            u"Pausing due to database lag: " + info)
        self.site.throttle.lag(int(lag.group("lag")))
        continue

https://phabricator.wikimedia.org/diffusion/MW/browse/master/includes/api/ApiMain.php;cfa04ceb2b8878ab237d8dd2bdbd578972bce80e$1122

Parsing a string in the api output is just horrible, but seems to be the only way to retrieve the maxlag output. The output was an integer in seconds, but now a float so the regex doesn't match anymore. Simple fix is to update the regex to r"Waiting for [\d.]+: (?P<lag>\d+(\.\d+)?) seconds? lagged"

Bigger question is if we want to know the maximum lag and if we want to parse strings. I see several options here:


Just patch it up (short term solution, should do this whatever longer term we choose)
Stop using the lag info for backing up and instead just use the standard incremental backing off
Update the mediawiki api to include the lag in the output in a structured format so we don't have to do evil string parsing

TASK DETAIL

https://phabricator.wikimedia.org/T144023

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Multichill
Cc: Xqt, jayvdb, valhallasw, Aklapper, pywikibot-bugs-list, Multichill, Sethakill, Mdupont, JJMC89, dg711, Anomie, Legoktm

_______________________________________________
pywikibot-bugs mailing list
pywikibot-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs