Masti created this task.
Masti added a project: Pywikibot.
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper.
TASK DESCRIPTION
while running pywikibot on pl.wikipedia I quite often encounter code crashes
after page generator encounters 502 Server Error from wikimedia API
**What happens?**:
script throws exception and crashes
Retrieving 50 pages from wikipedia:pl.
ERROR: Traceback (most recent call last):
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 502 Server Error: Server Hangup
Traceback (most recent call last):
File "/home/masti/pw/core/pwb.py", line 40, in <module>
sys.exit(main())
File "/home/masti/pw/core/pwb.py", line 36, in main
runpy.run_path(str(path), run_name='__main__')
File "/usr/lib/python3.10/runpy.py", line 289, in run_path
return _run_module_code(code, init_globals, run_name,
File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 521, in
<module>
main()
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 505, in main
if not execute():
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 492, in
execute
run_python_file(filename, script_args, module)
File "/home/masti/pw/core/pywikibot/scripts/wrapper.py", line 149, in
run_python_file
exec(compile(source, filename, 'exec', dont_inherit=True),
File "masti/ms-contains.py", line 433, in <module>
main()
File "masti/ms-contains.py", line 427, in main
bot.run() # guess what it does
File "masti/ms-contains.py", line 147, in run
for page in self.generator:
File "/home/masti/pw/core/pywikibot/pagegenerators/__init__.py", line 660,
in PreloadingGenerator
yield from site.preloadpages(group, groupsize=groupsize,
File "/home/masti/pw/core/pywikibot/site/_generators.py", line 202, in
preloadpages
for pagedata in rvgen:
File "/usr/lib/python3.10/_collections_abc.py", line 330, in __next__
return self.send(None)
File "/home/masti/pw/core/pywikibot/tools/collections.py", line 279, in send
return next(self._started_gen)
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 781, in
generator
yield from super().generator
File "/home/masti/pw/core/pywikibot/data/api/_generators.py", line 607, in
generator
self.data = self.request.submit()
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 993, in
submit
response, use_get = self._http_request(use_get, uri, body, headers,
File "/home/masti/pw/core/pywikibot/data/api/_requests.py", line 684, in
_http_request
response = http.request(self.site, uri=uri,
File "/home/masti/pw/core/pywikibot/comms/http.py", line 283, in request
r = fetch(baseuri, headers=headers, **kwargs)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 457, in fetch
callback(response)
File "/home/masti/pw/core/pywikibot/comms/http.py", line 354, in
error_handling_callback
raise ServerError(
pywikibot.exceptions.ServerError: 502 Server Error: Server Hangup
CRITICAL: Exiting due to uncaught exception ServerError: 502 Server Error:
Server Hangup
**What should have happened instead?**:
wait for server to come back as the errors are temporary
**Software version** (on `Special:Version` page; skip for WMF-hosted wikis
like Wikipedia):
Pywikibot: [https] masti01-pywikibot.git (04383ba
<https://phabricator.wikimedia.org/rPWBC04383baed7b64bb9ca0152e8147df8ff80d269b0>,
g18666, 2024/05/06, 16:04:47, master)
Release version: 9.2.0.dev2
packaging version: 24.0
mwparserfromhell version: 0.6.6
wikitextparser version: n/a
requests version: 2.31.0
cacerts:
/home/masti/pw/core/venv/lib/python3.10/site-packages/certifi/cacert.pem
certificate test: ok
Python: 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0]
**Other information** (browser name/version, screenshots, etc.):
TASK DETAIL
https://phabricator.wikimedia.org/T364393
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Masti
Cc: Aklapper, Masti, pywikibot-bugs-list, mevo, PotsdamLamb, Jyoo1011,
JohnsonLee01, SHEKH, Dijkstra, Khutuck, Zkhalido, Viztor, Wenyi, Tbscho, MayS,
Mdupont, JJMC89, Dvorapa, Altostratus, Avicennasis, mys_721tx, Xqt, jayvdb,
Alchimista
_______________________________________________
pywikibot-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
