| Dalba added a comment. |
@Magul Well, I think it would be better if we increased the timeouts as low as necessary. Especially that in multi-threaded environments these timeouts make the application completely unresponsive. When there is actually a problem with the connection somewhere an early timeout could be a bless. Users could actually depend on these timeouts, trying to catch and skip them or just using the built-in retry mechanism. Increasing the timeouts could greatly increase the overall retry times. An early timeout when there is actually no more hope can be a blessing...
On the other hand I'm not sure that increasing the timeout will resolve the problem here. I tried to reproduce the issue (many times) and only a few of my attempts where successful. All of which ended up with a 503 error instead of finishing the request:
WARNING: Http response status 503 WARNING: Non-JSON response received from server wikipedia:en; the server may be down. Set uclimit = ['2'] WARNING: Waiting 5 seconds before retrying. WARNING: Http response status 503 WARNING: Non-JSON response received from server wikipedia:en; the server may be down. Set uclimit = ['1'] 246.508s Error Traceback (most recent call last): File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 1987, in submit result = json.loads(rawdata) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\__init__.py", line 319, in loads return _default_decoder.decode(s) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
During handling of the above exception, another exception occurred: Traceback (most recent call last): File "G:\programming\python\wiki\pywikibot-core\tests\site_tests.py", line 1493, in test_user_prefix_range total=5): File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2753, in __iter__ self.data = self.request.submit() File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2005, in submit self.wait() File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2197, in wait raise TimeoutError("Maximum retries attempted without success.") pywikibot.data.api.TimeoutError: Maximum retries attempted without success. WARNING: Http response status 503 WARNING: Non-JSON response received from server wikipedia:en; the server may be down. Set uclimit = ['2'] WARNING: Waiting 5 seconds before retrying. 190.370s Process finished with exit code 0
(It seems that some caching mechanism is involved and you can only get these errors in the first attempt, then you'll have to wait a while, otherwise all subsequent retries will end up passing without error.)
tests: max_retries reduced from 25 to 1 WARNING: Http response status 503 WARNING: Non-JSON response received from server wikipedia:en; the server may be down. Set uclimit = ['2'] WARNING: Waiting 5 seconds before retrying. WARNING: Http response status 503 WARNING: Non-JSON response received from server wikipedia:en; the server may be down. Set uclimit = ['1'] 250.990s Error Traceback (most recent call last): File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 1987, in submit result = json.loads(rawdata) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\__init__.py", line 319, in loads return _default_decoder.decode(s) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\a\AppData\Local\Programs\Python\Python35\lib\json\decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "G:\programming\python\wiki\pywikibot-core\tests\site_tests.py", line 1516, in test_user_prefix_reverse reverse=True, total=5): File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2753, in __iter__ self.data = self.request.submit() File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2005, in submit self.wait() File "G:\programming\python\wiki\pywikibot-core\pywikibot\data\api.py", line 2197, in wait raise TimeoutError("Maximum retries attempted without success.") pywikibot.data.api.TimeoutError: Maximum retries attempted without success. Process finished with exit code 0
(The timeout was set to 300 seconds in both cases)
Cc: MtDu, Lokal_Profil, Dalba, gerritbot, valhallasw, XZise, Aklapper, jayvdb, Zppix, Xqt, pywikibot-bugs-list, Ramalepe, Liugev6, Magul, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
