Dalba has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/351241 )
Change subject: getInternetArchiveURL: Retry http.fetch if there is a
ConnectionError
......................................................................
getInternetArchiveURL: Retry http.fetch if there is a ConnectionError
The server sometimes returns an invalid response. Retrying should mitigate
the issue.
Bug: T164208
Change-Id: I19e287bacb5f6165ed50d8cb82e92b7880347066
---
M pywikibot/weblib.py
1 file changed, 17 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/pywikibot/core
refs/changes/41/351241/1
diff --git a/pywikibot/weblib.py b/pywikibot/weblib.py
index 4c27358..2b31fa1 100644
--- a/pywikibot/weblib.py
+++ b/pywikibot/weblib.py
@@ -11,6 +11,7 @@
import json
import sys
+from time import sleep
import xml.etree.ElementTree as ET
if sys.version_info[0] > 2:
@@ -18,7 +19,10 @@
else:
from urllib import urlencode
+from requests.exceptions import ConnectionError
+
from pywikibot.comms import http
+from pywikibot.config2 import max_retries, retry_wait
from pywikibot.tools import deprecated
@@ -42,7 +46,19 @@
query['timestamp'] = timestamp
uri = uri + urlencode(query)
- jsontext = http.fetch(uri).content
+
+ retry_count = 0
+ while retry_count <= max_retries:
+ try:
+ jsontext = http.fetch(uri).content
+ break
+ except ConnectionError as e:
+ error = e
+ retry_count += 1
+ sleep(retry_wait)
+ else:
+ raise error
+
if "closest" in jsontext:
data = json.loads(jsontext)
return data['archived_snapshots']['closest']['url']
--
To view, visit https://gerrit.wikimedia.org/r/351241
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I19e287bacb5f6165ed50d8cb82e92b7880347066
Gerrit-PatchSet: 1
Gerrit-Project: pywikibot/core
Gerrit-Branch: master
Gerrit-Owner: Dalba <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
