jenkins-bot has submitted this change and it was merged. ( https://gerrit.wikimedia.org/r/371602 )
Change subject: Handle sparql http timeout issue ...................................................................... Handle sparql http timeout issue This is done by catching it, waiting, then trying again. On the second try the sparql endpoint has already generated the result so the reply is fast. A proposal for an upstream fix in pywikibot has been made in https://gerrit.wikimedia.org/r/371697 but this is a stop-gap messure. Bug: T172690 Change-Id: I7c9d8a213a595ac9adac335020cd1179c199fdda --- M erfgoedbot/update_database.py 1 file changed, 10 insertions(+), 1 deletion(-) Approvals: Jean-Frédéric: Looks good to me, approved jenkins-bot: Verified diff --git a/erfgoedbot/update_database.py b/erfgoedbot/update_database.py index 7d86e93..c459085 100755 --- a/erfgoedbot/update_database.py +++ b/erfgoedbot/update_database.py @@ -15,6 +15,9 @@ import warnings import datetime import urlparse +import time + +from requests.exceptions import Timeout import pywikibot import pywikibot.data.sparql @@ -482,7 +485,13 @@ ) # print sparql_query sq = pywikibot.data.sparql.SparqlQuery() - query_result = sq.select(sparql_query, full_data=True) + try: + query_result = sq.select(sparql_query, full_data=True) + except Timeout: + pywikibot.output('Sparql endpoint being slow, giving it a moment...') + time.sleep(10) + query_result = sq.select(sparql_query, full_data=True) + for resultitem in query_result: process_monument_wikidata(resultitem, countryconfig, conn, cursor) -- To view, visit https://gerrit.wikimedia.org/r/371602 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: merged Gerrit-Change-Id: I7c9d8a213a595ac9adac335020cd1179c199fdda Gerrit-PatchSet: 2 Gerrit-Project: labs/tools/heritage Gerrit-Branch: wikidata Gerrit-Owner: Lokal Profil <[email protected]> Gerrit-Reviewer: Jean-Frédéric <[email protected]> Gerrit-Reviewer: Multichill <[email protected]> Gerrit-Reviewer: jenkins-bot <> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
