Lokal Profil has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/371602 )
Change subject: Handle sparql http timeout issue
......................................................................
Handle sparql http timeout issue
This is done by catching it, waiting, then trying again. On the second try
the sparql endpoint has already generated the result so the reply is fast.
Bug: T172690
Change-Id: I7c9d8a213a595ac9adac335020cd1179c199fdda
---
M erfgoedbot/update_database.py
1 file changed, 10 insertions(+), 1 deletion(-)
git pull ssh://gerrit.wikimedia.org:29418/labs/tools/heritage
refs/changes/02/371602/1
diff --git a/erfgoedbot/update_database.py b/erfgoedbot/update_database.py
index 7d86e93..c459085 100755
--- a/erfgoedbot/update_database.py
+++ b/erfgoedbot/update_database.py
@@ -15,6 +15,9 @@
import warnings
import datetime
import urlparse
+import time
+
+from requests.exceptions import Timeout
import pywikibot
import pywikibot.data.sparql
@@ -482,7 +485,13 @@
)
# print sparql_query
sq = pywikibot.data.sparql.SparqlQuery()
- query_result = sq.select(sparql_query, full_data=True)
+ try:
+ query_result = sq.select(sparql_query, full_data=True)
+ except Timeout:
+ pywikibot.output('Sparql endpoint being slow, giving it a moment...')
+ time.sleep(10)
+ query_result = sq.select(sparql_query, full_data=True)
+
for resultitem in query_result:
process_monument_wikidata(resultitem, countryconfig, conn, cursor)
--
To view, visit https://gerrit.wikimedia.org/r/371602
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I7c9d8a213a595ac9adac335020cd1179c199fdda
Gerrit-PatchSet: 1
Gerrit-Project: labs/tools/heritage
Gerrit-Branch: wikidata
Gerrit-Owner: Lokal Profil <lokal.pro...@gmail.com>
_______________________________________________
MediaWiki-commits mailing list
MediaWiki-commits@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
