Addshore added a comment.
addshore@stat1005:~$ cat /srv/analytics-wmde/graphite/log/daily.03.log |grep wikidata-sparql-instanceof |grep Q11173
2017-09-26 03:01:59 wikidata-sparql-instanceof Result for: Q11173 null
2017-09-27 03:02:07 wikidata-sparql-instanceof WikidataInstanceOf::doSparqlQuery: PREFIX wd: PREFIX wdt: SELECT (count(distinct(?s)) AS ?scount) WHERE {?s wdt:P31/wdt:P279* wd:Q11173} [{"http_code":"HTTP\/1.1 500 Server Error","Date":"Wed, 27 Sep 2017 03:02:07 GMT","Content-Type":"text\/plain","Content-Length":"3093","Connection":"keep-alive","Server":"nginx\/1.11.13","X-Served-By":"wdqs1004","Access-Control-Allow-Origin":"*","Vary":"Accept-Encoding","X-Varnish":"21878624, 304358865","Via":"1.1 varnish-v4, 1.1 varnish-v4","Age":"0","X-Cache":"cp1051 miss, cp1058 miss","X-Cache-Status":"miss","Strict-Transport-Security":"max-age=106384710; includeSubDomains; preload","Set-Cookie":"WMF-Last-Access-Global=27-Sep-2017;Path=\/;Domain=.wikidata.org;HttpOnly;secure;Expires=Sun, 29 Oct 2017 00:00:00 GMT","X-Analytics":"https=1;nocookies=1","X-Client-IP":"2620:0:861:108:1a66:daff:fefa:bc4"},"SPARQL-QUERY: queryStr=PREFIX wd: PREFIX wdt: SELECT (count(distinct(?s)) AS ?scount) WHERE {?s wdt:P31\/wdt:P279* wd:Q11173}\njava.util.concurrent.TimeoutException\n\tat java.util.concurrent.FutureTask.get(FutureTask.java:205)\n\tat com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:293)\n\tat com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:654)\n\tat com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:288)\n\tat com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240)\n\tat com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:271)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:687)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1667)\n\tat org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter.doFilter(ThrottlingFilter.java:244)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:49)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat org.wikidata.query.rdf.blazegraph.filters.ClientIPFilter.doFilter(ClientIPFilter.java:42)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)\n\tat java.lang.Thread.run(Thread.java:748)\n"]
So it does indeed look like these are timing out.
We have 2 options:
Rewrite this so that the numbers come from the dumps instead of from the SPARQL endpoint. This will mean only having weekly data
Use the open port (for the analytics cluster only) that allows for longer running SPARQL queries. This will mean we keep the daily data. I guess one day in the future we might run into the same problem though. (Relevant ticket T119941)
TASK DETAILhttps://phabricator.wikimedia.org/T176577EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: AddshoreCc: gerritbot, Addshore, Sjoerddebruin, Aklapper, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Izno, Wikidata-bugs, aude, Mbch331___
