[Wikidata-bugs] [Maniphest] [Updated] T176577: Some values of "Instance of counts" are not updating anymore

Wed, 27 Sep 2017 06:35:23 -0700

Addshore added a comment. 

addshore@stat1005:~$ cat /srv/analytics-wmde/graphite/log/daily.03.log |grep wikidata-sparql-instanceof |grep Q11173
2017-09-26 03:01:59 wikidata-sparql-instanceof Result for: Q11173 null
2017-09-27 03:02:07 wikidata-sparql-instanceof WikidataInstanceOf::doSparqlQuery: PREFIX wd: <http://www.wikidata.org/entity/>PREFIX wdt: <http://www.wikidata.org/prop/direct/>SELECT (count(distinct(?s)) AS ?scount) WHERE {?s wdt:P31/wdt:P279* wd:Q11173} [{"http_code":"HTTP\/1.1 500 Server Error","Date":"Wed, 27 Sep 2017 03:02:07 GMT","Content-Type":"text\/plain","Content-Length":"3093","Connection":"keep-alive","Server":"nginx\/1.11.13","X-Served-By":"wdqs1004","Access-Control-Allow-Origin":"*","Vary":"Accept-Encoding","X-Varnish":"21878624, 304358865","Via":"1.1 varnish-v4, 1.1 varnish-v4","Age":"0","X-Cache":"cp1051 miss, cp1058 miss","X-Cache-Status":"miss","Strict-Transport-Security":"max-age=106384710; includeSubDomains; preload","Set-Cookie":"WMF-Last-Access-Global=27-Sep-2017;Path=\/;Domain=.wikidata.org;HttpOnly;secure;Expires=Sun, 29 Oct 2017 00:00:00 GMT","X-Analytics":"https=1;nocookies=1","X-Client-IP":"2620:0:861:108:1a66:daff:fefa:bc4"},"SPARQL-QUERY: queryStr=PREFIX wd: <http:\/\/www.wikidata.org\/entity\/>PREFIX wdt: <http:\/\/www.wikidata.org\/prop\/direct\/>SELECT (count(distinct(?s)) AS ?scount) WHERE {?s wdt:P31\/wdt:P279* wd:Q11173}\njava.util.concurrent.TimeoutException\n\tat java.util.concurrent.FutureTask.get(FutureTask.java:205)\n\tat com.bigdata.rdf.sail.webapp.BigdataServlet.submitApiTask(BigdataServlet.java:293)\n\tat com.bigdata.rdf.sail.webapp.QueryServlet.doSparqlQuery(QueryServlet.java:654)\n\tat com.bigdata.rdf.sail.webapp.QueryServlet.doGet(QueryServlet.java:288)\n\tat com.bigdata.rdf.sail.webapp.RESTServlet.doGet(RESTServlet.java:240)\n\tat com.bigdata.rdf.sail.webapp.MultiTenancyServlet.doGet(MultiTenancyServlet.java:271)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:687)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:769)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1667)\n\tat org.wikidata.query.rdf.blazegraph.throttling.ThrottlingFilter.doFilter(ThrottlingFilter.java:244)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat ch.qos.logback.classic.helpers.MDCInsertingServletFilter.doFilter(MDCInsertingServletFilter.java:49)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat org.wikidata.query.rdf.blazegraph.filters.ClientIPFilter.doFilter(ClientIPFilter.java:42)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1650)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:583)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1125)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1059)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:497)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:311)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:248)\n\tat org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:610)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:539)\n\tat java.lang.Thread.run(Thread.java:748)\n"]


So it does indeed look like these are timing out.



We have 2 options:



    

  1. Rewrite this so that the numbers come from the dumps instead of from the SPARQL endpoint. This will mean only having weekly data
    2. 

  2. Use the open port (for the analytics cluster only) that allows for longer running SPARQL queries. This will mean we keep the daily data. I guess one day in the future we might run into the same problem though. (Relevant ticket T119941)
    3.

TASK DETAIL
https://phabricator.wikimedia.org/T176577

EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: Addshore
Cc: gerritbot, Addshore, Sjoerddebruin, Aklapper, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Izno, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs

Reply via email to