[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-03-26 Thread Smalyshev
Smalyshev added a comment. Looking at the distribution of Special:EntityData fetches, if we cache entities under 10K, we will capture about 90% of them. Most frequent sizes are 1 to 4K. So caching probably worth trying. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL

[Wikidata-bugs] [Maniphest] [Closed] T218174: Add metric for skip-ahead revisions

2019-03-27 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. Smalyshev added a comment. Metric name is `wdqs_updater_updates_skip`. TASK DETAIL https://phabricator.wikimedia.org/T218174 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailprefer

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-27 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, thiemowmde, Addshore

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-27 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, thiemowmde, Addshore

[Wikidata-bugs] [Maniphest] [Updated] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-27 Thread Smalyshev
Smalyshev removed a project: MW-1.33-notes (1.33.0-wmf.22; 2019-03-19). Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF

[Wikidata-bugs] [Maniphest] [Updated] T190022: Separate the CirrusSearch/Elastic-specific code from Wikibase code base

2019-03-27 Thread Smalyshev
Smalyshev removed projects: MW-1.33-notes (1.33.0-wmf.22; 2019-03-19), Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T190022 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jony, Lucas_Werkmeister_WMDE, gerritbot

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-03-27 Thread Smalyshev
Smalyshev added a comment. > the cache we are talking about there would be unnecessary if the wdqs just hit varnish. It is problematic for WDQS to "just hit varnish", because varnish does not know if certain revision is the latest one available or not. Wikidata on the ot

[Wikidata-bugs] [Maniphest] [Created] T219455: wikidata.org search uses tons of namespaces by default

2019-03-27 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Discovery-Search (Current work). Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION If I open wikidata.org with clean environment and go to Special:Search and search, I get lots of namespaces searched

[Wikidata-bugs] [Maniphest] [Updated] T219455: wikidata.org search uses tons of namespaces by default

2019-03-27 Thread Smalyshev
Smalyshev added a project: Advanced-Search. Restricted Application added a project: TCB-Team. TASK DETAIL https://phabricator.wikimedia.org/T219455 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: EBernhardson, dcausse, Aklapper

[Wikidata-bugs] [Maniphest] [Edited] T219455: wikidata.org search uses tons of namespaces by default

2019-03-27 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T219455 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: EBernhardson, dcausse, Aklapper, Smalyshev, alaa_wmde, Nandana, Lahi, Gq86, GoranSMilovanovic

[Wikidata-bugs] [Maniphest] [Edited] T219455: wikidata.org search uses tons of namespaces by default

2019-03-27 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T219455 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: EBernhardson, dcausse, Aklapper, Smalyshev, alaa_wmde, CucyNoiD, Nandana, NebulousIris

[Wikidata-bugs] [Maniphest] [Created] T219477: Automatic detection of distributed bots

2019-03-27 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION We've had a number of times where bots evaded throttling mechanisms of WDQS by using e

[Wikidata-bugs] [Maniphest] [Updated] T219455: AdvancedSearch extension changes behaviour of default namespaces to be search for anon and logged in users

2019-03-27 Thread Smalyshev
Smalyshev changed the subtype of this task from "Task" to "Bug Report". TASK DETAIL https://phabricator.wikimedia.org/T219455 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: IKhitron, EBernhardson, dcausse,

[Wikidata-bugs] [Maniphest] [Commented On] T219452: Cannot add a Wikidata sitelink [2019-03-27]

2019-03-27 Thread Smalyshev
Smalyshev added a comment. I don't think I did any changes to Wikidata related to sitelinks in any way... Could be wrong of course but nothing comes to mind. TASK DETAIL https://phabricator.wikimedia.org/T219452 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-28 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-03-28 Thread Smalyshev
Smalyshev added a comment. > WDQS does know what the latest version of the entity that it is trying to get updates for is, But "last version that WDQS knows of" can be very different from "last version that Wikidata has". That's the whole issue. I had an

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-28 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-28 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-03-28 Thread Smalyshev
Smalyshev added a comment. @Addshore btw do I understand right that constraints can not be fetched per-revision? In this case, do we still need cache-busting there? Or constrains manage their caches? I am not sure what to do here. TASK DETAIL https://phabricator.wikimedia.org/T217897

[Wikidata-bugs] [Maniphest] [Updated] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-28 Thread Smalyshev
Smalyshev removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-03-28 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Created] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-01 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata Query UI. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION After merging 2144a682d48168555368d8aa5df1bd9dfa369fb2 <https://phabricator.wikimedia.

[Wikidata-bugs] [Maniphest] [Edited] T216206: Set up WikibaseLexemeCirrusSearch extension for Elastic code in WikibaseLexeme

2019-04-01 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T216206 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Jdforrester-WMF, gerritbot, Aklapper, Lydia_Pintscher, WMDE-leszek, Addshore

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-01 Thread Smalyshev
Smalyshev added a comment. Now I get this: 17:42:14 [INFO] Selenium started 17:42:14 [ERROR] 17:42:14.826 INFO [ActiveSessionFactory.apply] - Capabilities are: { 17:42:14 [ERROR] "browserName": "firefox", 17:42:14 [ERROR] "handlesAlerts&qu

[Wikidata-bugs] [Maniphest] [Changed Project Column] T218716: Remove Elastic/Cirrus code from Wikibase repo

2019-04-02 Thread Smalyshev
Smalyshev moved this task from Wikidata Search to Current work on the Discovery-Search board. Smalyshev edited projects, added Discovery-Search (Current work); removed Discovery-Search. TASK DETAIL https://phabricator.wikimedia.org/T218716 WORKBOARD https://phabricator.wikimedia.org

[Wikidata-bugs] [Maniphest] [Triaged] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-02 Thread Smalyshev
Smalyshev triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Greta_Doci_WMDE, Aklapper, Smalyshev, alaa_wmde,

[Wikidata-bugs] [Maniphest] [Commented On] T87283: Wikidata dumps should have revision ID or other sequence mark

2019-04-02 Thread Smalyshev
Smalyshev added a comment. I think, if I can recall correctly (this is 4+ years old by now :) that the idea was to have means to quickly identify that one dump is more recent than the other one, or less recent, and where in the sequence of dumps the particular dump falls. Individual

[Wikidata-bugs] [Maniphest] [Commented On] T87283: Wikidata dumps should have revision ID or other sequence mark

2019-04-02 Thread Smalyshev
Smalyshev added a comment. OK it seems to be a bit unclear whether this was asking for revision IDs on particular entity or on the dump as a whole. I think that we need both, but the patch above seems to add the revision ID to entities. I think it makes sense. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Commented On] T87283: Wikidata dumps should have revision ID or other sequence mark

2019-04-02 Thread Smalyshev
Smalyshev added a comment. It's a 4 year old task, so I myself is not 100% clear which one it was back then. So I think having it here is fine. TASK DETAIL https://phabricator.wikimedia.org/T87283 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Updated] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-02 Thread Smalyshev
Smalyshev removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Greta_Doci_WMDE, Aklapper, Smalyshev, alaa_wmde, Nandana, Lahi

[Wikidata-bugs] [Maniphest] [Updated] T219968: Enable the URLshortener for WDQS

2019-04-03 Thread Smalyshev
Smalyshev added a parent task: T112715: Enable different URL shorteners for WDQS. TASK DETAIL https://phabricator.wikimedia.org/T219968 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: B20180, Pigsonthewing, Smalyshev, Aklapper

[Wikidata-bugs] [Maniphest] [Triaged] T219968: Enable the URLshortener for WDQS

2019-04-03 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T219968 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: B20180, Pigsonthewing, Smalyshev, Aklapper, Ladsgroup, Lydia

[Wikidata-bugs] [Maniphest] [Updated] T112715: Enable different URL shorteners for WDQS

2019-04-03 Thread Smalyshev
Smalyshev added a subtask: T219968: Enable the URLshortener for WDQS. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing, Lucas_Werkmeister_WMDE, Ladsgroup

[Wikidata-bugs] [Maniphest] [Created] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Discovery. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Since we're starting to work with @Igorkim78 we need to set up developer acces

[Wikidata-bugs] [Maniphest] [Updated] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev added a project: LDAP-Access-Requests. TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Nuria, Smalyshev, Igorkim78, alaa_wmde, ET4Eva, Nandana, Jony, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Edited] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Nuria, Smalyshev, Igorkim78, alaa_wmde, ET4Eva, Nandana, Jony, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-03 Thread Smalyshev
Smalyshev added a comment. GUI is still not mergeable, so I think I'll disable running wdio test on CI for now. TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalysh

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-03 Thread Smalyshev
Smalyshev added a subscriber: Lucas_Werkmeister_WMDE. Smalyshev added a comment. Looks like we have problem with redirects - they can not be fetched by-revision. E.g.: https://www.wikidata.org/wiki/Special:EntityData/Q33044800.ttl?flavor=dump&revision=901802292 produces 400. S

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-03 Thread Smalyshev
Smalyshev added a comment. I've also made a counter to check how many "forward skips" - i.e. loading revision further than we've asked in change - we get. The averages are between 0.1 and 0.5, sometimes going to 1 - i.e. we're saving up to one item fetch/update p

[Wikidata-bugs] [Maniphest] [Commented On] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev added a comment. @Aklapper: I see there that: > Admins can create an account while specifying the password, then mail it to the user who can log in with that and use the password change form Can it be done in this case? TASK DETAIL https://phabricator.wikimedia.

[Wikidata-bugs] [Maniphest] [Triaged] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Nuria, Smalyshev, Igorkim78, alaa_wmde, ET4Eva, Nandana, Jony,

[Wikidata-bugs] [Maniphest] [Commented On] T220046: Set up developer account for Igor Kim

2019-04-03 Thread Smalyshev
Smalyshev added a comment. @Aklapper also, could you do the necessary magic to make Igor's phabricator account approved? TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Akl

[Wikidata-bugs] [Maniphest] [Edited] T218716: Remove Elastic/Cirrus code from Wikibase repo

2019-04-03 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T218716 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Lydia_Pintscher, WMDE-leszek, RazShuty, thiemowmde, Aleksey_WMDE, Addshore

[Wikidata-bugs] [Maniphest] [Edited] T220046: Set up developer account for Igor Kim

2019-04-04 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Nuria, Smalyshev, Igorkim78, alaa_wmde, ET4Eva, Nandana, Jony, Lahi, Gq86

[Wikidata-bugs] [Maniphest] [Created] T220437: Allow force-push for wikidata-query on wikidata/query/blazegraph

2019-04-08 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Gerrit-Privilege-Requests. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION wikidata/query/blazegraph repo is our private fork of Blazegraph

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T200612: Wikidata's SPARQL endpoint doesn't escape commas in IRIs in CSV output

2019-04-08 Thread Smalyshev
Smalyshev added a subscriber: Igorkim78. Smalyshev added a comment. Getting this in the CI run <https://integration.wikimedia.org/ci/job/wikidata-query-blazegraph-maven-java8-docker/9/console>: 8:59:10 SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".

[Wikidata-bugs] [Maniphest] [Commented On] T200612: Wikidata's SPARQL endpoint doesn't escape commas in IRIs in CSV output

2019-04-08 Thread Smalyshev
Smalyshev added a comment. This seems to be a change in Sesame - before, LiteralImpl(String) produced null as datatype, now it produces `XMLSchema.STRING`. It was changed in this commit: https://bitbucket.org/openrdf/sesame/commits/675015e6b996cc8609fa735730baa49edf27d2e7 TASK DETAIL

[Wikidata-bugs] [Maniphest] [Edited] T218716: Remove Elastic/Cirrus code from Wikibase repo

2019-04-08 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T218716 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Lydia_Pintscher, WMDE-leszek, RazShuty, thiemowmde, Aleksey_WMDE, Addshore

[Wikidata-bugs] [Maniphest] [Commented On] T200612: Wikidata's SPARQL endpoint doesn't escape commas in IRIs in CSV output

2019-04-08 Thread Smalyshev
Smalyshev added a comment. Turns out Sesame 2.8 has pretty big difference from 2.7 - in RDF 1.1/SPARQL 1.1 there are no "simple literals" anymore, i.e. literal `"abc"` is the same as literal `"abc"^^xsd:string` and the only type of plain literal is language on

[Wikidata-bugs] [Maniphest] [Commented On] T220046: Set up developer account for Igor Kim

2019-04-09 Thread Smalyshev
Smalyshev added a comment. @thcipriani @mmodell is this something you would be able to do - create the Gerrit account for Igor manually? TASK DETAIL https://phabricator.wikimedia.org/T220046 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To

[Wikidata-bugs] [Maniphest] [Created] T220666: quibble-vendor-mysql-hhvm-docker fails in Wikibase

2019-04-10 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Campsite, Lexicographical data. Restricted Application added a project: Wikidata. TASK DESCRIPTION Getting this failure on Wikibase test: 19:00:25 1) Special:RecentChanges shows lemmas in title links to lexemes: 19:00:25

[Wikidata-bugs] [Maniphest] [Edited] T220666: quibble-vendor-mysql-hhvm-docker fails in Wikibase

2019-04-10 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T220666 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, alaa_wmde, Nandana, Mringgaard, Lahi, Gq86, GoranSMilovanovic, QZanden

[Wikidata-bugs] [Maniphest] [Commented On] T217557: Socket timeout on wdqs.svc.eqiad.wmnet

2019-04-10 Thread Smalyshev
Smalyshev added a comment. @Gehel any input on this? TASK DETAIL https://phabricator.wikimedia.org/T217557 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Mholloway, Aklapper, Mathew.onipe, Smalyshev, Gehel, alaa_wmde

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-10 Thread Smalyshev
Smalyshev added a comment. The latest patch doesn't seem to help, because test is run by pom.xml as it seems. Please see full log in https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-java8-docker/850/console TASK DETAIL https://phabricator.wikimedia.org/T219749

[Wikidata-bugs] [Maniphest] [Closed] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] [Closed] T219968: Enable the URLshortener for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T219968 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Salgo60, CennoxX, Sylvain_WM

[Wikidata-bugs] [Maniphest] [Unblock] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed subtask T219968: Enable the URLshortener for WDQS as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pigsonthewing, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] [Declined] T218568: Allow CORS from query.wikidata.org to production wikis

2019-04-15 Thread Smalyshev
Smalyshev closed this task as "Declined". Smalyshev added a comment. Looks like URL shortener works without it, so we don't need this for now. TASK DETAIL https://phabricator.wikimedia.org/T218568 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailp

[Wikidata-bugs] [Maniphest] [Unblock] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev closed subtask T218568: Allow CORS from query.wikidata.org to production wikis as "Declined". TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Pig

[Wikidata-bugs] [Maniphest] [Commented On] T112715: Enable different URL shorteners for WDQS

2019-04-15 Thread Smalyshev
Smalyshev added a comment. @Ladsgroup which one? With the query (top left) or the results (down right in the menu)? In any case, probably makes sense to open new task with a screen shot. TASK DETAIL https://phabricator.wikimedia.org/T112715 EMAIL PREFERENCES https

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-15 Thread Smalyshev
Smalyshev added a comment. @aaron I have started looking into it, and I am not sure how one can get an instance of ChronologyProtector in the code that needs it. The only place I can see that uses ChronologyProtector is LBFactory class, but getChronologyProtector() is protected there, so

[Wikidata-bugs] [Maniphest] [Edited] T190422: On WDQS some messages are not translated when switching languages

2019-04-16 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T190422 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Micru, alaa_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic

[Wikidata-bugs] [Maniphest] [Commented On] T112715: Enable different URL shorteners for WDQS

2019-04-16 Thread Smalyshev
Smalyshev added a comment. > So there is no difference between "failed because URL is too long" and "failed because so many people use the service that it's been blocked by rate limit" Yes, error reporting right now is most rudimentary. We probably should fix

[Wikidata-bugs] [Maniphest] [Created] T221126: Improve error reporting on URL shortener

2019-04-16 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata Query UI. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Right now, the only error URL shortener produces is "URL shortening failed". We proba

[Wikidata-bugs] [Maniphest] [Created] T221127: URL shortener widget is hard to copy

2019-04-16 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata Query UI. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION There are reports that URL shortener widget is hard to work with: @Ladsgroup: > whe

[Wikidata-bugs] [Maniphest] [Commented On] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-17 Thread Smalyshev
Smalyshev added a comment. Still getting failures, see: https://integration.wikimedia.org/ci/job/wikidata-query-rdf-maven-java8-docker/855/console TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences

[Wikidata-bugs] [Maniphest] [Commented On] T221315: Determine future of bare-metal hosting for services like WDQS, ElasticSearch, RESTBase Cassandra, etc.

2019-04-17 Thread Smalyshev
Smalyshev added a comment. The web UI frontend is just a bunch of HTML files, so there's not much to move. There's however nginx frontend, which serves in dual capacity as both gateway to database backend and UI server. It is possible to run either function on a separate co

[Wikidata-bugs] [Maniphest] [Updated] T217925: Keep global "last seen revision" map for Updater

2019-04-17 Thread Smalyshev
Smalyshev removed a project: Discovery-Wikidata-Query-Service-Sprint. TASK DETAIL https://phabricator.wikimedia.org/T217925 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Gehel, Aklapper, Smalyshev, alaa_wmde

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-18 Thread Smalyshev
Smalyshev added a comment. This is probably the request for @aaron, he knows the most about it. TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Ottomata, Aklapper, Smalyshev

[Wikidata-bugs] [Maniphest] [Created] T221405: DCATAP dump reload does not delete old namespace

2019-04-18 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION When we load new DCATAP dump into new namespace, old namespace is supposed to be deleted

[Wikidata-bugs] [Maniphest] [Created] T221407: Unrecognized subject messages in Updater

2019-04-18 Thread Smalyshev
Smalyshev created this task. Smalyshev added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Updater logs show these messages: 05:26:33.222 [update 5] INFO

[Wikidata-bugs] [Maniphest] [Updated] T221146: WDQS returns stale data

2019-04-18 Thread Smalyshev
Smalyshev added a comment. The data seems to be updated in the database (revision 915966700). I suspect it may be the same issue as with T197447 <https://phabricator.wikimedia.org/T197447> - Blazegraph conflates strings with some invisible characters and strings without them. TASK

[Wikidata-bugs] [Maniphest] [Retitled] T221146: WDQS confuses strings with and without U+00AD

2019-04-18 Thread Smalyshev
Smalyshev renamed this task from "WDQS returns stale data" to "WDQS confuses strings with and without U+00AD". Smalyshev triaged this task as "Normal" priority. TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricato

[Wikidata-bugs] [Maniphest] [Commented On] T221146: WDQS confuses strings with and without U+00AD

2019-04-18 Thread Smalyshev
Smalyshev added a comment. See also https://github.com/blazegraph/database/issues/93 TASK DETAIL https://phabricator.wikimedia.org/T221146 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Liridon, Bdijkstra, Aklapper

[Wikidata-bugs] [Maniphest] [Commented On] T212550: Implement support for ChronologyProtection in RDF export

2019-04-18 Thread Smalyshev
Smalyshev added a comment. @aaron Is there any docs how to use the client ID on the client side? I see there's support for `cpPosIndex` cookie which is `$index@$time#$clientId` but if I have only client ID that is not going to help me. So I am not sure how would one use it - I can&#

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Curiously, this happens only on hosts where revision-fetch is enabled. I wonder whether it's related, though I am not sure how. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. First error on wdq24 is from 2019-04-11 07:45:56.038. Patch was merged also on Apr 11 and Updater was restarted with it on 07:42:21.097. I'm pretty much sure it's connected - still not sure how. TASK DETAIL https://phabricator.wikimedia.org/T221

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Also, errors on different servers do not seem to match, even though the content is supposed to be exactly the same - this is the whole point of caching after all! Something weird is definitely going on. TASK DETAIL https://phabricator.wikimedia.org/T221407

[Wikidata-bugs] [Maniphest] [Commented On] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-19 Thread Smalyshev
Smalyshev added a comment. Results of caching can be seen here: https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1&from=now-30d&to=now&panelId=24&fullscreen&var-cluster_name=wdqs-internal Deploy date is Apr 11, fetch time drops from 135/195 ms

[Wikidata-bugs] [Maniphest] [Changed Subscribers] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev added subscribers: Lucas_Werkmeister_WMDE, WMDE-leszek, Addshore. Smalyshev added a comment. I think I know the reason... Constraint violations are always fetched for latest version, while revision fetches may be fetched for non-current ones. This means that constraint violation

[Wikidata-bugs] [Maniphest] [Triaged] T221407: Unrecognized subject messages in Updater

2019-04-19 Thread Smalyshev
Smalyshev triaged this task as "Normal" priority. Smalyshev added a project: Discovery-Wikidata-Query-Service-Sprint. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Adds

[Wikidata-bugs] [Maniphest] [Commented On] T221407: Unrecognized subject messages in Updater

2019-04-21 Thread Smalyshev
Smalyshev added a comment. Deploying the constraints patch on test servers seems to eliminate these messages. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Addshore, WMDE

[Wikidata-bugs] [Maniphest] [Closed] T221405: DCATAP dump reload does not delete old namespace

2019-04-23 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221405 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Smalyshev, alaa_wmde, joker88john, CucyNoi

[Wikidata-bugs] [Maniphest] [Commented On] T221632: Storage capacity upgrade for WDQS

2019-04-23 Thread Smalyshev
Smalyshev added a comment. This also makes me note that if we do introduce sharding (in any shape, either with Blazegraph or another solution) we'd need even more servers, since each shard would need to be at least on 2-3 servers to survive, so for sharding to make any sense we'

[Wikidata-bugs] [Maniphest] [Edited] T221632: Storage capacity upgrade for WDQS

2019-04-23 Thread Smalyshev
Smalyshev updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T221632 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Gehel, Smalyshev Cc: Smalyshev, Aklapper, Gehel, alaa_wmde, Nandana, Lahi, Gq86, Lucas_Werkmeister_WMDE

[Wikidata-bugs] [Maniphest] [Created] T221709: scap service restarts for WDQS are inconsistent

2019-04-23 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Scap. Restricted Application added a subscriber: Aklapper. Restricted Application added a project: Wikidata. TASK DESCRIPTION Right now, when WDQS is deployed, only service `wdqs-blazegraph` is restarted. This is

[Wikidata-bugs] [Maniphest] [Claimed] T221407: Unrecognized subject messages in Updater

2019-04-23 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Gehel, Addshore, WMDE-leszek, Lucas_Werkmeister_WMDE, Aklapper, Smalyshev, alaa_wmde, joker88john

[Wikidata-bugs] [Maniphest] [Closed] T221407: Unrecognized subject messages in Updater

2019-04-23 Thread Smalyshev
Smalyshev closed this task as "Resolved". TASK DETAIL https://phabricator.wikimedia.org/T221407 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Gehel, Addshore, WMDE-leszek, Lucas_Werkmeister_WMDE, Aklapper, Smalyshev,

[Wikidata-bugs] [Maniphest] [Retitled] T212550: Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata

2019-04-23 Thread Smalyshev
Smalyshev renamed this task from "Implement support for ChronologyProtection in RDF export" to "Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata". TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL

[Wikidata-bugs] [Maniphest] [Claimed] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-24 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Smalyshev, BBlack, Aklapper, Gehel, alaa_wmde, joker88john

[Wikidata-bugs] [Maniphest] [Claimed] T212550: Implement support for ChronologyProtection in events sent when editing Mediawiki/Wikidata

2019-04-24 Thread Smalyshev
Smalyshev claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T212550 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Krinkle, Ottomata, Aklapper, Smalyshev, aaron, alaa_wmde, joker88john, CucyNoiD, Nandana, NebulousIris

[Wikidata-bugs] [Maniphest] [Commented On] T221632: Storage capacity upgrade for WDQS

2019-04-25 Thread Smalyshev
Smalyshev added a comment. > I don't think it makes sense to perpetuate a vertical scaling model With regard to disk space, I don't think we have a choice at least for the next FY. Even if we find alternative backend that can be a drop-in replacement for Blazegrap

[Wikidata-bugs] [Maniphest] [Created] T221882: WDQS public cluster capacity upgrade

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata-Query-Service, Discovery. Restricted Application added a project: Wikidata. TASK DESCRIPTION As can be witnessed from the WDQS lag graph <https://grafana.wikimedia.org/d/00489/wikidata-query-service?orgId=1&from=now-90d

[Wikidata-bugs] [Maniphest] [Updated] T141602: [Story] Provide a SPARQL query service for structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev edited projects, added Wikidata-Query-Service, Epic; removed Story. TASK DETAIL https://phabricator.wikimedia.org/T141602 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Salgo60, MB-one, Tpt, Addshore

[Wikidata-bugs] [Maniphest] [Created] T221916: Create RDF export for structured data stored for files

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo. TASK DESCRIPTION Structured data information on Commons should have RDF representation, just like the information on Wikidata has. We need to create code that

[Wikidata-bugs] [Maniphest] [Created] T221917: Create RDF dump of structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev created this task. Smalyshev added projects: Wikidata, Commons, SDC General, Wikidata-Query-Service, WikibaseMediaInfo. TASK DESCRIPTION We should set up a regular dump of all structured data entities on Commons, akin to dumps of Wikidata entities we have now. TASK DETAIL https

[Wikidata-bugs] [Maniphest] [Updated] T221916: Create RDF export for structured data stored for files

2019-04-25 Thread Smalyshev
Smalyshev added a parent task: T221917: Create RDF dump of structured data on Commons. TASK DETAIL https://phabricator.wikimedia.org/T221916 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper

[Wikidata-bugs] [Maniphest] [Updated] T221917: Create RDF dump of structured data on Commons

2019-04-25 Thread Smalyshev
Smalyshev added a subtask: T221916: Create RDF export for structured data stored for files. TASK DETAIL https://phabricator.wikimedia.org/T221917 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Poyekhali, Steinsplitter, Aklapper

[Wikidata-bugs] [Maniphest] [Closed] T219749: WDQS GUI test broken after merging 2144a682d48168555368d8aa5df1bd9dfa369fb2

2019-04-26 Thread Smalyshev
Smalyshev closed this task as "Resolved". Smalyshev added a comment. Looks like it's not broken anymore. Thanks! TASK DETAIL https://phabricator.wikimedia.org/T219749 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: noarav

[Wikidata-bugs] [Maniphest] [Updated] T217897: Reduce / remove the aggessive cache busting behaviour of wdqs-updater

2019-04-26 Thread Smalyshev
Smalyshev removed a project: Patch-For-Review. TASK DETAIL https://phabricator.wikimedia.org/T217897 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Lucas_Werkmeister_WMDE, Addshore, Smalyshev, BBlack, Aklapper, Gehel, alaa_wmde

  1   2   3   4   5   6   7   8   9   10   >