[Wikidata-bugs] [Maniphest] [Commented On] T84902: deploy haedus and capella with os for orientdb testing
Joe added a comment. Capella only sees one disk, at pci-:01:00.0-scsi-0:2:0:0, this needs investigation. Moving the ticket to the codfw queue TASK DETAIL https://phabricator.wikimedia.org/T84902 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: Aklapper, RobH, Joe, aaron, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Up For Grabs] T84902: deploy haedus and capella with os for orientdb testing
Joe placed this task up for grabs. Joe added a project: ops-codfw. TASK DETAIL https://phabricator.wikimedia.org/T84902 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: Aklapper, RobH, Joe, aaron, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Claimed] T78267: Package opendjk-8 for trusty
Joe claimed this task. Joe added projects: operations, ops-core. TASK DETAIL https://phabricator.wikimedia.org/T78267 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: Aklapper, Joe, jkroll, Smalyshev, Wikidata-bugs, RobH, aude, GWicke, Manybubbles, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T85133: Introduce setting to allow import of entities from XML dumps.
daniel created this task. daniel added a subscriber: daniel. daniel added a project: MediaWiki-extensions-WikibaseRepository. daniel changed Security from none to none. TASK DESCRIPTION We generally disallow import of entities, since entities imported from other wikibase repositories would likely be using different IDs for items and properties. This may not only lead to dead links, but cause errors due to incompatible property definitions. However, at least importing into a blank wiki (e.g. restoring from backup or setting up a test environment) should be allowed. An easy way to achieve this would be to introduce a setting that overrides the code in WikibaseHooks::onImportHandleRevisionXMLTag that prevents the import. TASK DETAIL https://phabricator.wikimedia.org/T85133 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: daniel Cc: Aklapper, daniel, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T78267: Package opendjk-8 for trusty
Joe added a comment. I created the package; I will not upload it to reprepro for now - I want to test it, on Titan maybe? TASK DETAIL https://phabricator.wikimedia.org/T78267 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: Aklapper, Joe, jkroll, Smalyshev, Wikidata-bugs, RobH, aude, GWicke, Manybubbles, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T78267: Package opendjk-8 for trusty
Manybubbles added a subscriber: Manybubbles. Manybubbles added a comment. I can test it MW vagrant if you'd like. I have titan downloaded and building against built from source jdk8. TASK DETAIL https://phabricator.wikimedia.org/T78267 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe, Manybubbles Cc: Aklapper, Joe, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, RobH, aude, GWicke, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T84902: deploy haedus and capella with os for orientdb testing
RobH added a comment. It seems I accidentally didnt notice and break the raid config setup on capella, as it uses an H310. A quick look confirms haedus does as well. So now capella is reinstalled with the h310 in bypass mode, presenting the disks normally. So this should be good to go, going to remove the codfw onsite flag, as there isn't any onsite work to be done. If these aren't good for this testing, joe mentioned the old lsearch systems would also work. So this is allocated for the testing, but can be swapped if it turns out to not be ideal. TASK DETAIL https://phabricator.wikimedia.org/T84902 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RobH Cc: Aklapper, RobH, Joe, aaron, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T84902: deploy haedus and capella with os for orientdb testing
RobH removed a project: ops-codfw. TASK DETAIL https://phabricator.wikimedia.org/T84902 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RobH Cc: Aklapper, RobH, Joe, aaron, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T75813: Investigate OrientDB for WDQ
Joe added a project: wikidata-query-service. TASK DETAIL https://phabricator.wikimedia.org/T75813 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aaron, Joe Cc: aaron, Liuxinyu970226, Joe, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T78267: Package opendjk-8 for trusty
Joe closed this task as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T78267 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Joe Cc: Aklapper, Joe, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, RobH, aude, GWicke, daniel, mark, JohnLewis, faidon, akosiaris, fgiunchedi, jeremyb ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T84923: Reliable publish / subscribe event bus
JanZerebecki added a subscriber: JanZerebecki. JanZerebecki added a comment. The nature of these event type candidates is such that they are changes with a log existing at the provider. The only persistent state that each consuming service needs to know is the revision/time up to which it has applied the changes. The historic (recent) changes are already kept on the providing service, so a solution would not necessarily have to retain those. It is probably with a bit of care possible to apply changes out of order and in a way that it is idempotent so the consuming service could apply restartable and in parallel. The actual data can be pulled by the consumer from the provider; duplicating it in the queue probably has no benefit. I think the only thing here where pub/sub brings anything worth is the event that new changes are available to decrease latency of updates. Which means only ever one event in queue for all consumers, as new events contain all information of the previous ones. How is the job queue currently not reliable? ( https://www.mediawiki.org/wiki/Job_queue_redesign and https://www.mediawiki.org/wiki/Manual:Job_queue mention other problems but nothing regarding reliability problems. ) TASK DETAIL https://phabricator.wikimedia.org/T84923 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: GWicke, aaron, JanZerebecki, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, RobH, aude, Manybubbles, daniel, mark, Joe ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Merged] T1095: Investigate Titan for WDQ
JanZerebecki closed this task as a duplicate of T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service. TASK DETAIL https://phabricator.wikimedia.org/T1095 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JanZerebecki Cc: aaron, Smalyshev, jkroll, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
JanZerebecki added a project: MediaWiki-Core-Team. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, JanZerebecki Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
JanZerebecki added a subscriber: aaron. JanZerebecki merged a task: T1095: Investigate Titan for WDQ. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, JanZerebecki Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T85159: Deploy a Wikidata complex query service into production
RobLa-WMF created this task. RobLa-WMF added a subscriber: RobLa-WMF. RobLa-WMF added projects: wikidata-query-service, Epic, MediaWiki-Core-Team. RobLa-WMF changed Security from none to none. TASK DESCRIPTION Create a service which supports complex queries on Wikidata: - **external** requests return within a few seconds, use reasonable resources - **internal** requests are allowed to use more resources time See https://www.mediawiki.org/wiki/Wikibase/Indexing. TASK DETAIL https://phabricator.wikimedia.org/T85159 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RobLa-WMF Cc: Aklapper, RobLa-WMF, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Updated] T85159: Deploy a Wikidata complex query service into production
RobLa-WMF added blocking tasks: T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service, T76372: Evaluate TinkerPop GraphSail frontend, T1028: Experiment with Wikidata Query Engine, T75813: Investigate OrientDB for WDQ. TASK DETAIL https://phabricator.wikimedia.org/T85159 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: RobLa-WMF Cc: Aklapper, RobLa-WMF, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T75813: Investigate OrientDB for WDQ
aaron moved this task to Done on the MediaWiki-Core-Team workboard. TASK DETAIL https://phabricator.wikimedia.org/T75813 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aaron Cc: aaron, Liuxinyu970226, Joe, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T76827: Image thumbnail urls should be included where applicable in wikidata API response for commonsMedia
Jdlrobson moved this task to Tracking / Involving other Projects on the Mobile-Web workboard. TASK DETAIL https://phabricator.wikimedia.org/T76827 WORKBOARD https://phabricator.wikimedia.org/project/board/67/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Jdlrobson Cc: Aklapper, Jdlrobson, daniel, aude, JanZerebecki, Lydia_Pintscher, MZMcBride, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T35704: GeoData should pull coordinates from Wikidata
Multichill added a subscriber: Multichill. TASK DETAIL https://phabricator.wikimedia.org/T35704 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MaxSem, Multichill Cc: wikibugs-l, Elitre, bearND, Krenair, Nikerabbit, Tfinc, Amire80, KLans_WMF, Deskana, Dbrant, Andyrom75, eranroz, He7d3r, Nemo_bis, Multichill, Wikidata-bugs, aude ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T85159: Deploy a Wikidata complex query service into production
Krenair added a subscriber: Krenair. TASK DETAIL https://phabricator.wikimedia.org/T85159 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Krenair Cc: Aklapper, RobLa-WMF, Krenair, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T78662: Change public interface to support multiple Entity types
JeroenDeDauw closed this task as Resolved. JeroenDeDauw claimed this task. TASK DETAIL https://phabricator.wikimedia.org/T78662 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JeroenDeDauw Cc: Aklapper, Lucie, JeroenDeDauw, Wikidata-bugs, aude ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T78662: Change public interface to support multiple Entity types
JeroenDeDauw added a subscriber: JeroenDeDauw. JeroenDeDauw added a comment. I already resolved this. Relevant commits are linked on GitHub. TASK DETAIL https://phabricator.wikimedia.org/T78662 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: JeroenDeDauw Cc: Aklapper, Lucie, JeroenDeDauw, Wikidata-bugs, aude ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T1095: Investigate Titan for WDQ
bd808 moved this task to Done on the MediaWiki-Core-Team workboard. TASK DETAIL https://phabricator.wikimedia.org/T1095 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: aaron, Smalyshev, jkroll, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T85045: Figure out how to detect that claim is updated
Smalyshev closed this task as Resolved. Smalyshev added a comment. We'll use content hash instead of claim ID to detect changes. We'll also use lastrevid on the item to track revisions. TASK DETAIL https://phabricator.wikimedia.org/T85045 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Aklapper, Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, daniel, jkroll, Wikidata-bugs ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Unblock] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
Smalyshev closed blocking task T85045: Figure out how to detect that claim is updated as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T85173: Investigate Titan supernode issues
Manybubbles created this task. Manybubbles claimed this task. Manybubbles added a subscriber: Manybubbles. Manybubbles added projects: MediaWiki-Core-Team, wikidata-query-service. Manybubbles moved this task to In Dev/Progress on the MediaWiki-Core-Team workboard. TASK DESCRIPTION Supernode cause sadness in Titan. Why it? TASK DETAIL https://phabricator.wikimedia.org/T85173 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Manybubbles Cc: Aklapper, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Claimed] T85174: Validate Java 8 package
Manybubbles claimed this task. Manybubbles added a project: wikidata-query-service. Manybubbles set Security to none. TASK DETAIL https://phabricator.wikimedia.org/T85174 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Manybubbles Cc: Aklapper, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
Manybubbles moved this task to In Dev/Progress on the MediaWiki-Core-Team workboard. TASK DETAIL https://phabricator.wikimedia.org/T76373 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, Manybubbles Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T1095: Investigate Titan for WDQ
bd808 moved this task to Archive on the MediaWiki-Core-Team workboard. TASK DETAIL https://phabricator.wikimedia.org/T1095 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: aaron, Smalyshev, jkroll, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Unblock] T85159: Deploy a Wikidata complex query service into production
bd808 closed blocking task T75813: Investigate OrientDB for WDQ as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T85159 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: bd808 Cc: Aklapper, RobLa-WMF, Krenair, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed Project Column] T75813: Investigate OrientDB for WDQ
bd808 moved this task to Archive on the MediaWiki-Core-Team workboard. TASK DETAIL https://phabricator.wikimedia.org/T75813 WORKBOARD https://phabricator.wikimedia.org/project/board/37/ REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aaron, bd808 Cc: aaron, Liuxinyu970226, Joe, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Closed] T75813: Investigate OrientDB for WDQ
bd808 closed this task as Resolved. TASK DETAIL https://phabricator.wikimedia.org/T75813 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: aaron, bd808 Cc: aaron, Liuxinyu970226, Joe, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Created] T85181: Investigate design public API, possibly using MQL
GWicke created this task. GWicke added a subscriber: GWicke. GWicke added a project: wikidata-query-service. TASK DESCRIPTION We need to investigate design a public query API. Goals for this API are: - Easy to build queries with a variety of clients (probably JSON) without dealing with low-level issues like escaping. - Only expose queries with reasonable resource consumption (CPU and IO, 1 minute run time?). If this cannot be statically guaranteed, have a way to reliably kill long-running queries, ideally before they consume a lot of resources. The Freebase API with [MQL](http://wiki.freebase.com/wiki/MQL) is already in use and seems to be useful to existing consumers. See http://www.freebase.com/query for an example query editor. With the announced integration of Freebase data into wikidata maintaining the same API sounds attractive, and could potentially let us reuse existing tools like the query editor. Another option could be to use an existing higher-level query language like [SPARQL](http://en.wikipedia.org/wiki/SPARQL), which also has [a JSON serialization](http://www.w3.org/TR/rdf-sparql-json-res/). TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: Aklapper, GWicke, jkroll, Smalyshev, Wikidata-bugs, aude, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T85181: Investigate design public API, possibly using MQL
GWicke removed a subscriber: Aklapper. GWicke added a project: Services. GWicke set Security to none. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: GWicke Cc: GWicke, jkroll, Smalyshev, Wikidata-bugs, Jdouglas, aude, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Changed CC] T85181: Investigate design public API, possibly using MQL
Smalyshev added a subscriber: Smalyshev. TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: GWicke, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T85181: Investigate design public API, possibly using MQL
Smalyshev added a comment. Are we considering supporting WDQ API mini-language as the option for the queries or it's not a viable option? TASK DETAIL https://phabricator.wikimedia.org/T85181 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: GWicke, Smalyshev, jkroll, Wikidata-bugs, Jdouglas, aude, Manybubbles, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
GWicke added a comment. Re performance and indexing, from a mail thread: Earlier today Stas I were looking a bit into what is happening behind the scenes in some of the slower queries like https://www.mediawiki.org/wiki/Wikibase/Indexing/Benchmarks#List_of_humans_having_occupation_writer_but_not_author We found that in a query that uses predicates on more than a single property per vertex, the default titan query strategy is to retrieve a list of candidates based on the first index (fast), and then load details for each vertex in order to filter on the second predicate. This is pretty slow, as it involves O(candidates) queries to Cassandra. I think mixed indexes as documented in http://s3.thinkaurelius.com/docs/titan/current/indexes.html#index-mixed should help here, as they support efficient matching on an arbitrary combination of attributes, along with advanced range and full text queries. They do require an indexing backend like Elasticsearch to be configured. Nik, is there an Elasticsearch setup we could use, or would it be reasonably easy to spin up a new one? Stas replied: I've looked into this matter, and there's one wrinkle in this - mixed indexes now don't support Date and SET fields, which means we can not index everything - some fields (among them frequently used date start/end fields and P*link fields which are SET) would not be indexable with Elastic. They say it's a temporary limitations but so far 0.9 still has it. Dates are actually interesting re the ranges we need to represent. A signed long second resolution unix timestamp (as used by Java Date)nbsp;reaches from 292269055 BC to 292278994 AD http://stackoverflow.com/questions/5488038/valid-range-for-java-util-date. Not quite enough to represent the current estimates for the Big Bang http://en.wikipedia.org/wiki/Big_Bang for example. ISO 8601 timestamps http://en.wikipedia.org/wiki/ISO_8601 support much less range, even in the six-digit extended mode. Maybe it makes sense to index years separately as a long? This will also be fun in JSON. For SET I could imagine a full-text index on a string with each P*link property separated by a space. Elasticsearch should be good at indexing that. Not sure how much more complex that would be to maintain compared to the current SET index. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev, GWicke Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T76373: Evaluate Titan as graph storage/query engine for Wikidata Query service
Smalyshev added a comment. For SET it won't be more complex to maintain, probably, but I'm not sure if the lookups would be fast enough. I could create an additional field for that and see how it behaves, and then we could drop the field that is not needed. For Date, I wonder if support can't be added to Titan, since Elastic AFAIK supports dates. Also, while Java can support negative years, right now the import does not support them since the Java parser fails to properly parse them. If we're moving to Java 8, there are better date APIs AFAIR, so maybe they allow to handle it properly. TASK DETAIL https://phabricator.wikimedia.org/T76373 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign username. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Smalyshev, Manybubbles, GWicke, JanZerebecki, aude, Lydia_Pintscher, Eloquence, aaron, jkroll, Wikidata-bugs, daniel ___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
