[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Bawolff added a comment. In T289760#7874365 , @DD063520 wrote: > Hi, I saw the document, but I'm a bit confused. Because the criteria are basically checked for the shortlisted solutions. But in which part do you explain why for example

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

DD063520 added a comment. Hi, I saw the document, but I'm a bit confused. Because the criteria are basically checked for the shortlisted solutions. But in which part do you explain why for example Oxigraph was not shortlisted. I did not read the full document from start maybe i miss

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

MPhamWMF added a comment. @DD063520 , thanks for asking. Please refer to the attached paper for more details on the criteria we used to decide on our shortlist, which includes a note on Oxigraph: https://upload.wikimedia.org/wikipedia/commons/e/ea/WDQS_Backend_Alternatives_working_paper.pdf

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

DD063520 added a comment. What is the reason Oxigraph was shortlisted? Is this documented somewhere? TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: DD063520 Cc: DD063520, MPhamWMF, Jerven,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

MPhamWMF added a comment. I'm closing this task since Oxigraph is not on the shortlist of Blazegraph alternatives, as per https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_update/WDQS_backend_alternatives TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

MPhamWMF closed this task as "Declined". TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: MPhamWMF, Jerven, AndySeaborne, BenAtOlive, Justin0x2004, Izno, Gehel, Thadguidry, Tpt, So9q,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

MPhamWMF lowered the priority of this task from "Medium" to "Low". TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Jerven, AndySeaborne, BenAtOlive, Justin0x2004, Izno, Gehel,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Tpt Cc: Jerven, AndySeaborne, BenAtOlive, Justin0x2004, Izno, Gehel, Thadguidry, Tpt, So9q, Aklapper, Invadibot,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Thadguidry added a comment. @BenAtOlive I think for bikeshedding or hand-waving discussions, you can just start an new discussion thread in Oxigraph's GitHub Discussions (not Issues). Here: https://github.com/oxigraph/oxigraph/discussions TASK DETAIL

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt added a comment. @Thadguidry @BenAtOlive It's a great idea to move the conversation somewhere else. Oxigraph has a discussion space and a Gitter channel . TASK DETAIL

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

BenAtOlive added a comment. @Thadguidry - that's a very good point. We've done the non-temporary N replica model with Fuseki before and have the pods fronted by an ALB, but this doesn't really distribute anything--it just gives some horizontal scalability. I don't want to take over this

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Thadguidry added a comment. As someone who has "been there, done that" (even with Apache Geode)... I can tell you that **data locality** is very important when you want to maximize performance. But if the data is maintained as distributed, then the only way to squeeze out improved

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

BenAtOlive added a comment. @Tpt - thank you for the detailed response. I've been wondering about this lately, and the tradeoffs you outlined are definitely a strong reason to not do it. Distributed SQL/NoSQL databases have proven to be quite powerful and it is inevitable to think can we

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt added a comment. @BenAtOlive If I understand correctly, you are asking if Oxigraph can be used to build a distributed SPARQL database with reader nodes and writer nodes? Its definitely possible to reuse Oxigraph components for that. For example I am currently moving the SPARQL parser

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

BenAtOlive added a comment. @Tpt - is there a plan to decouple the Sled store and the query handlers in Oxigraph to allow the use case that @Thadguidry was asking about? Under high, concurrent write situations (i.e. if writers to Oxigraph weren't limited to a constrained number that kept a

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt added a comment. @Thadguidry Great! Thank you! @So9q Sadly it's not possible with Sled. But Sled and Oxigraph should both work fairly well with multiple parallel thread so updating data while serving query should work ok. I have started to run some benchmarks around it. I still

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q added a comment. @Tpt is it possible to run a cluster of say 2 oxigraph instances on the same sled store? I could then manually update the sled store in some way and the changes are visible in both of the oxigraph endpoints? TASK DETAIL https://phabricator.wikimedia.org/T289760

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q added a comment. @Tpt thanks for the corrections! TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: Gehel, Thadguidry, Tpt, So9q, Aklapper, Invadibot, MPhamWMF, maantietaja,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: Gehel, Thadguidry, Tpt, So9q, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen, Akuckartz, Nandana,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Thadguidry added a comment. @Tpt Looks great! The ROADMAP file was a suggested alternative to the Milestones, sorry didn't make that clear. I much prefer grouping or tagging issues against Milestones as you have done! You have the right idea regarding a single source of truth and exactly

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt added a comment. @Thadguidry Hi! It's a great idea to improve the roadmap. Thank you for pushing me to do that! I have updated the GitHub issues and milestones and added a link to them in the README:

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Gehel triaged this task as "Medium" priority. Gehel moved this task from All WDQS-related tasks to Scaling on the Wikidata-Query-Service board. Gehel added a comment. The Search Platform team will dig into this when we start work on evaluating Blazegraph alternatives TASK DETAIL

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Thadguidry added a comment. Hi @Tpt Can you elaborate more in your Milestones and create more Milestone as necessary for your future vision? Like what you mean by "no storage format stability for now", and what that really means to users and what you are thinking about in the long term

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Tpt added a comment. Thank you @So9q for opening this task! Some comments: - Oxigraph is dual licensed under Apache2 or MIT, not under GPLv2. - Oxigraph plans to drop the RocksDB backend in the next release and focus on Sled. It will make the development easier. Some other

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

Maintenance_bot added a project: Wikidata. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Maintenance_bot Cc: Thadguidry, Tpt, So9q, Aklapper, Invadibot, MPhamWMF, maantietaja, CBogen,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: So9q, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: So9q, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q updated the task description. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: So9q, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, Jonas,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q added a parent task: T206560: [Epic] Evaluate alternatives to Blazegraph. TASK DETAIL https://phabricator.wikimedia.org/T289760 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: So9q, Aklapper, MPhamWMF, CBogen, Namenlos314, Gq86,

[Wikidata-bugs] [Maniphest] T289760: Evaluate Oxigraph as alternative to Blazegraph

So9q created this task. So9q added a project: Wikidata-Query-Service. Restricted Application added a subscriber: Aklapper. TASK DESCRIPTION pros: - sparql 1.1 and sparql update - storage in RocksDB which is a scalable key-value store - gplv2 - no enterprise version cons: -