[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
AWesterinen closed this task as "Resolved". AWesterinen added a comment. Criteria are defined in the paper, WDQS Backend Alternatives, published on the page, https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_alternatives. TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AWesterinen Cc: Jneubert, Daniel_Mietchen, nguyenm9, AndySeaborne, YULdigitalpreservation, Iamamz3, Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, CBogen, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
Gehel assigned this task to AWesterinen. TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AWesterinen, Gehel Cc: Jneubert, Daniel_Mietchen, nguyenm9, AndySeaborne, YULdigitalpreservation, Iamamz3, Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, karapayneWMDE, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
MPhamWMF moved this task from Scaling to Current work on the Wikidata-Query-Service board. MPhamWMF added a project: Discovery-Search (Current work). TASK DETAIL https://phabricator.wikimedia.org/T291207 WORKBOARD https://phabricator.wikimedia.org/project/board/891/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Jneubert, Daniel_Mietchen, nguyenm9, AndySeaborne, YULdigitalpreservation, Iamamz3, Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
Iamamz3 added a comment. Re the prior candidate list: > Prior candidate list and survey from when Blazegraph was chosen: https://docs.google.com/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB-ZkMqT8Y5b2NYVKbU/edit?usp=sharing It would be great to have annotated scale to help figure what software looks like the best candidate, and avoid gut jugdment. Given a "multi operation ACID", it might look like: - 0: No ACID guarantees - 1: ACID guarantees for primary representation, but async secondary representations (indices) - 2: ACID both primary and secondary representations Regarding ACID in particular, there is another lever that is "isolation level", see https://www.postgresql.org/docs/current/transaction-iso.html. Also the current scale 0-10 is way to large, it is too much work to document for every row what every number means between zero and ten. It seems clear that that sheet is just an indicator, and only gives clues of what might work best, and grading well on that can not be the primary motivation for picking a solution. > Design for ~10X growth, but plan to rewrite before ~100X > > Jeff Dean, “Challenges in Building Large-Scale Information Retrieval Systems,” Google, http://static.googleusercontent.com/media/research.google.com/en//people/jeff/WSDM09-keynote.pdf TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Iamamz3 Cc: Iamamz3, Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
Versant.2612 added a comment. Consider graph databases that support RDF-star and SPARQL-star such as RDF4J, AnzoGraph and GraphDB since they are proposed extensions to the RDF and SPARQL standards to provide a more convenient way to annotate RDF statements and to query such annotations (wikidata qualifiers and references), bridging the gap between the RDF world and the Property Graph world. See W3C Draft Community Group Report 01 July 2021 https://www.w3.org/community/rdf-dev/2021/07/02/new-public-draft-of-the-rdf-star-report/ https://rdf4j.org/documentation/programming/rdfstar/ https://graphdb.ontotext.com/enterprise/devhub/rdf-sparql-star.html https://cambridgesemantics.com/anzo-platform/ TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Versant.2612 Cc: Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
Versant.2612 added a comment. Hernández, Daniel & Hogan, A. & Krötzsch, M.. (2015). Reifying RDF: What works well with wikidata?. 1457. 32-47. Abstract: In this paper, we compare various options for reifying RDF triples. We are motivated by the goal of representing Wikidata as RDF, which would allow legacy Semantic Web languages, techniques and tools - for example, SPARQL engines - to be used for Wikidata. However, Wikidata annotates statements with qualifiers and references, which require some notion of reification to model in RDF. We thus investigate four such options: (1) standard reification, (2) n-ary relations, (3) singleton properties, and (4) named graphs. Taking a recent dump of Wikidata, we generate the four RDF datasets pertaining to each model and discuss high-level aspects relating to data sizes, etc. To empirically compare the effect of the different models on query times, we collect a set of benchmark queries with four model-specific versions of each query. We present the results of running these queries against five popular SPARQL implementations: 4 store, BlazeGraph, GraphDB, Jena TDB and Virtuoso. TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Versant.2612 Cc: Versant.2612, Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
Fnielsen added a comment. QLever - https://github.com/ad-freiburg/qlever - https://scholia.toolforge.org/work/Q108730896 The paper reports benchmarks favorable for QLever. I cannot get path queries working on the public endpoint. TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Fnielsen Cc: Fnielsen, Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
MPhamWMF triaged this task as "High" priority. TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
[Wikidata-bugs] [Maniphest] T291207: Create list of criteria for graph backend candidates for WDQS
MPhamWMF created this task. MPhamWMF added projects: Wikidata, Wikidata-Query-Service. TASK DESCRIPTION As a WDQS maintainer, I want to be able to evaluate graph backend candidates for migrating WDQS off of Blazegraph, so that I can create a ranking/survey of alternatives, and ultimately choose the optimal one. Prior candidate list and survey from when Blazegraph was chosen: https://docs.google.com/spreadsheets/d/1MXikljoSUVP77w7JKf9EXN40OB-ZkMqT8Y5b2NYVKbU/edit?usp=sharing We will likely not use the same list as before, and will need to create a new list of criteria (and weighting of those criteria). The final list will aim to combine technical scaling considerations, as well as a relatively small finite list of community-sourced criteria (in the case that they do not totally overlap). While there will eventually be a better process for consolidating a final list of community-sourced criteria, (comments in) this ticket can be used to start collecting ideas for criteria. AC: - a list of criteria to evaluate graph backends for the purpose of scaling WDQS TASK DETAIL https://phabricator.wikimedia.org/T291207 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Aklapper, Lucas_Werkmeister_WMDE, Justin0x2004, MPhamWMF, Invadibot, maantietaja, CBogen, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331 ___ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
