[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Pulquero added a comment. That is a shame, I've been making improvements to really give the other stores a run for their money. Well, for reference, the results so far are that in a single-box configuration, the current dump requires <1.5TB of disk space loaded. And a preliminary test of

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

MPhamWMF added a comment. I'm closing this task since Halyard is not on the shortlist of Blazegraph alternatives, as per https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/WDQS_backend_update/WDQS_backend_alternatives TASK DETAIL https://phabricator.wikimedia.org/T289621 EMAIL

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

MPhamWMF closed this task as "Declined". TASK DETAIL https://phabricator.wikimedia.org/T289621 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: MPhamWMF, Pulquero, Hannah_Bast, nguyenm9, Smalyshev, Aklapper, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

MPhamWMF lowered the priority of this task from "Medium" to "Low". TASK DETAIL https://phabricator.wikimedia.org/T289621 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: MPhamWMF Cc: Pulquero, Hannah_Bast, nguyenm9, Smalyshev, Aklapper,

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Pulquero added a comment. For clarity, RDF-star. TASK DETAIL https://phabricator.wikimedia.org/T289621 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Pulquero Cc: Pulquero, Hannah_Bast, nguyenm9, Smalyshev, Aklapper, Lucas_Werkmeister_WMDE,

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

TallTed added a comment. In T289621#7685585 , @Pulquero wrote: > Hi, I would like to suggest my Halyard fork, Halyard* (supports RDF*), https://github.com/pulquero/Halyard, for consideration. Note that `RDF*` is no longer being

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Hannah_Bast added a comment. @Pulquero AFAIK the two main problems with Blazegraph are: 1. The project is not really active anymore: https://github.com/blazegraph/database . The reason is that the Blazegraph team was acqui-hired by Amazon a few years ago, and Blazegraph essentially

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Pulquero added a comment. 1. No. 2. I'm happy to try to work with someone here to make that happen. I've already been in discussions with @nguyenm9. 3. You need at least a single HBase node. It can all be run on a single machine. What is the spec of the current machine you are using?

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Hannah_Bast added a comment. > @Hannah_Bast maybe this interests you? Do you think this system would perform well considering the load on WDQS and the type of queries we have? @So9q Sorry, Dennis, I forgot to answer this question of yours. In general, I think that any system that is

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Hannah_Bast added a comment. @Pulquero Thank you for this interesting piece of information. I have a few questions: 1. Do you have a running SPARQL endpoint for Halyard* on Wikidata for us to play around with? 2. If not, how hard would it be for you to set one up? 3. Is it a

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Pulquero added a comment. Hi, I would like to suggest my Halyard fork, Halyard* (supports RDF*), https://github.com/pulquero/Halyard, for consideration. It contains numerous non-trivial changes beyond the original to handle high-volume transactional type queries. And more crucially, it

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

nguyenm9 added a comment. I wouldn't discount this project just yet as Merck may have decided to move away from rdf altogether. It is a bit of a commitment and for profit companies need to juggle costs. I also say that bc I have been able to get it to compile against fairly recent libs and

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

TheKtk added a comment. The project was mainly driven by Merck and according to one of the (at least former) persons involved in the project, it is no longer developed there: https://twitter.com/jindrichmynarz/status/1424976369495199744 TASK DETAIL

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

So9q added a comment. In T289621#7660272 , @nguyenm9 wrote: > That's a bit disappointing b/c it does look like it can scale and has been run through some paces.

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

So9q added a comment. In T289621#7660272 , @nguyenm9 wrote: > That's a bit disappointing b/c it does look like it can scale and has been run through some paces.

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

So9q added a subscriber: Hannah_Bast. So9q added a comment. Here is their sparql evaluation strategy: > Actual Halyard Evaluation Strategy turns the previous model inside-out. I call it "PUSH Model". The SPARQL query is transformed into a chain (or tree) of pipes (Binding Set Pipe) and

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

nguyenm9 added a comment. That's a bit disappointing b/c it does look like it can scale and has been run through some paces. https://www.linkedin.com/pulse/halyard-tipstricks-trillion-statements-challenge-adam-sotona/ I'm actually trying to get this to compile with the latest versions

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

So9q added a comment. I researched this solution a little: https://merck.github.io/Halyard/img/architecture.png F34934409: architecture.png - Here is an overview of the mapping and explanation of the choice of hashing of triples and quads

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Gehel moved this task from All WDQS-related tasks to Scaling on the Wikidata-Query-Service board. Gehel triaged this task as "Medium" priority. Gehel added a comment. The Search Platform team will dig into this when we start work on evaluating Blazegraph alternatives TASK DETAIL

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

So9q added a comment. cons: - no commits since dec 2019 https://github.com/Merck/Halyard/commits/master TASK DETAIL https://phabricator.wikimedia.org/T289621 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: So9q Cc: Smalyshev, Aklapper,

[Wikidata-bugs] [Maniphest] T289621: Evaluate Halyard as alternative to Blazegraph

Jerven created this task. Jerven added projects: Wikidata-Query-Service, Epic, Wikidata, MediaWiki-Stakeholders-Group. TASK DESCRIPTION An HBase/RDF4J based horizontal scaling sparql service originally developed by a team at Merck. https://github.com/Merck/Halyard TASK DETAIL