dcausse moved this task from In Progress to Needs Reporting on the Discovery-Search (Current work) board. dcausse added a comment.
**TFT** does not seem appropriate for the kind of tests we have to make within the scope of the graph split project, it does not provide anything to ease the creation of test sets, I created T351819 <https://phabricator.wikimedia.org/T351819> to create such tooling. **IGUANA** does seem more promising, after a couple fixes I was able to run a simple test against our public endpoint, User:AndreaWest/WDQS_Testing/Running_Iguana#How_to_Change_or_Extend_the_Tests_and_Infrastructure <https://wikitech.wikimedia.org/wiki/User:AndreaWest/WDQS_Testing/Running_Iguana#How_to_Change_or_Extend_the_Tests_and_Infrastructure> gives a good overview of the extracted metrics. The test output in its current format (RDF) might not be directly actionable by a data-analyst and might need some transformation to a kind of tabular data. I used IGUANA version 3.3.3 and had to apply several fixes visible in https://github.com/nomoa/IGUANA/commits/wip, we should consider using the new metrics added by Andrea in QPSMetric <https://github.com/AndreaWesterinen/IGUANA/blob/333797d05c7d960d1efebc1e9fdc434771487047/iguana.resultprocessor/src/main/java/org/aksw/iguana/rp/metrics/impl/QPSMetric.java> (meanQueryTime, geometricMeanQueryTime, min, max and their penalized version). I attempted to upstream my fixes but it appears that the IGUANA maintainers no longer want to maintain the 3.x branch because they're working on a v4 version which is a major refactor, I did not have time to look too deeply into it... For the wdqs graph split needs it might OK to use our v3.3.3 fork (https://gitlab.wikimedia.org/repos/search-platform/IGUANA) but for blazegraph alternatives evaluation we might consider rebasing Andrea's work on the upcoming v4 branch depending on its status. TASK DETAIL https://phabricator.wikimedia.org/T349519 WORKBOARD https://phabricator.wikimedia.org/project/board/1227/ EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: AWesterinen, dcausse, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Namenlos314, Lahi, Gq86, Lucas_Werkmeister_WMDE, GoranSMilovanovic, QZanden, EBjune, merbst, LawExplorer, _jensen, rosalieper, Scott_WUaS, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
