dr0ptp4kt added a comment.
In T355037#9508760 <https://phabricator.wikimedia.org/T355037#9508760>, @dcausse wrote: > @dr0ptp4kt thanks! is the difference in the number of successful queries only explained by the improvement in query time or are there some improvements in the number of queries that timeout as well? Good question! It appears to be related to query time. Looking at this latest run, for example, there were no recorded timeouts according to the CSV of the IGUANA `.nt` (🤔). Taking things on a head-to-head basis for identical queries between the endpoints, here's what we see for the difference in speed for `wikidata_main_graph` minus `baseline`. It's unsurprising in a way given the distribution shown in the prior Phabricator comment, but it is another way of knowing that, under this parameters of this test anyway, that about 70% of the queries noted as successful seemed to be faster when run against the `wikidata_main_graph`. Note that about 16% of the queries hit `wrongCodes` / `failed`, which are discussed after the table. | Per-query wikidata_main_graph QPS minus baseline QPS | descriptor | | ---------------------------------------------------- | ----------------------------------------------------- | | 0.722596509877809 | average | | 0.244672300065055 | median | | 79.4339558877256 | 100% max (i.e., wikidata_main_graph's biggest winner) | | 21.0654641024791 | 99% | | 6.88080533343067 | 95% | | 1.38414473312972 | 75% | | 0.244672300065055 | 50% | | 0.013982881368447 | 42% | | 0 | 41% | | 0 | 26% | | -0.00701117502390231 | 25% | | -0.215374628998983 | 20% | | -0.598658931613195 | 15% | | -1.41867399989265 | 10% | | -4.16152316076897 | 5% | | -18.0068429593504 | 1% | | -80.2800161266253 | 0% min (i.e., baseline's biggest winner) | | About 58% of queries titled toward `wikidata_main_graph`, and about 25% tilted toward `baseline`, and 58/(58+25) is about 0.7. The stuff where the difference is negligible probably don't matter that much. Yet, there's a bit more detail to consider in IGUANA's conception here... For the sake of completeness, and because this may be interesting to consider later on or to contextualize the QPS distributions in the prior Phabricator comment: looking at a different class of issues, let's suppose that we use `wrongCodes`as a proxy for things that could have gone wrong.`wrongCodes` and `failed` map to each other in the CSV, and their QPSes land as 0 for these (`penalizedQPS`, not included in the tables above, lands by default as 0.01666666667 for these records, but this is close enough to 0 if we wanted to look at it that way). These sorts of records thus drive down summary mean, median, and so on. As an aside, in terms of actual time (`totalTime`), these `wrongCodes` ones occupy very little time. | Endpoint Label | count wrongCodes | sum wrongCodes | count failed | sum failed | count timeout | count QPS < 1.0 | count QPS < 5.0 | count QPS < 20.0 | count QPS < 80.0 | count QPS < 200.0 | | ------------------- | ---------------- | -------------- | ------------ | ---------- | ------------- | --------------- | --------------- | ---------------- | ---------------- | ----------------- | | baseline | 12784 | 61863 | 12784 | 61863 | 0 | 12973 | 19750 | 46667 | 78649 | 78710 | | wikidata_main_graph | 12786 | 64129 | 12786 | 64129 | 0 | 12934 | 18909 | 44848 | 78642 | 78710 | | The `wrongCodes` resulted for basically the same queries across both endpoints (and each row of the CSVs is a per-query row, as we know) - they constitute about 16% of the queries. The ratio of `wikidata_main_graph` to `baseline` for `sum wrongCodes` (1.036629326091525) is very close to the ratio up above for `successfullQueries` (1.036316228814752). TASK DETAIL https://phabricator.wikimedia.org/T355037 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dr0ptp4kt Cc: dr0ptp4kt, dcausse, Aklapper, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, EBjune, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
