AndrewTavis_WMDE added a comment.
Quick counts as in the sampling task to check uniqueness of queries and HTTP
statuses (I don't think that other measures like variance over weeks, duration
or char size would add much). Note that percentages below are for the
sub-groups, not for all Scholia queries. Period for the following is all
queries 90 days to the date of posting.
All queries with the `#tool: scholia` comment
---------------------------------------------
| query_count | total_queries | percent_of_queries |
| ----------- | ------------- | ------------------ |
| 1 | 3,089,287 | 68.0434662425413 |
| 2 | 725,985 | 15.9902708424602 |
| 3 | 187,262 | 4.124561937919904 |
| 11 | 68,887 | 1.5172789899578585 |
| 4 | 67,923 | 1.4960462908082455 |
| 10 | 44,083 | 0.9709554736642947 |
| 12 | 33,874 | 0.7460959035207295 |
| 5 | 27,986 | 0.6164090439845055 |
| 9 | 27,554 | 0.6068939754859237 |
| 13 | 26,602 | 0.5859255837946049 |
|
| http_status | total_queries | percent_of_queries |
| ----------- | ------------- | --------------------- |
| 200 | 12,283,936 | 99.12437793762604 |
| 500 | 106,062 | 0.8558600250620397 |
| 429 | 2,305 | 0.01860003920129737 |
| 400 | 122 | 9.84470621500338E-4 |
| 503 | 22 | 1.7752748912301178E-4 |
|
Spider Comment Queries
----------------------
| query_count | total_queries | percent_of_queries |
| ----------- | ------------- | ------------------ |
| 1 | 2,884,240 | 68.91420417646493 |
| 2 | 636,348 | 15.204496158185558 |
| 3 | 166,386 | 3.975521723610135 |
| 11 | 68,714 | 1.6418088043233612 |
| 4 | 52,954 | 1.2652493440076154 |
| 10 | 43,303 | 1.0346544612977635 |
| 12 | 33,187 | 0.7929491630392554 |
| 9 | 26,470 | 0.6324574184364085 |
| 13 | 26,275 | 0.6277982119160043 |
| 14 | 25,814 | 0.6167833698344333 |
|
Non-spider Comment Queries
--------------------------
| query_count | total_queries | percent_of_queries |
| ----------- | ------------- | ------------------ |
| 1 | 292,734 | 66.14231938940128 |
| 2 | 94,555 | 21.364402528796926 |
| 3 | 15,615 | 3.528159753446819 |
| 4 | 14,178 | 3.203474158461031 |
| 5 | 4,670 | 1.0551716969962628 |
| 6 | 4,602 | 1.0398073125432123 |
| 8 | 2,513 | 0.5678043842722931 |
| 7 | 2,475 | 0.5592184047250001 |
| 10 | 1,476 | 0.33349752136327276 |
| 9 | 1,398 | 0.31587366860830307 |
|
All queries with the `"Scholia"` user agent
-------------------------------------------
| query_count | total_queries | percent_of_queries |
| ----------- | ------------- | ------------------ |
| 1 | 3,938,171 | 53.733959138039936 |
| 2 | 1,578,024 | 21.531182148983962 |
| 3 | 808,041 | 11.025230259392222 |
| 4 | 431,361 | 5.885659700339077 |
| 5 | 234,441 | 3.1988055151188766 |
| 6 | 131,593 | 1.7955068189908687 |
| 7 | 77,142 | 1.0525558884636235 |
| 8 | 46,645 | 0.6364427862563288 |
| 9 | 28,437 | 0.3880056493251414 |
| 10 | 17,087 | 0.23314177058123892 |
|
| http_status | total_queries | percent_of_queries |
| ----------- | ------------- | -------------------- |
| 200 | 15,193,039 | 99.9989205699417 |
| 500 | 86 | 5.660425915457063E-4 |
| 429 | 60 | 3.949134359621207E-4 |
| 503 | 12 | 7.898268719242414E-5 |
| 400 | 6 | 3.949134359621207E-5 |
|
Discussion
----------
Nothing jumps out per say from the above. Big thing is that we have a higher
percentage of unique queries coming from those identified via a comment than
those that are identified via a user agent. I don't think that we can say that
one is for traffic/API and another is for general tooling because:
- Higher unique queries from those with a comment would possibly indicate
unique user searches
- But then the commented queries also seem more "routine" where we have a
higher percentage of queries being ran 11, 10 and 12 times in the period
- The distribution of the user agent identified queries is more in line
with user behavior
As far as HTTP status, 99+% regardless of all the queries is great :)
Final thing from what's been discussed so far is looking at some examples
from the above breakdowns.
TASK DETAIL
https://phabricator.wikimedia.org/T353453
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: AndrewTavis_WMDE
Cc: Lydia_Pintscher, dcausse, Aklapper, Manuel, Danny_Benjafield_WMDE,
Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz,
Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, LawExplorer,
_jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________
Wikidata-bugs mailing list -- [email protected]
To unsubscribe send an email to [email protected]
