AndrewTavis_WMDE added a comment.
Having derived quick samples (`DISTRIBUTE BY rand()` to mix it up, but nothing more), what I'm seeing is that the comment queries look to be very similar to one another regardless of if they're spiders or non-spiders. Could be that what we're thinking of as a non-spider just isn't being picked up by the UDF. Each of them has a `PREFIX target: <http://www.wikidata.org/entity/QID_TARGET>` at the top that then is assigned further on down the query. A final check for this could be to see how our counts above would change if we took the part of the query after this assignment is made such that we just have the template query. My expectation is that the whole distribution will shift to the left such that we have dramatically more unique queries at all levels in that the distinct part for most of these seems to be the QID in question. The queries themselves are varied based on all manner of things that could be found out about researchers: student-supervisor relationships, number of publications, etc, with default views like maps, graphs, bar charts and others being assigned at the top between the `#tool: scholia` comment and the prefix assignment. The user agent based queries are totally different and normally extremely small. Templates for these are: select ?class where { wd:QID_TARGET wdt:P279+ ?class } SELECT ?class { wd:QID_TARGET wdt:P31 ?class } SELECT ?doi { wd:QID_TARGET wdt:P356 ?doi } So it seems that the assumption that these are for automated tooling was correct :) Minor helper queries to check what things are. Let me know if a check of the queries based after removing the the prefix assignment would be helpful. At the very least this would give us an idea of the total number of query templates they have. TASK DETAIL https://phabricator.wikimedia.org/T353453 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: AndrewTavis_WMDE Cc: Lydia_Pintscher, dcausse, Aklapper, Manuel, Danny_Benjafield_WMDE, Astuthiodit_1, karapayneWMDE, Invadibot, maantietaja, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, KimKelting, LawExplorer, _jensen, rosalieper, Scott_WUaS, Wikidata-bugs, aude, Mbch331
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
