Dear all,I am happy to announce that as part of an ongoing research collaboration between TU Dresden researchers and Wikimedia [1], we could now release pre-processed logs from the Wikidata SPARQL Query Service [2]. You can find details and download links on the following page:
https://iccl.inf.tu-dresden.de/web/Wikidata_SPARQL_Logs/enThe data so far comprises over 200 million queries answered in June-August 2017. There is also an accompanying publication that describes the workings of and practical experiences with the SPARQL query service [3].
The logs have been pre-processed to remove information that could potentially be used for identifying individual users (e.g., comments were removed, geo-coordinates coarsened, and query strings reformatted completely -- see above page for details). Nevertheless, one can still learn many interesting things from the logs, e.g., which properties and entities are used in queries, which SPARQL features are most prominent, or which languages are requested.
We also have preserved some amount of user agent information, but without overly detailed software versions and only in cases where the agents occurred many times across several weeks. This can at least be used to recognise the (significant amount) of queries generated, e.g., by Magnus' tools, or to do a rough analysis of which software platforms are mostly used to send queries from. We used #TOOL comments found in queries to refine user agent information in some cases.
We also made an effort to identify those queries that come from browser agents *and* also behave like one would expect from a browser (not all "browsers" did). We called such queries "organic" and provide this classification with the logs (there is also a filtered dump of only organic queries, which is much smaller and therefore nicer to process, also for testing). See the paper for details on our methodology.
Finally, the data contains the time of each request, so one can reconstruct query loads over time.
Feedback is very welcome, both in terms of comments on the data (is it useful to you? would you like to see more? do you have concerns?) and in terms of insights that you can get from it (we did some analyses but one can surely do more).
Cheers, Markus [1] https://meta.wikimedia.org/wiki/Research:Understanding_Wikidata_Queries[2] https://query.wikidata.org/ (or rather the web service that powers this UI and many other applications). [3] Stanislav Malyshev, Markus Krötzsch, Larry González, Julius Gonsior, Adrian Bielefeldt: Getting the Most out of Wikidata: Semantic Technology Usage in Wikipedia’s Knowledge Graph. In Proceedings of the 17th International Semantic Web Conference (ISWC-18), Springer 2018. https://iccl.inf.tu-dresden.de/web/Wikidata_SPARQL_Logs/en
-- Prof. Dr. Markus Kroetzsch Knowledge-Based Systems Group Center for Advancing Electronics Dresden (cfaed) Faculty of Computer Science TU Dresden +49 351 463 38486 https://kbs.inf.tu-dresden.de/
smime.p7s
Description: S/MIME Cryptographic Signature
_______________________________________________ Wikidata mailing list Wikidata@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata
