Lucas_Werkmeister_WMDE added a comment.
The requests prior to 2021-05-05 came from a variety of user agents, so I’m
willing to rule out the possibility that there were genuinely fewer accesses
after that date; I think it’s okay to post the top user agents (none of them
look very private):
$ zgrep -E -e '(latest|wikidata-[0-9]{8})-all\.json\.(gz|bz2)' -e
'(latest|wikidata-[0-9]{8})-all-BETA\.ttl\.(gz|bz2)' -e
'wikidatawiki-(latest|[0-9]{8})-(pages-articles-multistream|pages-meta-history|pages-meta-current|pages-articles)1?\.xml\.(gz|bz2)'
-e 'wikidatawiki-[0-9]{8}-pages-meta-hist-incr.xml\.(gz|bz2)'
/srv/log/webrequest/archive/dumps.wikimedia.org/access.log-20210[45]*.gz | awk
'$9 == "200" || $9 == "206"' | awk -F'"' '{ print $6 }' | sort | uniq -c | sort
-rn | head -25
11688 Go-http-client/1.1
6817 Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)
2565 Mozilla/5.0 (compatible; YandexOntoDB/1.0; +http://yandex.com/bots)
1259 Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36
980 Apache-HttpClient/4.5.9 (Java/1.8.0_265)
726 Apache-HttpClient/4.5.2 (Java/1.8.0_262)
707 curl/7.29.0
560 aria2/1.34.0
493 Wget/1.19.4 (linux-gnu)
421 python-requests/2.23.0
387 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36
329 Wget/1.21.1
136 python-requests/2.25.1
129 Wget/1.20.3 (linux-gnu)
122 Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1) QQBrowser/6.0
119 -
109 Mozilla/5.0 (compatible;AspiegelBot)
106 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/90.0.4430.93 Safari/537.36
85 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36
83 Wget/1.14 (linux-gnu)
72 Wget/1.20.1 (linux-gnu)
70 aria2/1.35.0
65 Python-urllib/3.7
62 python-requests/2.21.0
62 Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36
I also did a download myself, with a custom user agent, just to be sure:
$ curl -H 'User-Agent: Lucas-Werkmeister (really wants to see this request
in the access logs lucas.werkmeis...@wikimedia.de) curl/7.74.0'
https://dumps.wikimedia.org/wikidatawiki/entities/20220110/wikidata-20220110-all.json.bz2
> /dev/null
That was on Friday evening, and there’s no trace of it in the access logs:
$ zgrep -c Lucas
/srv/log/webrequest/archive/dumps.wikimedia.org/access.log-2022011{4..6}.gz
/srv/log/webrequest/archive/dumps.wikimedia.org/access.log-20220114.gz:0
/srv/log/webrequest/archive/dumps.wikimedia.org/access.log-20220115.gz:0
/srv/log/webrequest/archive/dumps.wikimedia.org/access.log-20220116.gz:0
TASK DETAIL
https://phabricator.wikimedia.org/T299358
EMAIL PREFERENCES
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
To: Lucas_Werkmeister_WMDE
Cc: Lucas_Werkmeister_WMDE, Aklapper, EChetty, Invadibot, maantietaja,
Akuckartz, 4748kitoko, Nandana, Akovalyov, Lahi, Gq86, GoranSMilovanovic,
QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, JAllemandou,
terrrydactyl, Wikidata-bugs, aude, Mbch331, jeremyb
_______________________________________________
Wikidata-bugs mailing list -- wikidata-bugs@lists.wikimedia.org
To unsubscribe send an email to wikidata-bugs-le...@lists.wikimedia.org
