https://bugzilla.wikimedia.org/show_bug.cgi?id=70721
Erik Zachte <ezac...@wikimedia.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |ezac...@wikimedia.org
--- Comment #11 from Erik Zachte <ezac...@wikimedia.org> ---
I scanned one day 1:1000 sampled squid log, so multiply all numbers by 1000
I find 1587 lines with index.html, of which only 34 without curid.
Most lines are like https://en.wikipedia.org/wiki/index.html?curid=32681660
Out of 1587 only 68 had a user agent that did not contain crawl,spider,bot or
http (http is by unofficial convention only user for bots)
Of the lines with index.html?curid= the following bots were found:
8 Android (compatible baidu spider)
13 AhrefsBot
113 Googlebot
1 Mail.RU_Bot
3 YandexBot
1337 bingbot
21 iPhone etc (but compatible GoogleBot)
1 Sogu web spider
Of course bingbot doesn't have to be Bing really. Some bots cloak.
Does this answer your question?
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
