Your you could cut out the ones that can’t be relevant such as foreign language countries. One of my web logs shows substantial access from Russian Federation, China, Latvia, Ukraine, France, Germany, Moldovia, Netherlands, Slovakia, Malaysia, KAZAKHSTAN, and Israel.
Is this normal – or am I being spied on !!?? From: [email protected] [mailto:[email protected]] On Behalf Of mike smith Sent: Sunday, 21 July 2013 9:15 PM To: ozDotNet Subject: Re: [OT] Search engines (not a complaint) But if you can't be found on a search engine, who's going to come to your site? Word of mouth isn't how you get most traffic. If you cut out the 52% of search engine bots, the other 48% won't know you exist. On Fri, Jul 19, 2013 at 5:46 PM, Greg Keogh <[email protected]> wrote: Just a bit of Friday technical trivia that might warn others ... I analysed all of the 2013 data so far from my IIS web logs and found the following interesting facts: robots.txt was read 11270 times My total sent bytes was 10.2GB of which 5.3GB was searching engines and bots 143374 requests came from bots (%46 of the total) So %52 of all traffic out of my web site was food for bots. I think this is an extraordinary volume of data and I have declared war on them. I found a sample robots.txt file on the web which looks quite comprehensive, then I added others ones I found in my logs. I installed the "IP Address and Domain Restrictions" Role to IIS 7.5 so I can totally block the worst offenders (I'm looking at you Ezooms and GoogleImages!). It will be interesting to see how my web request stats change over the coming months. I think this "search engine" zoo is now well out of control in the wild. So I've got charities and market researchers pestering me on the phone all day while my web server is pestered with a flood of bots. Greg -- Meski <http://courteous.ly/aAOZcv> http://courteous.ly/aAOZcv "Going to Starbucks for coffee is like going to prison for sex. Sure, you'll get it, but it's going to be rough" - Adam Hills No virus found in this message. Checked by AVG - www.avg.com Version: 2013.0.3349 / Virus Database: 3204/6507 - Release Date: 07/20/13
