I noticed that one of the sqlite contrib files had 1500 downloads in a single day this past week, which is around 100X normal. I'm guessing it was a robot.
I see that Google and many other websites' robots.txt prefer this form: Disallow: /contrib/download/ over this form: Disallow: /contrib/download for directories. Is the trailing slash required to disallow directories for (some) robots? Also, what does the trailing question mark mean in robots.txt? Disallow: /foo/bar? Is it a wildcard or does it literally mean the '?' character? ____________________________________________________________________________________ The fish are biting. Get more visitors on your site using Yahoo! Search Marketing. http://searchmarketing.yahoo.com/arp/sponsoredsearch_v2.php ----------------------------------------------------------------------------- To unsubscribe, send email to [EMAIL PROTECTED] -----------------------------------------------------------------------------
