On Wed, 8 Mar 2000, Tony Wasserman wrote:
> 1) Making the change to
> REQFLOOR 1r
> didn't do the job. The request report still doesn't find the .jsp pages.
> REQFLOOR -100r
>
> Adding
> REQINCLUDE *
> didn't help either, but Analog found *lots* of GIF files that were previously
> excluded by the
> REQINCLUDE pages
> command
>
Well, if REQFLOOR 1r and REQINCLUDE * together don't find them, then you
must have excluded or aliased them somehow, or else those lines are corrupt.
I suggest you make a logfile with just half a dozen of the *.jsp requests
and that should enable you to identify what's going on.
> 2) The Keynote requests come from lots of different hosts. The idea is
> that they
> are testing the responsiveness of various web sites (particularly the most
> popular
> ones) to provide statistics on the overall performance of the web for
> commercial
> interests. As a result, they originate their requests from lots of
> different hosts
> and IP addresses. The only way to exclude is on some kind of browser exclude.
>
Well, I don't know what these ones have for their browser string. But from
your previous message, I gathered that it was indistinguishable from a
regular Mozilla.
In general, you can't identify all spiders. If they don't come from
a distinctive host, and don't have a distinctive browser name, there's
nothing to separate them from normal requests.
--
Stephen Turner http://www.statslab.cam.ac.uk/~sret1/
Statistical Laboratory, 16 Mill Lane, Cambridge CB2 1SB, England
"8th March 2000. National No Smoking Day. Ash Wednesday." (On a calendar)
------------------------------------------------------------------------
This is the analog-help mailing list. To unsubscribe from this
mailing list, send mail to [EMAIL PROTECTED]
with "unsubscribe" in the main BODY OF THE MESSAGE.
List archived at http://www.mail-archive.com/[email protected]/
------------------------------------------------------------------------
