If I want to index a site that has a policy of not allowing more than 1 hit
per second
And
If I set up the following:
MinDelay 1s
Server http://www.SiteName.com/
Then
If I execute index, I will only hit the above site once every second,
correct?
However
If I execute index -N 10
Then
I will hit the above site 10 times a second, correct?
This would violate the site policy and possibly result in banning my IP,
correct?
How can I make it so that all URLs with
http://www.SiteName.com%
are hit once a second
BUT
any referenced links are hit n times a second as I only want to hit the
first page of the referenced link so there should not be a problem.
Impossible?
Thanks in advance.
John
