Does anyone have a spidering/ops/engineering contact at
FAST?
One of our customers was overloaded by a robot from FAST
(source IP: 216.204.56.75, hostname: qa75.bos.fast.no).
The pages had a "nofollow" robots meta tag, so I would
have expected the robot to give up, but it was keeping
eight server threads busy for minutes at a time.
The customer has added a robots.txt, which we hope will
calm things down.
>From the hostname, this is probably a QA machine, and
everybody makes mistakes (thus the QA department).
FAST's website gives sales contacts and the spider's
user-agent ("fast"), but no contact for spider problems.
wunder
--
Walter Underwood
[EMAIL PROTECTED]
Senior Staff Engineer, Inktomi
http://www.inktomi.com/
