On Tue, 4 Nov 2003 [EMAIL PROTECTED] wrote: > Hello Robots list > > Well maybe this list can finally put to rest a great deal of the "30 second wait" > issue. > > Can we all collectively research into an adaptive routine?
Interesting topic... With one hat on, I operate one of those little servers with thousands of pages. I guess I'm lucky; I don't pay bandwidth and the connection is naturally limited to a T-1. With my other hat on, at TRIUMF we have started to have issues with bandwidth management. We now have a gigabit link to the research networks with no byte charges, so don't care if someone sucks our site from ESnet (CERN, Fermilab, Los Alamos etc.). However, we have a 100Mbit link to commercial backbone and can't afford to fill it - P2P is a problem. Our current "solution" is to limit outgoing traffic to 1Mbit - except our central webserver and mailserver. So we would be financially embarrassed if a lot of robots from the commercial side all decided to mirror our servers. I guess what I'm trying to say is that the issue of instantaneous hit rate is not really a problem any more, but that volume might be. However, I guess that the people running robots also have finite storage and have to pay for bandwidth, so that perhaps this is a non-problem except where there is a serious asymmetry between source and destination. -- Andrew Daviel, TRIUMF, Canada Tel. +1 (604) 222-7376 [EMAIL PROTECTED] _______________________________________________ Robots mailing list [EMAIL PROTECTED] http://www.mccmedia.com/mailman/listinfo/robots
