On Thu, 2005-10-13 at 13:35 -0700, AJ Chen wrote: > I try to fetch as fast as it can by using more threads on a large fetch > list. But, the fetcher starts download at speed much lower than the full > bandwidth allows. And the start download speed varies a lot from run to run, > 200kb/s to 1200kb/s on my DSL line. This variation also happens on T1 line > that I just tested. > Could someone share experience on how to make fetcher use the full > bandwidth? We know the speed drops gradually during a long fetch run. But, > can the fetch achieve the highest speed allowed by the bandwidth when fetch > starts?
I found that for high bandwidth (50Mbits and above) DNS seems to be a limiting factor. 4000 threads with a local caching DNS server seems to be enough to fill the pipe though -- Rod Taylor <[EMAIL PROTECTED]>
