Ok, this isn't true because it is sorted by using HashComparator. For
some reason
generated list contains some parts wich are more or less sorted by host
and some
parts looks more "random".

This is consistent with what I am seeing; the Fetcher slowing down for
a while, sometimes coming to a virtual halt (a lot of repeated fetch
speed statements, for instance "412052 pages, 95915 errors, 3.0
pages/s, 638 kb/s,"), then the Fetcher speeding up again and fetching
at acceptable fetch speeds.

Rgrds, Thomas

Reply via email to