Hi, to avoid "rapid fire" in current CVS version implemented urls merging by seed. Try it if you want.
Martin Ebert wrote: > Author: Martin Ebert > Email: [EMAIL PROTECTED] > Message: > Hi all, > > system parameters at first: > P4/1,6GHz, 1,5GBmain, mnogo 3.1.19, cache-mode. > About 5.000 "Server"-Directives, about 700.000 URLs with Status 0 (not indexed yet). > > 7 indexer parallel; started with ./indexer -e -o -q & > CPU-Load: 100%; Performance: 300 URL/min. > > Big problem: Today I've received a mail from a webmaster of a spidered site. He >says: "Hups, big load! About 5 request per second..." > > Ohoh. My idea of mnogo was: mnogo has a system like "load banancing". Something like >"we have 700.000 URLs from 5.000 servers; let me rotate the requests between the >servers". > > But mnogo hasn't such a feature, right? > > What can I do in this case? > > The option "-p 5" is not a good way: I've understood, that these option says: >"indexer, stop for 5 seconds". Right? > > Any ideas? > > Thnaks, Martin Ebert -- Maxim Zakharov http://sochi.net.ru/~maxime/ Sochi, Russia http://www.sochi.com/ ___________________________________________ If you want to unsubscribe send "unsubscribe general" to [EMAIL PROTECTED]
