On 5/24/07, Vishal Shah <[EMAIL PROTECTED]> wrote: > Hi Dogacan, > > I haven't yet checked it with the 474 patch. Thanks for that update! > > What I was trying to say was that even if fetcher2 works perfectly, it > can only give you a significant performance boost when the number of threads > per task is much less than the number of hosts that will be fetched by that > task.
I don't really want Fetcher2 to be faster. I just want it to be as fast as Fetcher with a smaller number of threads which is a reasonable expectation :) > > Do you have any numbers about the number of hosts from which you are > fetching? I have ~30000 urls with ~1000 hosts. Hosts have at most 500 urls and there are 23 hosts that have 500 urls. I generally run Fetcher with 100-200 threads and Fetcher2 with 50 threads. > > -vishal. > [snip] -- Doğacan Güney ------------------------------------------------------------------------- This SF.net email is sponsored by DB2 Express Download DB2 Express C - the FREE version of DB2 express and take control of your XML. No limits. Just data. Click to get it now. http://sourceforge.net/powerbar/db2/ _______________________________________________ Nutch-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/nutch-general
