Hi Stefan,
Stefan Groschupf wrote:
I haved tested it with net.nutch.fetcher.Fetcher in the Nutch 0.4 version and on the latest version.Massimo,
I have checked by ps -ef and see that 100 and up threads is started. The new version is so slow? :(
Before, Nutch-0.4, the Fetcher with 100 threads get about 60-70 pages a second with 6Mbit connection.
In the old version (0.4), same fetchlist size and threads number, the Fetcher get about 60-60 pages a second.
With new version the fetcher get about 2-3 pages a second.
does you test the Fetcher.java before as well or do you compair the old RequestScheduler.java with the Fetcher.java?
Can you provide more information where you guess the problem could be?
No because also with the verbose log al sems to be regular.
Massimo
I will try to take a look and make some profiling work next week.
By the way I dont think the problem is the 'plugin- system'. Anyway we will see and i will try to improve it.
Stefan
------------------------------------------------------- This SF.Net email is sponsored by: GNOME Foundation Hackers Unite! GUADEC: The world's #1 Open Source Desktop Event. GNOME Users and Developers European Conference, 28-30th June in Norway http://2004/guadec.org _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
------------------------------------------------------- This SF.Net email is sponsored by: GNOME Foundation Hackers Unite! GUADEC: The world's #1 Open Source Desktop Event. GNOME Users and Developers European Conference, 28-30th June in Norway http://2004/guadec.org _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
