Hi,
I ran the mapreduce starting with 10 URL into the sixth cycle where it
fetched 400K pages and everything was fine.
060127 001055 TOTAL urls: 1877326
060127 001055 avg score: 1.099
060127 001055 max score: 1666.305
060127 001055 min score: 1.0
060127 001055 retry 0: 1865721
060127 001055 retry 1: 10887
060127 001055 retry 2: 621
060127 001055 retry 3: 92
060127 001055 retry 4: 4
060127 001055 retry 5: 1
060127 001055 status 1 (DB_unfetched): 1477634
060127 001055 status 2 (DB_fetched): 374736
060127 001055 status 3 (DB_gone): 24956
Then I tried another scenario starting with 80K urls, and the first cycle
was OK, but the second cycle where it supposed to fetch 800K failed after
100% reduce.
I ran it with three machines 1 name node and 2 datanodes.
One of my datanode has the next Exception:
060128 083726 task_r_alfaaq Child Error
java.io.IOException: Task process exit with nonzero status.
at org.apache.nutch.mapred.TaskRunner.runChild(TaskRunner.java:139)
at org.apache.nutch.mapred.TaskRunner.run(TaskRunner.java:92)
Which appeared more than once.
The other Data Node had the next Exception:
060128 142626 Lost connection to JobTracker [server.name/i.i.i.i:50020].
ex=java.lang.reflect.UndeclaredThrowableException Retrying...
Any idea?
Thanks,
Rafit
_________________________________________________________________
Express yourself instantly with MSN Messenger! Download today it's FREE!
http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
