Am 30.01.2006 um 00:50 schrieb Mike Smith:
I do have the same problem and this problem is killing. I have
tried all
sort of comfiguration and tricks.
I have 3 machines, all three are datanodes and 1 is jobtracker. It
3 tasktracker, 1 jobtracker, 3 datanodes and 1 namenode, right?
successfully fetches 300,000 pages, but when I try to fetch more
than that
by injecting more number of pages at the first cycle it always
crashes at
the end of the fetching reduce step:
060129 142220 reduce 95%
060129 142347 reduce 96%
060129 143401 reduce 100%
Exception in thread "main" java.io.IOException: Job failed!
at org.apache.nutch.mapred.JobClient.runJob(JobClient.java:
308)
at org.apache.nutch.fetcher.Fetcher.fetch(Fetcher.java:347)
at org.apache.nutch.fetcher.Fetcher.main(Fetcher.java:381)
This has happened at one of the tasktrackers:
060129 172145 task_r_ca2dxi 0.8677622% reduce > reduce
060129 172146 task_r_ca2dxi 0.868171% reduce > reduce
060129 173149 Task task_r_ca2dxi timed out. Killing.
060129 173149 Server connection on port 50050 from 164.67.195.26:
exiting
060129 173149 task_r_ca2dxi Child Error
java.io.IOException: Task process exit with nonzero status.
at org.apache.nutch.mapred.TaskRunner.runChild
(TaskRunner.java:139)
at org.apache.nutch.mapred.TaskRunner.run(TaskRunner.java:92)
060129 173153 task_m_bikodi done; removing files.
Strange!
Is that reproducible? You tried it several times?
What happens in case you increase the segment size and use something
like 300 001?
Are you sure all processes still run, no network problem?
-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems? Stop! Download the new AJAX search engine that makes
searching your log files as easy as surfing the web. DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Nutch-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/nutch-general
