This is the first time I have received this error while crawling.
During a crawl of 100K pages, one of the nodes had a task failed and
cited "Too Many Fetch Failures" as the reason. The job completed
successfully but took about 3 times longer than normal. Here is the
log output
2009-11-19 11:19:56,377 WARN mapred.TaskTracker - Error running child
java.io.IOException: Filesystem closed
at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:
197)
at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:
65)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.close
(DFSClient.java:1575)
at java.io.FilterInputStream.close(FilterInputStream.java:155)
at org.apache.hadoop.util.LineReader.close(LineReader.java:91)
at org.apache.hadoop.mapred.LineRecordReader.close
(LineRecordReader.java:169)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.close
(MapTask.java:198)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:346)
at org.apache.hadoop.mapred.Child.main(Child.java:158)
2009-11-19 11:19:56,380 WARN mapred.TaskRunner - Parent died.
Exiting attempt_200911191100_0001_m_000029_1
2009-11-19 11:20:21,135 WARN mapred.TaskRunner - Parent died.
Exiting attempt_200911191100_0001_r_000004_1
Can Anyone tell me how to resolve this error?
Thanks,
Eric Osgood
---------------------------------------------
Cal Poly - Computer Engineering, Moon Valley Software
---------------------------------------------
eosg...@calpoly.edu, e...@lakemeadonline.com
---------------------------------------------
www.calpoly.edu/~eosgood, www.lakemeadonline.com
