Re: java.io.IOException: Cannot create file (in reduce task)

Thu, 29 Sep 2005 11:33:57 -0700 

Doug Cutting wrote:
Thanks for the detailed report. This is a bug. The problem is that the default is not to permit files to be overwritten. But when a reduce task re-executes (because something failed) it needs to overwrite data. My guess is that the cause of the initial failure might have been the same: that this was not your first attempt to fetch this segment, that you were overwriting the last attempt. Is that right, or did something else first cause the reduce task to fail? 


I think the fix is to change the filesystem code (local and NDFS) so 
that overwriting is permitted by default.  With MapReduce, tasks may 
be re-executed, so overwriting is normal.  Application code should add 
error checking code at the start to check that output files do not 
already exist if we wish to prevent unintentional overwriting.


If there are no objections, I will make this change in the mapred branch.

Doug

Gal Nitzan wrote:

Hello,

I'm testing mapred on one machine only.


Everything worked fine from the start until I got the exception in 
the reduce task:


Diagnostic Text


java.io.IOException: Cannot create file 
/user/root/crawl-20050927142856/segments/20050928075732/crawl_fetch/part-00000/data 
at org.apache.nutch.ipc.Client.call(Client.java:294) at 
org.apache.nutch.ipc.RPC$Invoker.invoke(RPC.java:127) at 
$Proxy1.create(Unknown Source) at 
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.nextBlockOutputStream(NDFSClient.java:574) 
at 
org.apache.nutch.ndfs.NDFSClient$NDFSOutputStream.(NDFSClient.java:549) 
at org.apache.nutch.ndfs.NDFSClient.create(NDFSClient.java:83) at 
org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:76) at 
org.apache.nutch.fs.NDFSFileSystem.create(NDFSFileSystem.java:71) at 
org.apache.nutch.io.SequenceFile$Writer.(SequenceFile.java:94) at 
org.apache.nutch.io.MapFile$Writer.(MapFile.java:108) at 
org.apache.nutch.io.MapFile$Writer.(MapFile.java:76) at 
org.apache.nutch.crawl.FetcherOutputFormat.getRecordWriter(FetcherOutputFormat.java:48) 
at org.apache.nutch.mapred.ReduceTask.run(ReduceTask.java:245) at 
org.apache.nutch.mapred.TaskTracker$Child.main(TaskTracker.java:580)


In the jontracker log:

050928 155253 Server connection on port 8011 from 127.0.0.1: exiting
050928 160814 Server connection on port 8011 from 127.0.0.1: starting
050928 160814 parsing file:/mapred/conf/nutch-default.xml
050928 160814 parsing file:/mapred/conf/mapred-default.xml
050928 160814 parsing /nutch/mapred/local/job_s4isvd.xml
050928 160814 parsing file:/mapred/conf/nutch-site.xml
050928 160814 parsing file:/mapred/conf/nutch-default.xml
050928 160815 parsing file:/mapred/conf/mapred-default.xml
050928 160815 parsing /nutch/mapred/local/job_s4isvd.xml
050928 160815 parsing file:/mapred/conf/nutch-site.xml

050928 160815 Adding task 'task_m_ax7n90' to set for tracker 
'tracker_41883'

050928 160821 Task 'task_m_ax7n90' has finished successfully.

050928 160821 Adding task 'task_m_vl2bge' to set for tracker 
'tracker_41883'

050928 160827 Task 'task_m_vl2bge' has finished successfully.

050928 160827 Adding task 'task_m_i54kht' to set for tracker 
'tracker_41883'

050928 160830 Task 'task_m_i54kht' has finished successfully.

050928 160830 Adding task 'task_m_1eymym' to set for tracker 
'tracker_41883'

050928 160833 Task 'task_m_1eymym' has finished successfully.

050928 160833 Adding task 'task_r_w9azpi' to set for tracker 
'tracker_41883'

050928 160839 Task 'task_r_w9azpi' has finished successfully.
050928 160839 Server connection on port 8011 from 127.0.0.1: exiting
050928 171406 Task 'task_m_klo24y' has finished successfully.

050928 171406 Adding task 'task_r_x48xa3' to set for tracker 
'tracker_41883'

050928 171434 Task 'task_r_x48xa3' has been lost.

050928 171434 Adding task 'task_r_x48xa3' to set for tracker 
'tracker_41883'

050928 171501 Task 'task_r_x48xa3' has been lost.

050928 171501 Adding task 'task_r_x48xa3' to set for tracker 
'tracker_41883'

050928 171520 Task 'task_r_x48xa3' has been lost.

050928 171520 Adding task 'task_r_x48xa3' to set for tracker 
'tracker_41883'

050928 171551 Task 'task_r_x48xa3' has been lost.

050928 171551 Task task_r_x48xa3 has failed 4 times.  Aborting owning 
job job_mtzp7h

050928 171552 Server connection on port 8011 from 127.0.0.1: exiting

In namenode log


050928 171547 Server handler on 8009 call error: java.io.IOException: 
Cannot create file 
/user/root/crawl-20050927142856/segments/20050928075732/crawl_fetch/part-00000/data 

java.io.IOException: Cannot create file 
/user/root/crawl-20050927142856/segments/20050928075732/crawl_fetch/part-00000/data 


       at org.apache.nutch.ndfs.NameNode.create(NameNode.java:98)
       at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)

       at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 


       at java.lang.reflect.Method.invoke(Method.java:324)
       at org.apache.nutch.ipc.RPC$1.call(RPC.java:186)
       at org.apache.nutch.ipc.Server$Handler.run(Server.java:198)

In fetch log

050928 171526  reduce 47%
050928 171538  reduce 50%
050928 171551  reduce 100%
Exception in thread "main" java.io.IOException: Job failed!
       at org.apache.nutch.mapred.JobClient.runJob(JobClient.java:309)
       at org.apache.nutch.crawl.Fetcher.fetch(Fetcher.java:335)
       at org.apache.nutch.crawl.Fetcher.main(Fetcher.java:364)

Any idea, anyone?

Thanks, Gal



.


I believe you are right. When I checked the file I noticed it exists.

However, I run the fetcher only once on that segment.

Gal






















					Reply via email to