Thanks a bunch Arun. I am ignoring these files for now as you suggest. Thanks for pointing to the JIRA.
Ashish -----Original Message----- From: Arun C Murthy [mailto:[EMAIL PROTECTED] Sent: Saturday, February 09, 2008 10:00 PM To: [email protected] Subject: Re: Speculative execution and output directory On Feb 9, 2008, at 3:52 PM, Ashish Thusoo wrote: > Hi Hadoop users, > > > > We have intermittently hit issues with speculative execution and > hadoop > streaming where we see a directory of the form > > _task_200...._...._m_..._. > It's an unfortunate side-effect of the current implementation of speculative execution, we have been discussing ways to fix it here: http://issues.apache.org/jira/browse/HADOOP-2391 In the meanwhile, it is completely safe to ignore directories starting with with underscore ("_") and use the rest... Arun > > > formed in the output directory. Has anyone out there hit similar > issues > or knows what might be happening here? We did scan the JIRA but could > not find any related issues reported there. > > > > Thanks, > > Ashish > > >
