Jay,
What does your job do? Create files directly on HDFS? If so, do you
follow this method?:
http://wiki.apache.org/hadoop/FAQ#Can_I_write_create.2BAC8-write-to_hdfs_files_directly_from_map.2BAC8-reduce_tasks.3F
A local filesystem may not complain if you re-create an existent file.
HDFS' behavior here is different. This simple Python test is what I
mean:
>>> a = open('a', 'w')
>>> a.write('f')
>>> b = open('a', 'w')
>>> b.write('s')
>>> a.close(), b.close()
>>> open('a').read()
's'
Hence it is best to use the FileOutputCommitter framework as detailed
in the mentioned link.
On Mon, Apr 2, 2012 at 7:09 PM, Jay Vyas <jayunit...@gmail.com> wrote:
> Hi guys:
>
> I have a map reduce job that runs normally on local file system from
> eclipse, *but* it fails on HDFS running in psuedo distributed mode.
>
> The exception I see is
>
> *org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException:*
>
>
> Any thoughts on why this might occur in psuedo distributed mode, but not in
> regular file system ?
--
Harsh J
