Hi, I would like to know if the *rename* operation (i.e. renaming a directory or a single file) can be consider as an atomic operation in HDFS.
Basically what i am trying to achieve is having one process that continiously add new file into the HDFS and another process that will start every 15 minutes a map/reduce flow on file that were newly added into the HDFS. In other words a process A continuously read a *local directory "A/in"*where new files are moved there continuously and put each file in a *"A/tmp" directory on the HDFS*. When A finish to put one file in "*A/tmp"*it will *move/rename that file into a "B/in" directory*. At the same time a process B will, every 15 minutes, push all the files present in "B/in" to a map/reduce flow. Regards, -- JP
