Pretty sure it's not atomic. I can read files I write via thrift well before they're done. Rename has always worked for me...
Take care, -stu -----Original Message----- From: Ishaaq Chandy <ish...@gmail.com> Date: Tue, 1 Mar 2011 15:22:24 To: <hdfs-user@hadoop.apache.org> Reply-To: hdfs-user@hadoop.apache.org Subject: atomicity of copyFromLocal Hi all, How "atomic" is the copyFromLocal call? i.e. on process is in the midst of uploading a file to HDFS is it possible for another process to start reading it before the upload is complete? I am currently safeguarding my code from this possibility by uploading it to a temporary directory and the renaming it to its final destination (the assumption being that a rename is "more atomic" than copyFromLocal), but I'd like to avoid doing this in two steps if possible. Regards, Ishaaq
