I was under the impression that a file doesn't 'exist' in hdfs global namespace until it has been closed, and that appends are not supported (yet?). So this situation, at current time, should not be possible, correct?
I remember seeing a jira issue where someone wanted to change it so files show up before they are closed, but I didn't catch what the resolution was for that issue. Am I missing something here? -jorgenj -----Original Message----- From: Owen O'Malley [mailto:[EMAIL PROTECTED] Sent: Monday, November 05, 2007 9:24 AM To: [email protected] Subject: Re: rename dir while writing On Nov 5, 2007, at 2:05 AM, Torsten Curdt wrote: > Is there anywhere documented the expected behavior of concurrent > changes in the filesystem? As an example: > > Hdfs client C1 is slowly writing to "/path/a/file". Now another > hdfs client C2 renames "/path/a" to /path/b". > > What happens? Will C1 continue to write but the file will be in "/ > path/b" when it closes the file? Will C2 get an exception because > C1 is writing to "/path/a" so it cannot be renamed? And so on... > > I will probably run off an do some test myself ...but is this > documented somewhere? I couldn't find it. The short answer is that currently bad things happen. All of the hdfs protocols call files by name and so changing the name of the file while someone is writing to it will break the writer. In my opinion, it would be better if dfs.ClientProtocol.create returned a lease handle, which was used for the follow up addBlock, complete, and renewLease. -- Owen
