On Tue, Sep 25, 2012 at 9:28 AM, Peter Sheridan <[email protected]> wrote: > We're using Hadoop 1.0.3. We need to pick up a set of large (4+GB) files > when they've finished being written to HDFS by a different process.
The common way to solve this problem is to modify the writing application to write to a temporary filename and then rename the temporary to the target filename when the write is complete. That way, if the file exists without the temporary tag, the reader can be confident the file is complete. -andy
