File length not reported correctly after application crash
----------------------------------------------------------
Key: HADOOP-5157
URL: https://issues.apache.org/jira/browse/HADOOP-5157
Project: Hadoop Core
Issue Type: Bug
Components: dfs
Affects Versions: 0.19.0
Reporter: Doug Judd
Fix For: 0.20.0
Our application (Hypertable) creates a transaction log in HDFS. This log is
written with the following pattern:
out_stream.write(header, 0, 7);
out_stream.sync()
out_stream.write(data, 0, amount);
out_stream.sync()
[...]
However, if the application crashes and then comes back up again, the following
statement
length = mFilesystem.getFileStatus(new Path(fileName)).getLen();
returns the wrong length. Apparently this is because this method fetches
length information from the NameNode which is stale. Ideally, a call to
getFileStatus() would return the accurate file length by fetching the size of
the last block from the primary datanode.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
