[
https://issues.apache.org/jira/browse/HDFS-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865509#action_12865509
]
dhruba borthakur commented on HDFS-1137:
----------------------------------------
This is by design so that all transactions are not serialized on the edits log.
As Todd mentions, a client gets a response only after the transaction has been
sync-ed to stable storage. if the sync fails, and this is the only location
specified in fs.name.dir, then the NameNode actually shuts down!
so, technically you are right that a client can see a newly file in the
namespace even before it is written to the transaction log. and if that write
to the transaction log fails, then the client would have seen a file that never
will exist. This window should be small.
An alternative would be to first write to the transaction log and then update
the in-memory data structures. The performance would be impacted unless to go
to a per-inode locking model.
> Name node is using the write-ahead log improperly
> -------------------------------------------------
>
> Key: HDFS-1137
> URL: https://issues.apache.org/jira/browse/HDFS-1137
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Reporter: Benjamin Reed
>
> The Name node is doing the write-ahead log (WAL) (aka edit log) improperly.
> Usually when using WAL, changes are written to the log before they are
> applied to the state. Currently the Namenode does the WAL after applying the
> change. This means that read may see changes before they are durable. A
> client may read information and the server fail before the information is
> written to the WAL, which results in the client reading state that
> disappears. To fix the Namenode should write changes before (aka ahead of)
> applying the change.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
