[ http://issues.apache.org/jira/browse/HADOOP-334?page=comments#action_12447542 ] Sameer Paranjpye commented on HADOOP-334: -----------------------------------------
Copy on write helps, but the global lock needs to be acquired at the end of the checkpointing process nevertheless. This still has the effect of locking clients out of the namespace while the entire namespace is traversed and the clone pointers are reset. Instead of copy on write, how about changing the locking model so that for any change: 1. Acquire read locks on all structures between the root and the change, acquire a write lock on the changed node. 2. To checkpoint, traverse the namespace acquiring read locks on the path between the root and the node being checkpointed. Serialize each node to a new image file on disk. This way we never lock down the whole tree, for any operation. At the start of the checkpointing process, a new edits file is created. Edits that occur while the checkpoint is in progress are sent to the new file. This implies that there will be some overlap between the checkpointed image and the edits file, but this is ok. We require that the union of the image and the edits give us the current state of the namespace but the two do not have to be disjoint. > Redesign the dfs namespace datastructures to be copy on write > ------------------------------------------------------------- > > Key: HADOOP-334 > URL: http://issues.apache.org/jira/browse/HADOOP-334 > Project: Hadoop > Issue Type: Improvement > Components: dfs > Affects Versions: 0.4.0 > Reporter: Owen O'Malley > Assigned To: Konstantin Shvachko > > The namespace datastructures should be copy on write so that the namespace > does not need to be completely locked down from user changes while the > checkpoint is being made. -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira
