[
https://issues.apache.org/jira/browse/HDFS-988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13041934#comment-13041934
]
Todd Lipcon commented on HDFS-988:
----------------------------------
Didn't go through the new tests yet, but here are some comments from a first
pass through FSN:
- checks for if (auditLog.isInfoEnabled()) should probably now be
(auditLog.isInfoEnabled() && isExternalInvocation()) -- otherwise we're doing a
needless directory traversal for fsck
- The following methods currently do logSync() while holding the writeLock,
which is expensive:
-- setPermission
-- setOwner
-- commitBlockSynchronization (in some exit paths)
-- updatePipeline
- These methods should probably just set a local boolean within the
synchronized section, then logSync() in the finally clause if it's flagged
- seems strange that some of the xInternal() methods take the write lock
themselves (eg setReplicationInternal) whereas others assume the caller takes
the write lock (eg createSymlinkInternal). We should be consistent
- for those methods that don't explicitly take the write lock, we should either
add an {{assert hasWriteLock()}} or a comment explaining why the lock is not
necessary (eg internalReleaseLease, reassignLease,
finalizeINodeFileUnderConstruction)
- why doesn't getListing need the read lock?
- comment for endCheckpoint says "not started" but should say "not ended"
- same with updatePipeline
- I noticed that nextGenerationStamp() doesn't logSync() -- that seems
dangerous, since after a restart we might hand out a duplicate genstamp.
> saveNamespace can corrupt edits log, apparently due to race conditions
> ----------------------------------------------------------------------
>
> Key: HDFS-988
> URL: https://issues.apache.org/jira/browse/HDFS-988
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: name-node
> Affects Versions: 0.20-append, 0.21.0, 0.22.0
> Reporter: dhruba borthakur
> Assignee: Eli Collins
> Priority: Blocker
> Fix For: 0.20-append, 0.22.0
>
> Attachments: HDFS-988_fix_synchs.patch, hdfs-988-2.patch,
> hdfs-988-3.patch, hdfs-988-4.patch, hdfs-988.txt, saveNamespace.txt,
> saveNamespace_20-append.patch
>
>
> The adminstrator puts the namenode is safemode and then issues the
> savenamespace command. This can corrupt the edits log. The problem is that
> when the NN enters safemode, there could still be pending logSycs occuring
> from other threads. Now, the saveNamespace command, when executed, would save
> a edits log with partial writes. I have seen this happen on 0.20.
> https://issues.apache.org/jira/browse/HDFS-909?focusedCommentId=12828853&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12828853
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
