[
https://issues.apache.org/jira/browse/HDFS-15059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16996429#comment-16996429
]
Yang Yun commented on HDFS-15059:
---------------------------------
Thanks [~kihwal] for the review.
For finalized replica, it looks the most possible missmatch is that the replica
is removed from volumeMap, but the file is still on the disk. the
DirectoryScanner can correct this after some time. and the cache info is only
valid for a few minutes. I'm not sure if there are other cases?
> Cache finalized replica info during datanode shutdown for fast restarting
> -------------------------------------------------------------------------
>
> Key: HDFS-15059
> URL: https://issues.apache.org/jira/browse/HDFS-15059
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Reporter: Yang Yun
> Assignee: Yang Yun
> Priority: Minor
> Attachments: HDFS-15059.patch
>
>
> HDFS-7928 persist the whole volume map on the disk and let the datanode read
> that file and create the volume map during startup. This significantly
> improve the datanode startup time.
> But it not works for any cases. For example, if datanode is killed, it's no
> chance to save the volume map.
> Add a shutdown hook to make sure the volume map is saved during shutdown.
> If the datanode is terminated by kill or other methods, we only reload the
> finalized replca from cache. It's is safe because the DirectoryScanner will
> run soon to correct possible missmatch.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
