[
https://issues.apache.org/jira/browse/HDFS-3300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13257801#comment-13257801
]
Brandon Li commented on HDFS-3300:
----------------------------------
{quote}What is the plan of action when there are errors in streaming? NN will
shutdown with error?{quote}
If there are errors in streamed data, the NN can reload the namespace with a
different active journal service. If NN gets errors with all active journal
services, it can shutdown.
{quote}Will the streamed edits also be dumped to disk in parallel? So that next
time around they are available locally instead of having to stream them
again.{quote}
Caching a copy of the edit segment can be helpful when NN doesn't have new
checkpoint before its next reboot. Otherwise the cached edit segments will not
be used for the next NN startup.
> stream the edit segments to NameNode when NameNode starts up
> ------------------------------------------------------------
>
> Key: HDFS-3300
> URL: https://issues.apache.org/jira/browse/HDFS-3300
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Reporter: Brandon Li
>
> Edit logs are saved on Journal daemon. When NameNode starts, it loads the
> latest image file and then streams the edit logs from an active Journal
> daemon.
> Currently we are using http to transfer edit files between two journal
> daemons/nodes or between a journal daemon and a NameNode. To get edit file
> from Journal daemon, the NameNode has to download it first and then read it
> from the disk.
> To avoid the slow start-up time, NameNode should be enhance to read the http
> data stream and update its in memory name space instead of saving the
> streamed data on disk first.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
