[
https://issues.apache.org/jira/browse/HADOOP-3631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12620442#action_12620442
]
Lohit Vijayarenu commented on HADOOP-3631:
------------------------------------------
Thanks Dhruba. The idea of 3 separate disks looks very good, but if we are
maintaining multiple copies in dfs.name.dir, then number of disks might be a
problem.
we might also have the option of storing image and edits in two separate disks,
but then again there would be contention for writing of edits while it is being
read for transfer. I ran namenode and secondarynamenode to monitor the heap,
there was not much of a problem regarding heap space while this transfer
happened. Koji also helped me point to a case where we saw the client timeout
happened, I could see that sync time and write time was high on the disk
causing namenode to slow down.
> Transfer of image from secondary name node should not interrupt service
> -----------------------------------------------------------------------
>
> Key: HADOOP-3631
> URL: https://issues.apache.org/jira/browse/HADOOP-3631
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.17.0
> Reporter: Robert Chansler
> Priority: Critical
> Fix For: 0.19.0
>
>
> The transfer of the new image prepared by the secondary name node can
> interfere with client services. Clients observe delays in completing RPCs. In
> general, administrative activities should not be observed by the clients. For
> large clusters, administrators are reluctant to run the secondary name node
> leading to excessive edit logs. (Excessive in the sense that if the cluster
> must be restarted, a long time is required to process the log.)
> Maybe the new image does not have to be transfered; it could be fetched when
> needed.
> Maybe the priority of the transfer task can be reduced so that the transfer
> is not observed.
> Maybe a different transfer protocol is more appropriate.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
