[
https://issues.apache.org/jira/browse/HDFS-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13680112#comment-13680112
]
Jing Zhao commented on HDFS-4372:
---------------------------------
Hi Chris,
The patch looks very good to me. Currently the patch needs rebase (mainly
for the changes in FSImageFormat.java). Some thoughts about the patch:
1. FSImageFormat.java
After snapshot is supported, the number of files/directories that is read in
the head of the fsimage (denoted as N_total), may no longer be equal to the
number of times of invocation of loadINode(...). This is because when we
compute the total namespace quota usage, some nodes may be counted multiple
times (in case that rename happened with existence of snapshots). Thus when
using a counter to track the number of inodes loaded from the fsimage, it is
challenging to keep the final value of the counter equal to N_total.
So instead of adding extra complex counting logic to fsimage loading, do you
think we can simply set the counter of inodes to N_total after finishing
loading the inodes from a fsimage?
2. FSNamesystem.java, in loadFSImage(...),
{code}
+ } else {
+ // No need to save, so mark the phase done.
+ StartupProgress prog = NameNode.getStartupProgress();
+ prog.beginPhase(Phase.SAVING_CHECKPOINT);
+ prog.endPhase(Phase.SAVING_CHECKPOINT);
{code}
Here can we present that we have not saved the namespace in the WebUI? (Users
may also get the same information when they see a very small elapsed time for
the phase).
Some minor issues:
3. FSEditLogLoader.java
Why do we need to create a step in both loadFSEdits(...) and
loadEditRecords(...)?
4. StartupProgress.java
4.1 There are some unused imports.
4.2 In lazyInitStep(...), maybe the code
{code}
if (!steps.containsKey(step)) {
steps.putIfAbsent(step, new StepTracking());
}
return steps.get(step);
{code}
can be simplified to
{code}
steps.putIfAbsent(step, new StepTracking());
return steps.get(step);
{code}
or something like
{code}
StepTracking st = new StepTracking();
StepTracking oldValue = steps.putIfAbsent(step, st);
return oldValue != null ? oldValue : st;
{code}
5. StepTracking.java and PhaseTracking.java
Can we use long instead of Long for beginTime/endTime/total? We can initialize
them with -1.
> Track NameNode startup progress
> -------------------------------
>
> Key: HDFS-4372
> URL: https://issues.apache.org/jira/browse/HDFS-4372
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HDFS-4372.1.patch
>
>
> Track detailed progress information about the steps of NameNode startup to
> enable display to users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
