[
https://issues.apache.org/jira/browse/HDFS-4372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HDFS-4372:
--------------------------------
Attachment: HDFS-4372.1.patch
I'm attaching a patch that implements a model for tracking NameNode startup
progress information and adds calls throughout the NameNode codebase to update
status and increment various counters. Additionally, this patch exposes the
new metrics via JMX, so you can use jconsole to see that the information is
being updated.
A good starting point for reviewers is the new sub-package:
{{org.apache.hadoop.hdfs.server.namenode.startupprogress}}. This contains a
package-info.java that summarizes the implementation, and all new classes have
JavaDocs that explain in greater detail.
It might not be obvious why I've changed {{EditLogInputStream}} and its
subclasses. Here are a few notes on that:
# I ran into a challenge integrating with QJM. One of the goals was to display
the size of the edits log being loaded. With QJM, the size is unknown until
the stream actually gets fetched via HTTP, where the response contains a
Content-Length header with the stream length. To work around this, the patch
adds a new method, {{EditLogInputStream#isLengthKnown}} to determine whether or
not it is safe to retrieve the length. This returns false when using QJM.
# {{RedundantEditLogInputStream#getName}} returns a comma-separated list of the
names of all underlying wrapped streams. This would clutter the UI and make it
difficult to see which file was actually selected for loading. To resolve
this, I've introduced a new method:
{{EditLogInputStream#getCurrentStreamName}}. The implementation in
{{RedundantEditLogInputStream}} returns only the name of the stream actively
being loaded.
I've been testing with these changes for a few weeks by running existing and
new test suites. I've also run the changes on live clusters.
> Track NameNode startup progress
> -------------------------------
>
> Key: HDFS-4372
> URL: https://issues.apache.org/jira/browse/HDFS-4372
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HDFS-4372.1.patch
>
>
> Track detailed progress information about the steps of NameNode startup to
> enable display to users.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
