[
https://issues.apache.org/jira/browse/HDFS-2737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated HDFS-2737:
---------------------------------
Attachment: HDFS-2737-HDFS-1623.patch
Here's an update to Todd's last patch. Not much has changed. It's rebased on
the tip of HDFS-1623, and has some cleanup, including:
# Changing the config option units from milliseconds to seconds.
# Improved logging in a few places.
# s/interval/period/g in the new config options, to be consistent with the
checkpoint options.
# Roll the edit log before trying to tail the logs, which should keep the
standby a little hotter.
In addition to the unit test included in the patch, I also did some manual
cluster testing, which worked very well.
> HA: Automatically trigger log rolls periodically on the active NN
> -----------------------------------------------------------------
>
> Key: HDFS-2737
> URL: https://issues.apache.org/jira/browse/HDFS-2737
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, name-node
> Affects Versions: HA branch (HDFS-1623)
> Reporter: Todd Lipcon
> Assignee: Aaron T. Myers
> Attachments: HDFS-2737-HDFS-1623.patch, hdfs-2737-prelim.txt
>
>
> Currently, the edit log tailing process can only read finalized log segments.
> So, if the active NN is not rolling its logs periodically, the SBN will lag a
> lot. This also causes many datanode messages to be queued up in the
> PendingDatanodeMessage structure.
> To combat this, the active NN needs to roll its logs periodically -- perhaps
> based on a time threshold, or perhaps based on a number of transactions. I'm
> not sure yet whether it's better to have the NN roll on its own or to have
> the SBN ask the active NN to roll its logs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
