[
https://issues.apache.org/jira/browse/HDFS-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17143296#comment-17143296
]
Konstantin Shvachko commented on HDFS-15421:
--------------------------------------------
Good catch [~kihwal], thanks for debugging this. [~aajisaka] thanks for the
patch.
Clearly HDFS-14941 missed some append and truncate cases, which update blocks
with new genStamp while tailing.
Took a look at v02 patch. It seems you caught correctly all other cases of
block updates during tailing. Would be good if [~vagarychen] could take a look
as well.
One suggestion for tests is to move all test cases into {{TestAddBlockTailing}}
if possible, potentially renaming it to something like
{{TestUpdateBlockTailing}}. The two new tests have a lot of code similarities
with {{TestAddBlockTailing}. And if merged will avoid extra MiniCluster
startups, making tests run faster.
> IBR leak causes standby NN to be stuck in safe mode
> ---------------------------------------------------
>
> Key: HDFS-15421
> URL: https://issues.apache.org/jira/browse/HDFS-15421
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Kihwal Lee
> Assignee: Akira Ajisaka
> Priority: Blocker
> Labels: release-blocker
> Attachments: HDFS-15421-000.patch, HDFS-15421-001.patch,
> HDFS-15421.002.patch
>
>
> After HDFS-14941, update of the global gen stamp is delayed in certain
> situations. This makes the last set of incremental block reports from append
> "from future", which causes it to be simply re-queued to the pending DN
> message queue, rather than processed to complete the block. The last set of
> IBRs will leak and never cleaned until it transitions to active. The size of
> {{pendingDNMessages}} constantly grows until then.
> If a leak happens while in a startup safe mode, the namenode will never be
> able to come out of safe mode on its own.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
