[
https://issues.apache.org/jira/browse/HDFS-15421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17142884#comment-17142884
]
Akira Ajisaka commented on HDFS-15421:
--------------------------------------
{quote}I think we need to update genstamp when rolling {{OP_APPEND}}. In
{{OP_TRUNCATE}}, it is the same.
{quote}
This change does not fix the problem for append. When appending a block without
{{CreateFlag.NEW_BLOCK}}, the edit log becomes as follows:
* {{OP_APPEND}}: prepare for append
* {{OP_SET_GENSTAMP_V2}}: update pipeline
That way SNN will tail {{OP_SET_GENSTAMP_V2}} after {{OP_APPEND}}, so apply
impending genstamp in {{OP_APPEND}} does not fix this problem.
I'll attach a patch with some regression tests.
> IBR leak causes standby NN to be stuck in safe mode
> ---------------------------------------------------
>
> Key: HDFS-15421
> URL: https://issues.apache.org/jira/browse/HDFS-15421
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Reporter: Kihwal Lee
> Priority: Blocker
> Labels: release-blocker
> Attachments: HDFS-15421-000.patch
>
>
> After HDFS-14941, update of the global gen stamp is delayed in certain
> situations. This makes the last set of incremental block reports from append
> "from future", which causes it to be simply re-queued to the pending DN
> message queue, rather than processed to complete the block. The last set of
> IBRs will leak and never cleaned until it transitions to active. The size of
> {{pendingDNMessages}} constantly grows until then.
> If a leak happens while in a startup safe mode, the namenode will never be
> able to come out of safe mode on its own.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
