[
https://issues.apache.org/jira/browse/HDFS-16513?focusedWorklogId=756133&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-756133
]
ASF GitHub Bot logged work on HDFS-16513:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Apr/22 02:03
Start Date: 13/Apr/22 02:03
Worklog Time Spent: 10m
Work Description: tomscut commented on PR #4087:
URL: https://github.com/apache/hadoop/pull/4087#issuecomment-1097474203
> Hi @tomscut, sorry for the delay in my response.
>
> I am inclined to agree with @sunchao that the approach laid out in
[HDFS-14378](https://issues.apache.org/jira/browse/HDFS-14378) is a better
long-term solution.
>
> > It might be risky(we can look at here
[HDFS-2737](https://issues.apache.org/jira/browse/HDFS-2737)) by simply
disabling all SNN to trigger active roll edits log.
>
> Can you clarify what from
[HDFS-2737](https://issues.apache.org/jira/browse/HDFS-2737) makes you feel
that it is risky? I skimmed the discussed and didn't notice anything alarming.
You may also want to see [this comment on
HDFS-14378](https://issues.apache.org/jira/browse/HDFS-14378?focusedCommentId=16907765&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16907765)
where this same point was discussed.
>
> That all being said, I think this PR may be a good step in the interim,
since [HDFS-14378](https://issues.apache.org/jira/browse/HDFS-14378) is a more
substantial change. I would appreciate some other opinions, though. cc
@simbadzina @aajisaka @shvachko
Thanks you @xkrogen very much for your comments.
It is mentioned in the description of HDFS-2737:
```
Currently, the edit log tailing process can only read finalized log
segments. So, if the active NN is not rolling its logs periodically, the SBN
will lag a lot. This also causes many datanode messages to be queued up in the
PendingDatanodeMessage structure.
To combat this, the active NN needs to roll its logs periodically – perhaps
based on a time threshold, or perhaps based on a number of transactions. I'm
not sure yet whether it's better to have the NN roll on its own or to have the
SBN ask the active NN to roll its logs.
```
The pendingDatanodeMessage issue mentioned here strikes me as a bit risky.
However, after supporting `SBN READ`, `Journal` supports `read inProgress`. If
we enable `read inProgress`, even if we disable all SNN to roll edits, the
pendingDatanodeMessage problem is not too serious.
I would also appreciate some other opinions.
Issue Time Tracking
-------------------
Worklog Id: (was: 756133)
Time Spent: 2h (was: 1h 50m)
> [SBN read] Observer Namenode should not trigger the edits rolling of active
> Namenode
> ------------------------------------------------------------------------------------
>
> Key: HDFS-16513
> URL: https://issues.apache.org/jira/browse/HDFS-16513
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: tomscut
> Assignee: tomscut
> Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> To avoid frequent edtis rolling, we should disable OBN from triggering the
> edits rolling of active Namenode.
> It is sufficient to retain only the triggering of SNN and the auto rolling of
> ANN.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
