[
https://issues.apache.org/jira/browse/HDDS-17334?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18130628#comment-18130628
]
Wei-Chiu Chuang commented on HDDS-17334:
----------------------------------------
This one looks like a real issue. Pretty sure i've seen this a few times in the
past.
> Open a Ratis pipeline only after its leader is ready
> ----------------------------------------------------
>
> Key: HDDS-17334
> URL: https://issues.apache.org/jira/browse/HDDS-17334
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Datanode, SCM
> Reporter: Siyao Meng
> Assignee: Andrey Yarovoy
> Priority: Major
>
> SCM can mark a Ratis pipeline OPEN before its Ratis leader is ready. A write
> in that window gets {{LeaderNotReadyException}} for WriteChunk while the
> leader, once ready, still accepts the following PutBlock. Applying PutBlock
> without its chunk fails with "Chunk file not found", the container is marked
> UNHEALTHY and the pipeline is closed. See HDDS-17213 for a test hitting this.
> h3. Current behavior
> * {{PipelineReportHandler#setPipelineLeaderId}} sets the leader for RATIS/ONE
> from any pipeline report ("ONE replica pipeline doesn't have leader flag",
> from HDDS-2034). {{Pipeline#isHealthy}} is then true and the pipeline is
> opened on the first report after the group is added, before any election. In
> the HDDS-17213 run: opened 05:39:58.528, leader elected 05:39:59.584, ready
> 05:39:59.615.
> * For RATIS/THREE, SCM waits for a report with {{isLeader}}. The datanode
> sets that flag in {{XceiverServerRatis#handleLeaderChangedNotification}},
> i.e. on election, not when the leader is ready, so a short window remains.
> h3. Proposal
> * Datanode: set {{isLeader}} in the pipeline report only after
> {{StateMachine.LeaderEventApi#notifyLeaderReady}}, and send a pipeline report
> then.
> * SCM: drop the RATIS/ONE special case in
> {{PipelineReportHandler#setPipelineLeaderId}}. Datanodes already report
> {{isLeader}} for single node groups.
> This covers newly created pipelines only. A re-election on an OPEN pipeline
> still has the same window.
> Once done, the leader ready wait added to {{TestFinalizeBlock}} in HDDS-17213
> can be removed.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
