[ 
https://issues.apache.org/jira/browse/HDDS-17845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tsz-wo Sze updated HDDS-17845:
------------------------------
    Description: 
(Assisted by Claude)

When a datanode restart, we need some way to tell the ReplicationType of an 
open container in order to recreate the ContainerStateMachine. Two approaches 
below are considered, SCM reply vs Per-pipeline RocksDB marker:
|| ||SCM reply||Per-pipeline RocksDB marker||
|On-disk change|none|new column family in the witnessed DB|
|Protocol change|PipelineReport field and handling|none|
|Unknown window after restart|~registration + 1 heartbeat, behaving 
permissively|none: known before log replay|
|Source of truth|SCM, so it can't go stale|the datanode, so a crash between 
steps can leave a stray marker|
|State machine type|mutable, set once|final, and the subclass split stays 
possible|

Although SCM reply does not requires on-disk change, there are may edge cases 
listed below. Therefore, we will implement Per-pipeline RocksDB marker
 - SCM doesn't know the pipeline: the existing ClosePipelineCommand path 
removes the group, so the type never matters.
 - Datanode was stale or dead: SCM has already closed the pipeline, which is 
the self-healing path from before.
 - Old SCM, new datanode: the old SCM ignores the flag, so the type stays 
unknown and the datanode keeps allowing both kinds. Safe, just without the 
checks.
 - New SCM, old datanode: an old datanode never sets the flag, so nothing 
changes for it.
 - SCM unreachable or not leader: the type stays unknown until the leader 
answers.


Regarding to HDDSLayoutFeature and HDDSVersion, see Note 1 & 2 in the first two 
comments.

  was:
(Assisted by Claude)

When a datanode restart, we need some way to tell the ReplicationType of an 
open container in order to recreate the ContainerStateMachine. Two approaches 
below are considered, SCM reply vs Per-pipeline RocksDB marker:
|| ||SCM reply||Per-pipeline RocksDB marker||
|On-disk change|none|new column family in the witnessed DB|
|Protocol change|PipelineReport field and handling|none|
|Unknown window after restart|~registration + 1 heartbeat, behaving 
permissively|none: known before log replay|
|Source of truth|SCM, so it can't go stale|the datanode, so a crash between 
steps can leave a stray marker|
|State machine type|mutable, set once|final, and the subclass split stays 
possible|

Although SCM reply does not requires on-disk change, there are may edge cases 
listed below. Therefore, we will implement Per-pipeline RocksDB marker
 - SCM doesn't know the pipeline: the existing ClosePipelineCommand path 
removes the group, so the type never matters.
 - Datanode was stale or dead: SCM has already closed the pipeline, which is 
the self-healing path from before.
 - Old SCM, new datanode: the old SCM ignores the flag, so the type stays 
unknown and the datanode keeps allowing both kinds. Safe, just without the 
checks.
 - New SCM, old datanode: an old datanode never sets the flag, so nothing 
changes for it.
 - SCM unreachable or not leader: the type stays unknown until the leader 
answers.


> Restore ReplicationType in ContainerStateMachine when a datanode restart
> ------------------------------------------------------------------------
>
>                 Key: HDDS-17845
>                 URL: https://issues.apache.org/jira/browse/HDDS-17845
>             Project: Apache Ozone
>          Issue Type: Sub-task
>          Components: Ozone Datanode
>            Reporter: Tsz-wo Sze
>            Assignee: Tsz-wo Sze
>            Priority: Major
>              Labels: pull-request-available
>
> (Assisted by Claude)
> When a datanode restart, we need some way to tell the ReplicationType of an 
> open container in order to recreate the ContainerStateMachine. Two approaches 
> below are considered, SCM reply vs Per-pipeline RocksDB marker:
> || ||SCM reply||Per-pipeline RocksDB marker||
> |On-disk change|none|new column family in the witnessed DB|
> |Protocol change|PipelineReport field and handling|none|
> |Unknown window after restart|~registration + 1 heartbeat, behaving 
> permissively|none: known before log replay|
> |Source of truth|SCM, so it can't go stale|the datanode, so a crash between 
> steps can leave a stray marker|
> |State machine type|mutable, set once|final, and the subclass split stays 
> possible|
> Although SCM reply does not requires on-disk change, there are may edge cases 
> listed below. Therefore, we will implement Per-pipeline RocksDB marker
>  - SCM doesn't know the pipeline: the existing ClosePipelineCommand path 
> removes the group, so the type never matters.
>  - Datanode was stale or dead: SCM has already closed the pipeline, which is 
> the self-healing path from before.
>  - Old SCM, new datanode: the old SCM ignores the flag, so the type stays 
> unknown and the datanode keeps allowing both kinds. Safe, just without the 
> checks.
>  - New SCM, old datanode: an old datanode never sets the flag, so nothing 
> changes for it.
>  - SCM unreachable or not leader: the type stays unknown until the leader 
> answers.
> Regarding to HDDSLayoutFeature and HDDSVersion, see Note 1 & 2 in the first 
> two comments.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to