[
https://issues.apache.org/jira/browse/HDDS-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tsz-wo Sze updated HDDS-17568:
------------------------------
Description:
Similar to RatisPipelineProvider, we need a ChainedPipelineProvider for V3
pipelines.
Claude:
CHAINED pipeline provider: a new ChainedPipelineProvider is needed.
- Why. PipelineFactory maps each ReplicationType to one PipelineProvider.
Today it registers STAND_ALONE,
RATIS and EC only, so createPipeline(CHAINED/THREE) has no provider.
RatisPipelineProvider cannot be reused
as-is: it extends PipelineProvider<RatisReplicationConfig> and hardcodes
RatisReplicationConfig.getInstance(factor)
in newPipelineBuilder, so the pipelines it creates would be RATIS.
- What it reuses. A CHAINED pipeline is created like a RATIS/THREE pipeline:
- datanode choice through PipelinePlacementPolicy;
- a leader from LeaderChoosePolicy;
- CreatePipelineCommand(pipelineID, CHAINED, THREE, dns, leader) sent to each
datanode;
- ClosePipelineCommand on close.
Shared code should be pulled out of RatisPipelineProvider (node choice,
leader, create and close command) rather than copied.
The RATIS_DATASTREAM port is required strictly.
RatisPipelineProvider#chooseThreeFactorDatanodes falls back to datanodes
without the stream port; CHAINED does not.
was:
Similar to RatisPipelineProvider, we need a ChainedPipelineProvider for V3
pipelines.
Claude:
CHAINED pipeline provider: a new ChainedPipelineProvider is needed.
- Why. PipelineFactory maps each ReplicationType to one PipelineProvider.
Today it registers STAND_ALONE,
RATIS and EC only, so createPipeline(CHAINED/THREE) has no provider.
RatisPipelineProvider cannot be reused
as-is: it extends PipelineProvider<RatisReplicationConfig> and hardcodes
RatisReplicationConfig.getInstance(factor)
in newPipelineBuilder, so the pipelines it creates would be RATIS.
- What it reuses. A CHAINED pipeline is created like a RATIS/THREE pipeline:
- datanode choice through PipelinePlacementPolicy;
- a leader from LeaderChoosePolicy;
- CreatePipelineCommand(pipelineID, CHAINED, THREE, dns, leader) sent to each
datanode;
- ClosePipelineCommand on close.
Shared code should be pulled out of RatisPipelineProvider (node choice,
leader, create and close command) rather
than copied.
> Add ChainedPipelineProvider for V3 pipelines
> --------------------------------------------
>
> Key: HDDS-17568
> URL: https://issues.apache.org/jira/browse/HDDS-17568
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: SCM
> Reporter: Tsz-wo Sze
> Assignee: Rui Wang
> Priority: Major
>
> Similar to RatisPipelineProvider, we need a ChainedPipelineProvider for V3
> pipelines.
> Claude:
> CHAINED pipeline provider: a new ChainedPipelineProvider is needed.
> - Why. PipelineFactory maps each ReplicationType to one PipelineProvider.
> Today it registers STAND_ALONE,
> RATIS and EC only, so createPipeline(CHAINED/THREE) has no provider.
> RatisPipelineProvider cannot be reused
> as-is: it extends PipelineProvider<RatisReplicationConfig> and hardcodes
> RatisReplicationConfig.getInstance(factor)
> in newPipelineBuilder, so the pipelines it creates would be RATIS.
> - What it reuses. A CHAINED pipeline is created like a RATIS/THREE pipeline:
> - datanode choice through PipelinePlacementPolicy;
> - a leader from LeaderChoosePolicy;
> - CreatePipelineCommand(pipelineID, CHAINED, THREE, dns, leader) sent to
> each datanode;
> - ClosePipelineCommand on close.
> Shared code should be pulled out of RatisPipelineProvider (node choice,
> leader, create and close command) rather than copied.
> The RATIS_DATASTREAM port is required strictly.
> RatisPipelineProvider#chooseThreeFactorDatanodes falls back to datanodes
> without the stream port; CHAINED does not.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
