[ 
https://issues.apache.org/jira/browse/HDDS-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145596#comment-18145596
 ] 

Stephen O'Donnell commented on HDDS-15463:
------------------------------------------

In checking on a recent PR about chained replication I found my way to this 
Jira.



Will new "chained pipelines" be long lived?

I am concerned that long lived Ratis pipelines have turned out to be a really 
bad idea in Ozone and they are at the root of many hard to deal with problems. 
We should not build on that IMO.

The EC mode forms a pipeline for a single container and then forms a new one 
for the next one, much more like HDFS. It allows for much more dynamic space 
balancing and load sharing.

I checked the chained design doc, which doesn't really answer the above 
questions.

Also have we considered a model like EC, where the client concurrently writes 
to all 3 DNs directly rather than chaining them? This is how EC works and it 
lets the client coordinate consensus and it feels like that would be faster 
than a forwarding approach as you have added latency as the DNs have to forward 
to each and ACK back all the way to the client.

Is it possible to adapt the EC code to write replicated data to see how it 
performs before implementing this totally new way?

Also will this new approach open the door for 2 or 4 way replicated containers, 
which are not possible with Ratis right now?

> Streaming Write Pipeline without Raft
> -------------------------------------
>
>                 Key: HDDS-15463
>                 URL: https://issues.apache.org/jira/browse/HDDS-15463
>             Project: Apache Ozone
>          Issue Type: New Feature
>          Components: Ozone Client, Ozone Datanode
>            Reporter: Tsz-wo Sze
>            Assignee: Tsz-wo Sze
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: 20260712HDDS-15463 Streaming Write Pipeline without 
> Raft.pdf
>
>
> - V1) Raft Pipeline: Use Raft for both WriteChunk and PutBlock
> - V2)Streaming Write Pipeline (HDDS-4454): Use Ratis streaming (RATIS-979) 
> for WriteChunk and Raft for PutBlock
> - V3) Streaming Write Pipeline without Raft: Use Ratis streaming for both 
> WriteChunk and PutBlock.
> We implement V3 in this JIRA.  Will post a design and create subtasks.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to