[ 
https://issues.apache.org/jira/browse/HDDS-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18145637#comment-18145637
 ] 

Andrey Yarovoy commented on HDDS-15463:
---------------------------------------

For ratis pipelines we also have problem with balancing. During performance 
testing I noticed that 1 or 2 nodes are getting 2x less number of pipelines 
comparing to other nodes. At least we have outstanding work for 
balancing/rebalancing RATIS pipelines. 

 

As far as I understand we have 2 option for N way replication: send N write to 
all the DN and mark write successful when we archive the write quorum. Or write 
first replica to one of DNs in the pipeline and let the DN to send out N-1 
writes to other DNs in the pipeline and mark the write successful as soon as we 
get the write quorum. 1st one would be less latency and more load to client, 
the 2nd one adds latency and increase the load to DNs. If we take 2nd approach 
we need a design for persistent connection pool between DNs and how to share 
these connection.

> Streaming Write Pipeline without Raft
> -------------------------------------
>
>                 Key: HDDS-15463
>                 URL: https://issues.apache.org/jira/browse/HDDS-15463
>             Project: Apache Ozone
>          Issue Type: New Feature
>          Components: Ozone Client, Ozone Datanode
>            Reporter: Tsz-wo Sze
>            Assignee: Tsz-wo Sze
>            Priority: Major
>              Labels: pull-request-available
>         Attachments: 20260712HDDS-15463 Streaming Write Pipeline without 
> Raft.pdf
>
>
> - V1) Raft Pipeline: Use Raft for both WriteChunk and PutBlock
> - V2)Streaming Write Pipeline (HDDS-4454): Use Ratis streaming (RATIS-979) 
> for WriteChunk and Raft for PutBlock
> - V3) Streaming Write Pipeline without Raft: Use Ratis streaming for both 
> WriteChunk and PutBlock.
> We implement V3 in this JIRA.  Will post a design and create subtasks.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to