[
https://issues.apache.org/jira/browse/HDDS-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17833016#comment-17833016
]
Attila Doroszlai commented on HDDS-10610:
-----------------------------------------
Thanks [~duongnguyen] for testing this.
bq. when clients write data to a pipeline, it writes to one node (one disk) at
a time. So, at any given time, only 1/5 (for rs-3-2) of the disks are busy.
Is that true even with multiple clients writing concurrently?
bq. Should we have the pipeline number = healthy disks?
Currently {{ozone.scm.ec.pipeline.per.volume.factor}} can be increased to get
more pipelines without code change, but this is a single value and various EC
schemes have different divisor. We can also remove the division and use number
of disks as you suggest.
CC [~sodonnell] [~siddhant]
> Reconsider the default number of EC pipelines
> ---------------------------------------------
>
> Key: HDDS-10610
> URL: https://issues.apache.org/jira/browse/HDDS-10610
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Duong
> Priority: Major
> Attachments: ECWrite-30Pipelines.png, ECWrite-6Pipelines.png
>
>
> Before HDDS-8674, the default number of EC pipelines is hardcoded as 5. This
> hardly utilizes the ability to write
> After HDDS-8674, the default number of EC pipelines is fixed to be a dynamic
> number, calculated by
> healthy_disks/number_of_required_nodes_per_EC_replication_config. For
> example, in a cluster with 90 disks, there'll be 18 pipelines for EC
> rs-3-2-1024k. While this is a bright move, the limited number of pipelines
> still underutilizes the ability of disks in the cluster. This is because
> when clients write data to a pipeline, it writes to one node (one disk) at a
> time. So, at any given time, only 1/5 (for rs-3-2) of the disks are busy.
> I did an EC write (rs-3-2) test on a small cluster with 30 disks on 10 nodes.
> The result with the default number of pipelines (6) is chaotic and it looks
> like the nodes/disks take turns to get the write load.
> !ECWrite-6Pipelines.png|width=846,height=402!
> When forcing the number of pipelines to 30 using
> ozone.scm.ec.pipeline.minimum, the write load is more evenly distributed over
> the nodes. And the total bandwidth increases significantly.
> !ECWrite-30Pipelines.png|width=850,height=400!
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
