Stephen O'Donnell created HDDS-5327:
---------------------------------------
Summary: EC: WritableEcContainerProvider should dynamically adjust
the open container groups
Key: HDDS-5327
URL: https://issues.apache.org/jira/browse/HDDS-5327
Project: Apache Ozone
Issue Type: Sub-task
Components: SCM
Reporter: Stephen O'Donnell
After some discussion we concluded that for any given EC policy, a minimum
number of pipelines should be allocated so writes can be directed to different
containers.
The absolute maximum number of pipelines can be calculated as some function of
the cluster nodes and disks, but we are still unsure about how to calculate
that limit.
The number of pipelines should be able to grow from the minimum toward the
maximum, depending on the write load on the cluster, or perhaps the age of the
oldest open container (to prevent too many containers remaining open for too
long).
The goal is to allow a sufficient number of open containers so the writes are
spread across different disks, without having to maintain too many open
containers on the cluster.
If the write load is very high, there should be more open containers than if
the write load is very low.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
