[
https://issues.apache.org/jira/browse/HDDS-8919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-8919:
---------------------------------
Labels: pull-request-available (was: )
> Allow a pipeline to be created and then added to pipeline manager in two steps
> ------------------------------------------------------------------------------
>
> Key: HDDS-8919
> URL: https://issues.apache.org/jira/browse/HDDS-8919
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Labels: pull-request-available
>
> As it stands, the method:
> {code}
> PipelineManager.createPipeline(ReplicationConfig replicationConfig,
> List<DatanodeDetails> excludedNodes, List<DatanodeDetails> favoredNodes)
> {code}
> Is used to create a new pipeline and add it to the Pipeline manager. This
> creation and adding is all performed under a single lock The somewhat
> expensive node selection process from the placement policy is performed under
> the lock.
> For Ratis pipelines, this could be important, as pipelines are created and
> limited based on the number of existing pipelines on the nodes.
> EC pipelines do not have such a limitation, and they also need to be created
> much more frequently due to their short lived nature.
> This PR changes the pipeline manager to have a buildPipeline Method, which
> calls the placement policy and creates the new pipeline object without any
> locks.
> Then there is an addPipeline method to add the created pipeline to the
> pipelineManager under the lock.
> This will allow for the WritableECPipelineProvider to build pipelines without
> blocking other threads and then add them to the manager under a lock,
> reducing the amount of work performed under a lock. This change would need to
> be made in another PR.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
