[jira] [Commented] (HDDS-2116) Create SCMAllocationManager as background thread for pipeline creation
[ https://issues.apache.org/jira/browse/HDDS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931689#comment-16931689 ] Siddharth Wagle commented on HDDS-2116: --- Hi [~timmylicheng], we can certainly punt it if there is no need for it. But, the intention was to encapsulate the instantiation of pipeline allocation strategies in the AllocationManager based on different configurations. The reason it might not be needed is if we are already capturing this behavior somewhere else. It is ok to leave this open, I suspect we would need it. > Create SCMAllocationManager as background thread for pipeline creation > -- > > Key: HDDS-2116 > URL: https://issues.apache.org/jira/browse/HDDS-2116 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Priority: Major > > SCMAllocationManager can be leveraged to get a candidate set of datanodes > based on placement policies. And it should make the pipeline creation process > to be async and multi-thread. This should be done when we encounter with > performance bottleneck in terms of pipeline creation. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-2116) Create SCMAllocationManager as background thread for pipeline creation
[ https://issues.apache.org/jira/browse/HDDS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931329#comment-16931329 ] Li Cheng commented on HDDS-2116: I agree on the multi-thread part. [~swagle] what else function do you think SCMAllocationManager should have? Or is SCMAllocationManager still needed? > Create SCMAllocationManager as background thread for pipeline creation > -- > > Key: HDDS-2116 > URL: https://issues.apache.org/jira/browse/HDDS-2116 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Priority: Major > > SCMAllocationManager can be leveraged to get a candidate set of datanodes > based on placement policies. And it should make the pipeline creation process > to be async and multi-thread. This should be done when we encounter with > performance bottleneck in terms of pipeline creation. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDDS-2116) Create SCMAllocationManager as background thread for pipeline creation
[ https://issues.apache.org/jira/browse/HDDS-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928846#comment-16928846 ] Siddharth Wagle commented on HDDS-2116: --- I do not believe pipeline creation needs to be multi-threaded. The maximum number of pipelines would be created when: 1. Ozone is deployed on a cluster, 2. Adding a rack to the cluster that results in rebalance pipeline operation (This will be the hard part). So, adding the additional complexity of having pipelines created concurrently is unnecessary IMO. > Create SCMAllocationManager as background thread for pipeline creation > -- > > Key: HDDS-2116 > URL: https://issues.apache.org/jira/browse/HDDS-2116 > Project: Hadoop Distributed Data Store > Issue Type: Sub-task >Reporter: Li Cheng >Priority: Major > > SCMAllocationManager can be leveraged to get a candidate set of datanodes > based on placement policies. And it should make the pipeline creation process > to be async and multi-thread. This should be done when we encounter with > performance bottleneck in terms of pipeline creation. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
