[
https://issues.apache.org/jira/browse/HDDS-981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16750068#comment-16750068
]
Shashikant Banerjee commented on HDDS-981:
------------------------------------------
Thanks [~ljain] for working on this. The patch overall looks good to me, Some
comments:
1.ContainerStateManager.java : 254 : Unintended changes. Please revert.
2. The pipeline creation is triggered 5000 ms once node is added or node state
is updated. Instead of using a hard coded value, let's make it a configure
value or make it a multiple of heartBeatInterval/StaleNodeInterval.
3. Currently, the block allocation happens over a random pipeline chosen over a
selected set of available pipelines . Instead, a better approach would be to
choose a pipeline based on policy or load on a pipeline. Let's add a TODO in
the code so that can be addressed later.
4.There are two methods with same name ContainerManager#getMatchingContainer
where one is not used anywhere in the code. Let's clean this up.
5. On a given pipeline selected, we should ensure multiple clients receive
blocks from different containers from successive allocateBlock calls. This can
be addressed in a follow up jira.
> Block allocation should involve pipeline selection and then container
> selection
> -------------------------------------------------------------------------------
>
> Key: HDDS-981
> URL: https://issues.apache.org/jira/browse/HDDS-981
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: SCM
> Reporter: Lokesh Jain
> Assignee: Lokesh Jain
> Priority: Major
> Attachments: HDDS-981.001.patch, HDDS-981.002.patch,
> HDDS-981.003.patch
>
>
> Currently SCM maintains a list of preallocated containers and allocates
> blocks from these containers. This approach does not work well with dynamics
> of the cluster where new nodes are being added and pipelines are destroyed.
> New containers are not created until all the preallocated containers are
> exhausted.
> The Jira aims to establish a criteria in block allocation where first a
> pipeline is selected amongst the available pipelines and then a container is
> selected in that pipeline. In order to handle the dynamics of the cluster a
> fixed interval pipeline creator job can be launched which creates pipelines
> in the system.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
