[
https://issues.apache.org/jira/browse/HDDS-4656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17264509#comment-17264509
]
Baolong Mao commented on HDDS-4656:
-----------------------------------
[~elek] [~ljain] Thank you for your reply and comments.
> it seems to be an improved placement policy, not a separated service, right?
Yeah, it can based on pipeline choose policy framework, it is another important
thing, but not the goal of this ticket.
> It's a good question. There were some thoughts earlier about improving the
> ReplicationManager (closed container replication) to handle smarter
> strategies and initiate container moves (re-replication + delete) just for
> better placement. But Nanda may have better memories. (Not the only way to do
> it, just sharing some earlier thoughts)
Thank for share this, it brings a new choice for us. I think we have two way to
move container replica and keep rack policy.
1. As your approach, improving the ReplicationManager and let
ReplicationManager handle the container replica placement during balance
container.
2. Copy or get a datanode topology from SCM, so balancer can choose the new
target as destination without break the rack policy.
Compare the two approach, 2 can make SCM simple and lots of the balance logic
code and load are only in the balancer and datanodes, 1 make SCM become a core
centralized service, during balance, maybe SCM can suffer low performance for
OM.
Now, I added a design doc draft link to the description of this ticket, it's
not finished, but I'm glad to see your comments and open to accept your
suggestion.
> Add a container balancer tool or service for SCM
> ------------------------------------------------
>
> Key: HDDS-4656
> URL: https://issues.apache.org/jira/browse/HDDS-4656
> Project: Hadoop Distributed Data Store
> Issue Type: New Feature
> Components: SCM, Tools
> Affects Versions: 1.1.0
> Reporter: Baolong Mao
> Assignee: Baolong Mao
> Priority: Major
>
> When an existing Ozone cluster is nearly full, we have to add more datanodes
> into the Ozone cluster, but there are two issue we must face.
> - When new allocate container request coming, SCM should better to choose the
> datanodes in low usage, if not, the performance will getting pool.
> - For read request, the existing datanodes stored lots of blocks, so they are
> responsible for serving the read request and supply the data stream service,
> meanwhile, the new coming datanodes can help nothing.
> If we have a balancer tool just like hdfs balancer, we can move the block or
> container from some high usage datanodes to low, I think this is one of
> necessary tools for Ozone.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
