[
https://issues.apache.org/jira/browse/HDDS-5413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Attila Doroszlai updated HDDS-5413:
-----------------------------------
Labels: (was: pull-request-available)
> Limit num of containers to process per round for ReplicationManager.
> --------------------------------------------------------------------
>
> Key: HDDS-5413
> URL: https://issues.apache.org/jira/browse/HDDS-5413
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Mark Gui
> Assignee: Mark Gui
> Priority: Major
>
> For now, ReplicationManager process all containers at once, this will
> potentially bring healthy load to datanodes if there are a lot of containers
> to be replicated/deleted/closed.
> So it is nice to have a bound for each round, HDFS has similar settings, and
> this issue tries to implement sth like
> 'dfs.block.misreplication.processing.limit: 10000' in HDFS.
>
> This is just a limit on the number of containers to be processed, note that
> ReplicationManager count each container as processed no mater it is under
> replicated or over replicated or good. And all the cmds are queued and will
> be sent with heartbeats.
> So this limit does not directly limit the data to be replicated, just to have
> a basic throttling for ReplicationManager in-memory processing, so 10000
> should be good for ozone as well.
> And we should add more throttling limits in the future.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
