[
https://issues.apache.org/jira/browse/HDDS-13922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18040311#comment-18040311
]
Siddhant Sangwan commented on HDDS-13922:
-----------------------------------------
[~ayarovoy] Since SCM already excludes a DN if it has less than 5 GB + min(20
GB, 2% of capacity) free, a more flexible way to do what you suggested (and
what we're trying to do here) is make that 5 GB also configurable.
[~rich7420] currently we just check for `dataSizeRequired` which is 5GB in
SCMCommonPlacementPolicy. We can increase the default to something like 50GB
and make that configurable via ozone-site.xml. You'll also need to put a
sensible configuration in the docker-config compose files that are present
under the hadoop-ozone/dist/src/main/compose directory.
> Increase available space requirement during pipeline/container allocation
> -------------------------------------------------------------------------
>
> Key: HDDS-13922
> URL: https://issues.apache.org/jira/browse/HDDS-13922
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: SCM
> Reporter: Siddhant Sangwan
> Assignee: Huang Guan Hao
> Priority: Major
>
> Currently we use "ozone.scm.container.size" (5GB default) in various
> implementations of PipelineProvider and in
> ContainerManagerImpl#getMatchingContainer when checking if a DN has enough
> disk space for a container.
> The problem is that if only 6 GB is remaining, this check will return true
> concurrently for multiple clients. But there's only space for 1 container,
> leading to container creation failures for all but 1 client. So we should
> increase this space requirement to a multiple of container size.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
