[
https://issues.apache.org/jira/browse/HDDS-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gargi Jaiswal updated HDDS-14926:
---------------------------------
Status: Patch Available (was: In Progress)
> Allow QUASI_CLOSED containers in DiskBalancer with improved debug logging for
> containers
> ----------------------------------------------------------------------------------------
>
> Key: HDDS-14926
> URL: https://issues.apache.org/jira/browse/HDDS-14926
> Project: Apache Ozone
> Issue Type: Sub-task
> Reporter: Gargi Jaiswal
> Assignee: Gargi Jaiswal
> Priority: Major
> Labels: pull-request-available
>
> Currently DiskBalancer only supports CLOSED containers to be moved but if
> user wants to also move quasi closed containers then we should support that
> as well.
> *1st Improvement:*
> Below scenario can happen in real, so Disk balancer can only attempt those
> {*}10 CLOSED containers{*}. Even if it moves all 10 successfully, disk A
> might drop only from 90% → 85% — nowhere near balance. The QUASI_CLOSED
> containers are the real bulk occupying the space and disk balancer is
> completely blind to them.
> {code:java}
> Disk A: 90% utilized
> ├── 600 containers: QUASI_CLOSED
> └── 10 containers: CLOSED
> Disk B: 10% utilized
> Disk C: 11% utilized{code}
> Added a config `hdds.datanode.disk.balancer.include.non.standard.containers`
> **default=false**. If true, balancer include non-standard states, i.e,
> QUASI_CLOSED. So both CLOSED and QUASI_CLOSED state containers are eligible
> for move. If false (default), balancer only moves CLOSED containers.
>
> *2nd Improvement:*
> We need to add debug logs for *chooseContainer* method as user might not
> understand why if they have over and under utilised volume still container is
> not moved. This parts needs more clarification. Because I see in escalation
> it helped a lot with balancer debug logs for container not choose to identify
> what state the contianer or volume was in.
> I suggest adding these logs for container not choose :
>
> {code:java}
> // This debug will help us to understand what state container was because of
> which it not moved
> LOG.debug("Skipping container {} from volume {}: state is {} (only CLOSED is
> eligible)", containerId, src.getStorageID(), containerData.getState());
> // It can happen a container size exceeds the usable space on that dn
> LOG.debug("Skipping container {} ({}B) from volume {}: exceeds destination {}
> " + "usable space {}B", containerId, containerSize, src.getStorageID(),
> dst.getStorageID(), usableSpace);
> // this shows would push destination utilization over upper threshold
> LOG.debug("Skipping container {} ({}B) from volume {}: moving to {} would " +
> "result in utilization {} exceeding upper threshold {}", containerId,
> containerSize, src.getStorageID(), dst.getStorageID(), newUtilization,
> upperThreshold);{code}
> For message like “{color:#de350b}{{No suitable container found for
> destination {}, trying next volume."}}{color}. we can improve it to
> {color:#00875a}“{{{}No suitable CLOSED state and QUASI_CLOSED state container
> found for destination {}, trying next volume.{}}}{color} something like this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
