[
https://issues.apache.org/jira/browse/HDDS-13220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17980049#comment-17980049
]
Ryan Blough commented on HDDS-13220:
------------------------------------
I have opted for the smallest possible change of literally changing the
loglevel of this specific message.
In the workaround applied to the cluster where I observed this problem, it was
necessary to change the loglevel of the entire class, which squelched the
logging of the entire task. While nothing important to the users or admins was
lost, it was still heavy-handed relative to the actual cause of the problem.
> Negative usedBytes messages flood Recon logs in large clusters
> --------------------------------------------------------------
>
> Key: HDDS-13220
> URL: https://issues.apache.org/jira/browse/HDDS-13220
> Project: Apache Ozone
> Issue Type: Bug
> Components: Ozone Recon
> Affects Versions: 1.4.1
> Reporter: Ryan Blough
> Assignee: Ryan Blough
> Priority: Minor
>
> In Recon, WARN messages of the type:
> {code:java}
> 2025-05-13 12:41:29,141 WARN
> [ContainerSizeCountTask]-org.apache.hadoop.ozone.recon.tasks.ContainerSizeCountTask:
> Negative usedBytes (-99999999) for container #555555 treating it as 0{code}
>
> Can flood the logs in large clusters. In a cluster with 150+ DataNodes and
> 50+ PB of data, I observed 90k such entries reporting once per minute,
> flooding the Recon logs into uselessness.
> The root cause of these messages is logging on the DataNode, which is
> aggregated into Recon. The appearance of the messages on the DataNode side
> has been resolved in HDDS-11267. The flood of messages on the Recon side can
> be resolved by switching the loglevel of this message from WARN to DEBUG.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
