Hemant Kumar created HDDS-12414:
-----------------------------------
Summary: Configure log4j to regulate logging traffic
Key: HDDS-12414
URL: https://issues.apache.org/jira/browse/HDDS-12414
Project: Apache Ozone
Issue Type: Improvement
Reporter: Hemant Kumar
Sometimes, a burst of repetitive logs caused by an internal service issue or
input error can roll over quite fast. We need to configure log4j in a way that
repetitive logs are rate limited.
Here are a few suggestions:
1.
[BurstFilter|https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/filter/BurstFilter.html]:
Throttles logs when they exceed a specified rate (e.g., 10 logs per minute).
{code:java}
<Configuration>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<BurstFilter
level="ERROR"
rate="10"
maxBurst="10"
timeUnit="minute"
onMatch="NEUTRAL"
onMismatch="DENY"
/>
<PatternLayout pattern="%d %-5level: %msg%n"/>
</Console>
</Appenders> <Loggers>
<Root level="error">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration> {code}
2. Using DuplicateFilter (Suppress identical Messages)
Ignores repeated identical logs after a threshold (e.g., 5 duplicates in 5
minutes).
{code:java}
<Configuration>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<DuplicateFilter
interval="5"
timeUnit="minutes"
maxRepetitions="5"
onMatch="NEUTRAL"
onMismatch="DENY"
/>
<PatternLayout pattern="%d %-5level: %msg%n"/>
</Console>
</Appenders> <Loggers>
<Root level="error">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>{code}
More on filters: [https://logging.apache.org/log4j/2.3.x/manual/filters.html]
DuplicateFilter gives us more flexibility over BurstFilter because it works on
message pattern rather than log-level.
It would be a nice improvement to rate limit the repetitive logs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
