[
https://issues.apache.org/jira/browse/HDDS-12414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Chuan Yu reassigned HDDS-12414:
------------------------------------
Assignee: Chia-Chuan Yu
> Configure log4j to regulate logging traffic
> -------------------------------------------
>
> Key: HDDS-12414
> URL: https://issues.apache.org/jira/browse/HDDS-12414
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Hemant Kumar
> Assignee: Chia-Chuan Yu
> Priority: Major
>
> Sometimes, a burst of repetitive logs caused by an internal service issue or
> input error can roll over quite fast. We need to configure log4j in a way
> that repetitive logs are rate limited.
> Here are a few suggestions:
> 1.
> [BurstFilter|https://logging.apache.org/log4j/2.x/javadoc/log4j-core/org/apache/logging/log4j/core/filter/BurstFilter.html]:
> Throttles logs when they exceed a specified rate (e.g., 10 logs per minute).
> {code:java}
> <Configuration>
> <Appenders>
> <Console name="Console" target="SYSTEM_OUT">
> <BurstFilter
> level="ERROR"
> rate="10"
> maxBurst="10"
> timeUnit="minute"
> onMatch="NEUTRAL"
> onMismatch="DENY"
> />
> <PatternLayout pattern="%d %-5level: %msg%n"/>
> </Console>
> </Appenders> <Loggers>
> <Root level="error">
> <AppenderRef ref="Console"/>
> </Root>
> </Loggers>
> </Configuration> {code}
> 2. Using DuplicateFilter (Suppress identical Messages)
> Ignores repeated identical logs after a threshold (e.g., 5 duplicates in 5
> minutes).
> {code:java}
> <Configuration>
> <Appenders>
> <Console name="Console" target="SYSTEM_OUT">
> <DuplicateFilter
> interval="5"
> timeUnit="minutes"
> maxRepetitions="5"
> onMatch="NEUTRAL"
> onMismatch="DENY"
> />
> <PatternLayout pattern="%d %-5level: %msg%n"/>
> </Console>
> </Appenders> <Loggers>
> <Root level="error">
> <AppenderRef ref="Console"/>
> </Root>
> </Loggers>
> </Configuration>{code}
> More on filters: [https://logging.apache.org/log4j/2.3.x/manual/filters.html]
> DuplicateFilter gives us more flexibility over BurstFilter because it works
> on message pattern rather than log-level.
> It would be a nice improvement to rate limit the repetitive logs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
