This is my log4j2.xml file:
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
<Filters>
<MarkerFilter marker="THROTTLE" onMatch="NEUTRAL" onMismatch="ACCEPT"/>
<BurstFilter level="WARN" rate="1" maxBurst="1"/>
</Filters>
<Appenders>
<RollingFile name="R" fileName="/var/log/project/logs/data/output.log"
bufferedIO="true" immediateFlush="false"
filePattern="/var/log/project/logs/project/output.log.%d{yyyy-MM-dd-HH}">
<ThresholdFilter level="INFO" onMatch="ACCEPT" onMismatch="DENY"/>
<--- I only use info, warn and error in my logging
<PatternLayout pattern="%d{MM/dd/yy HH:mm:ss,SSS} [%t] %p [%c]
(%F:%L) - %m%n"/>
<TimeBasedTriggeringPolicy modulate="true"/>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="org.project.clips" level="INFO">
<AppenderRef ref="clipslog" />
</Logger>
<Root level="INFO">
<AppenderRef ref="R"/>
</Root>
</Loggers>
</Configuration>
The Throttle marker is used for messages that I want to be throttled
because when the particular condition hits, it hits repeatedly and floods
my logs. So I want to throttle it.
Other places where logging occurs, I want them to be left as-is. If they
need throttling, I'll add the marker there too. Currently, I have narrowed
down about 10 places in my code base that require throttling of logs.
I have shown a single marker because currently I am not able to figure out
how to throttle differently based on different markers.
What I ideally want is something like:
<MarkerFilter marker="THROTTLE_LOW"> -> burst filter with rate=10, maxBurst
= 100
<MarkerFilter marker="THROTTLE_MED"> -> burst filter with rate = 5,
maxBurst = 10
<MarkerFilter marker="THROTTLE_HI"> -> burst filter with rate = 1, maxBurst
= 1
So, basically different burst filters associated with different markers.
If this is possible, I would use them as:
public class A
{
private static Logger logger = LoggerFactory.getLogger(A.class);
A()
{
..
catch(Exception ex)
{
logger.warn(throttle_low, "msg1");
}
}
}//class A ends
public class B
{
private static Logger logger = LoggerFactory.getLogger(B.class);
B()
{
..
catch(Exceptions ex)
{
logger.warn(throttle_hi, "msg2");
}
}
}// class B ends
The problem I have with using the same marker, say throttle_hi, everywhere
is that there is no guarantee that msg2 will get printed (i.e. msg1 might
always get printed) because it occurred earlier.
Thanks,
Bharat.
On Tue, Jul 28, 2020 at 4:01 PM Ralph Goers <[email protected]>
wrote:
> Just to make sure we are clear - ALL messages that don’t have a THROTTLE
> marker will be logged - trace, debug, etc from every component., including
> any third party libraries you are using. That would be highly unusual.
>
> It might help if you provided your complete logging configuration and a
> complete sample class. It isn’t clear to me why you will only have one
> Logger in your application. Surely other classes and third party libraries
> you might be using will refer to others.
>
> I also don’t understand what you mean by different throttling by different
> markers. Here you have only shown a single Marker.
>
> If you really only have a single Logger (which I find very hard to
> believe) but somehow have multiple markers then you are going to have a
> hard time with this.
>
> Ralph
>
> > On Jul 28, 2020, at 3:31 PM, bharat naik <[email protected]> wrote:
> >
> > Hi Ralph,
> >
> > Yes, that is the behavior I want ie all logs without a marker need to get
> > printed because they are placed in such a way that they don’t need any
> > throttling. The ones that need throttling will be logged using the
> marker.
> >
> > However, I want different throttling for different markers. They all go
> to
> > the same log ie there is only one logger/appender.
> >
> > So, is it possible to assign burst filters at a marker level?
> >
> > Thanks,
> > Bharat.
> >
> > On Mon, Jul 27, 2020 at 11:03 PM Ralph Goers <[email protected]
> >
> > wrote:
> >
> >> Your filter configuration says “If this is an event with a throttle
> marker
> >> continue checking. If it is not a throttle marker log it, regardless of
> its
> >> logging level.” Is that really what you want?
> >>
> >> If you want to throttle different calls to different loggers
> independently
> >> than just put the burst filter on the logger or the logger’s appender
> >> reference.
> >>
> >> Ralph
> >>
> >>> On Jul 27, 2020, at 10:32 PM, bharat naik <[email protected]>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> I am using log4j2 in my project (2.11.1) and there are several places
> in
> >> my
> >>> code where I need to throttle the logging, in case of exceptions, and
> >> hence
> >>> wanted to use the burst filter.
> >>>
> >>> However, I don’t see a way of using multiple burst filters with
> different
> >>> rate and maxBurst for different logging invocations, all going to the
> >> same
> >>> log file.
> >>>
> >>> For example, what I have so far is:
> >>>
> >>> <Filters>
> >>> <MarkerFilter marker="THROTTLE"onMatch="NEUTRAL"
> >> onMismatch="ACCEPT"
> >>> />
> >>> <BurstFilter level="WARN" rate="1"maxBurst="1”/>
> >>> </Filters>
> >>>
> >>> and invoking it via:
> >>>
> >>> A()
> >>> {
> >>> ..
> >>> logger.warn(throttle, “msg”); <—throttle is a marker for “THROTTLE"
> >>> ..
> >>> }
> >>>
> >>> B()
> >>> {
> >>> ..
> >>> logger.warn(throttle, “msg”);
> >>> ..
> >>> }
> >>>
> >>> However, instead if I could have multiple burst filters for different
> >>> markers that would be ideal since currently I could miss log messages
> >> from
> >>> B() and only log A() since they all go through one burst filter.
> >> Basically,
> >>> I want to ensure at least one log output for every call to
> >> logger.warn(..),
> >>> and throttle each of them independently.
> >>>
> >>> Is that possible?
> >>>
> >>> Thanks,
> >>> Bharat.
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [email protected]
> >> For additional commands, e-mail: [email protected]
> >>
> >>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
