Re: Race condition stopping context
Hello,
I've been taking a look at LoggerContext#stop() and for what I see there's
code to prevent disruptor to take new messages but there's nothing making
sure that the ring buffer is empty before actually closing up.
I did a simple test with an Appender wrapper like this:
final class StopConditionSafeAppenderWrapper extends BaseAppenderWrapper
{
private final LoggerContext loggerContext;
StopConditionSafeAppenderWrapper(Appender delegate, LoggerContext
loggerContext)
{
super(delegate);
this.loggerContext = loggerContext;
}
@Override
public void append(LogEvent event)
{
if (!loggerContext.isStarted()) {
return;
}
super.append(event);
}
}
Using this appender wrapper I still get the same error, because at the time
that append was closed the context was started but by the time disruptor
processed it it was already stopped (this I could verify with a debugger).
Should I open a bug? Any workaround you could suggest?
Thanks!
Re: Race condition stopping context
Hello Remko, I'm using 2.1. Any pointers ? On Fri, Dec 19, 2014 at 11:20 AM, Remko Popma wrote: > > We used to have this problem in older versions but these issues have been > addressed in 2.1 as far as I know. Which version are you using? > > Sent from my iPhone > > > On 2014/12/19, at 22:01, Mariano Gonzalez > wrote: > > > > Hello, > > > > I have an application in which I'm using all async loggers. When I stop > the > > LoggerContext, there're still some events waiting in disruptor's buffer > and > > when it tries to execute them the context is already closed and thus > those > > events are lost. > > > > Is there some kind of hook, callback or whatever I can use so that when > the > > context is stopped: > > > > > > - the loggers stop accepting new messages > > - the context waits for disruptor's buffer to be empty before actually > > stopping > > > > > > Thanks > > - > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > For additional commands, e-mail: log4j-user-h...@logging.apache.org > >
Race condition stopping context
Hello, I have an application in which I'm using all async loggers. When I stop the LoggerContext, there're still some events waiting in disruptor's buffer and when it tries to execute them the context is already closed and thus those events are lost. Is there some kind of hook, callback or whatever I can use so that when the context is stopped: - the loggers stop accepting new messages - the context waits for disruptor's buffer to be empty before actually stopping Thanks
Re: ArrayIndexOutOfBoundsException with RollingFileApender
Hello Gary,
What I meant by mandatory is that the only way of getting the fix was
upgrading to 2.1. I have code freeze on Thursday so I was trying to avoid a
last minute dependency change, but it seems like that's just the way it is.
Of course I can always fork the project but that seems overkill just for
this.
Thank you for your help!
On Tue, Dec 2, 2014 at 3:29 PM, Gary Gregory wrote:
> Good, I am glad the current version work for you.
>
> Why would it be mandatory to upgrade? Only you can decide that ;-) The
> released versions will live in the Maven repository "forever".
>
> That part of the code was probably changed as part of fixing LOG4J2-832.
>
> Gary
>
> On Tue, Dec 2, 2014 at 1:16 PM, Mariano Gonzalez <
> mariano.l.gonza...@gmail.com> wrote:
>
> > Hello Gary,
> >
> > Thank you for your response. 2.1 does fix the issue, is it mandatory to
> > upgrade? was it a logic error or just some setting I could force into my
> > current version?
> >
> > Thanks
> >
> > On Tue, Dec 2, 2014 at 3:07 PM, Gary Gregory
> > wrote:
> >
> > > FYI: the stack trace does not match with our current git master level
> > > sources which is not surprising since we are evolving and fixing
> things.
> > >
> > > If you had a little program that could reproduce the issue that would
> be
> > > great.
> > >
> > > You can also use a 2.2-SNAPSHOT from the Apache snapshot repo (or build
> > > from sources too).
> > >
> > > Gary
> > >
> > > On Tue, Dec 2, 2014 at 12:57 PM, Gary Gregory
> > > wrote:
> > >
> > > > Can you try version 2.1?
> > > >
> > > > Gary
> > > >
> > > > On Tue, Dec 2, 2014 at 12:46 PM, Mariano Gonzalez <
> > > > mariano.l.gonza...@gmail.com> wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> I'm progamatically creating an appender with the following code:
> > > >>
> > > >> private static final String PATTERN_LAYOUT = "%-5p %d [%t] %c:
> %m%n";
> > > >>
> > > >> private RollingFileAppender createRollingFileAppender(String
> > > logFilePath,
> > > >> String filePattern, String appenderName, Configuration
> configuration)
> > > >> {
> > > >> TriggeringPolicy triggeringPolicy =
> > > >> TimeBasedTriggeringPolicy.createPolicy("1", "true");
> > > >> RolloverStrategy rolloverStrategy =
> > > >> DefaultRolloverStrategy.createStrategy("30", "1", null,
> > > >> String.valueOf(Deflater.NO_COMPRESSION), configuration);
> > > >>
> > > >> return RollingFileAppender.createAppender(logFilePath,
> > > >> logFilePath + filePattern,
> > > >> "true",
> > > >> appenderName,
> > > >> "true",
> > > >> null, null,
> > > >> triggeringPolicy,
> > > >> rolloverStrategy,
> > > >> createLayout(configuration),
> > > >> null, null, null, null,
> > > >> configuration);
> > > >> }
> > > >>
> > > >> private Layout createLayout(Configuration
> > > >> configuration)
> > > >> {
> > > >> return PatternLayout.createLayout(PATTERN_LAYOUT, configuration,
> > > null,
> > > >> null, true, false, null, null);
> > > >> }
> > > >>
> > > >> Sometimes, not always, when an exception is thrown I end up seeing
> the
> > > >> following error:
> > > >>
> > > >> 2014-12-02 14:36:52,092 ERROR An exception occurred processing
> > Appender
> > > >> defaultFileAppender java.lang.ArrayIndexOutOfBoundsException: 53
> > > >>
> > > >> at
> > > >>
> > > >>
> > >
> >
> org.apache.logging.log4j.core.impl.ThrowableProxy.formatElements(ThrowableProxy.java:227)
> > > >>
> > > >> at
> > > >>
> > > >>
> > >
> >
> org.apache.logging.log4j.core.impl.ThrowableProxy.getExtendedStackTraceAsString(ThrowableProxy.java:401)
> > > >>
> > > >> at
> > > >>
> > > >>
> > >
> >
> org.apache.loggin
Re: ArrayIndexOutOfBoundsException with RollingFileApender
Hello Gary,
Thank you for your response. 2.1 does fix the issue, is it mandatory to
upgrade? was it a logic error or just some setting I could force into my
current version?
Thanks
On Tue, Dec 2, 2014 at 3:07 PM, Gary Gregory wrote:
> FYI: the stack trace does not match with our current git master level
> sources which is not surprising since we are evolving and fixing things.
>
> If you had a little program that could reproduce the issue that would be
> great.
>
> You can also use a 2.2-SNAPSHOT from the Apache snapshot repo (or build
> from sources too).
>
> Gary
>
> On Tue, Dec 2, 2014 at 12:57 PM, Gary Gregory
> wrote:
>
> > Can you try version 2.1?
> >
> > Gary
> >
> > On Tue, Dec 2, 2014 at 12:46 PM, Mariano Gonzalez <
> > mariano.l.gonza...@gmail.com> wrote:
> >
> >> Hello,
> >>
> >> I'm progamatically creating an appender with the following code:
> >>
> >> private static final String PATTERN_LAYOUT = "%-5p %d [%t] %c: %m%n";
> >>
> >> private RollingFileAppender createRollingFileAppender(String
> logFilePath,
> >> String filePattern, String appenderName, Configuration configuration)
> >> {
> >> TriggeringPolicy triggeringPolicy =
> >> TimeBasedTriggeringPolicy.createPolicy("1", "true");
> >> RolloverStrategy rolloverStrategy =
> >> DefaultRolloverStrategy.createStrategy("30", "1", null,
> >> String.valueOf(Deflater.NO_COMPRESSION), configuration);
> >>
> >> return RollingFileAppender.createAppender(logFilePath,
> >> logFilePath + filePattern,
> >> "true",
> >> appenderName,
> >> "true",
> >> null, null,
> >> triggeringPolicy,
> >> rolloverStrategy,
> >> createLayout(configuration),
> >> null, null, null, null,
> >> configuration);
> >> }
> >>
> >> private Layout createLayout(Configuration
> >> configuration)
> >> {
> >> return PatternLayout.createLayout(PATTERN_LAYOUT, configuration,
> null,
> >> null, true, false, null, null);
> >> }
> >>
> >> Sometimes, not always, when an exception is thrown I end up seeing the
> >> following error:
> >>
> >> 2014-12-02 14:36:52,092 ERROR An exception occurred processing Appender
> >> defaultFileAppender java.lang.ArrayIndexOutOfBoundsException: 53
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.impl.ThrowableProxy.formatElements(ThrowableProxy.java:227)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.impl.ThrowableProxy.getExtendedStackTraceAsString(ThrowableProxy.java:401)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:72)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:36)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:189)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:53)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.layout.AbstractStringLayout.toByteArray(AbstractStringLayout.java:52)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:104)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:86)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:97)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:428)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.async.AsyncLoggerConfig.asyncCallAppenders(AsyncLoggerConfig.java:118)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.async.AsyncLoggerConfigHelper$Log4jEventWrapperHandler.onEvent(AsyncLoggerConfigHelper.java:222)
> >>
> >> at
> >>
> >>
> org.apache.logging.log4j.core.async.AsyncLoggerConfigHelpe
ArrayIndexOutOfBoundsException with RollingFileApender
Hello,
I'm progamatically creating an appender with the following code:
private static final String PATTERN_LAYOUT = "%-5p %d [%t] %c: %m%n";
private RollingFileAppender createRollingFileAppender(String logFilePath,
String filePattern, String appenderName, Configuration configuration)
{
TriggeringPolicy triggeringPolicy =
TimeBasedTriggeringPolicy.createPolicy("1", "true");
RolloverStrategy rolloverStrategy =
DefaultRolloverStrategy.createStrategy("30", "1", null,
String.valueOf(Deflater.NO_COMPRESSION), configuration);
return RollingFileAppender.createAppender(logFilePath,
logFilePath + filePattern,
"true",
appenderName,
"true",
null, null,
triggeringPolicy,
rolloverStrategy,
createLayout(configuration),
null, null, null, null,
configuration);
}
private Layout createLayout(Configuration
configuration)
{
return PatternLayout.createLayout(PATTERN_LAYOUT, configuration, null,
null, true, false, null, null);
}
Sometimes, not always, when an exception is thrown I end up seeing the
following error:
2014-12-02 14:36:52,092 ERROR An exception occurred processing Appender
defaultFileAppender java.lang.ArrayIndexOutOfBoundsException: 53
at
org.apache.logging.log4j.core.impl.ThrowableProxy.formatElements(ThrowableProxy.java:227)
at
org.apache.logging.log4j.core.impl.ThrowableProxy.getExtendedStackTraceAsString(ThrowableProxy.java:401)
at
org.apache.logging.log4j.core.pattern.ExtendedThrowablePatternConverter.format(ExtendedThrowablePatternConverter.java:72)
at
org.apache.logging.log4j.core.pattern.PatternFormatter.format(PatternFormatter.java:36)
at
org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:189)
at
org.apache.logging.log4j.core.layout.PatternLayout.toSerializable(PatternLayout.java:53)
at
org.apache.logging.log4j.core.layout.AbstractStringLayout.toByteArray(AbstractStringLayout.java:52)
at
org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:104)
at
org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:86)
at
org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:97)
at
org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:428)
at
org.apache.logging.log4j.core.async.AsyncLoggerConfig.asyncCallAppenders(AsyncLoggerConfig.java:118)
at
org.apache.logging.log4j.core.async.AsyncLoggerConfigHelper$Log4jEventWrapperHandler.onEvent(AsyncLoggerConfigHelper.java:222)
at
org.apache.logging.log4j.core.async.AsyncLoggerConfigHelper$Log4jEventWrapperHandler.onEvent(AsyncLoggerConfigHelper.java:207)
at com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I noticed that it doesn't happen for every exception, but when it does
happen, it happens consistently. I also noticed that it's the causedTrace
array the one that usually falls short. Is this a known issue? Any way I
can get a fix? Maybe with a different layout? I'm using log4j 2.0.2
Thanks you!
What's the AsyncLoggerConfig thread?
Hello, Could you explain what's the purpose of this thread? It seems to keep on running when my application is undeployed. I looked at the code and it seems to be a thread responsible for dispatching events to Disruptor, but if that's the case then the name may not be very descriptive IMHO. Can you confirm what's the purpose of this thread and that it should stop after the logging context receives the stop() message? Thanks!
Re: DefaultRolloeverStrategy
Thank you Remko and Ralph! That clarifies a lot. Then I have to ask, if my pattern only contains a %d, what value should I use as max value? Does it matter at all? Thanks! On Wed, Jul 30, 2014 at 8:23 PM, Remko Popma wrote: > I stand corrected. Thanks, Ralph! > > Sent from my iPhone > > > On 2014/07/31, at 8:14, Ralph Goers wrote: > > > > Remko, that is not exactly correct. If you specify %i in the > filePattern once the max value is hit the oldest file that matches the > complete pattern will be deleted automatically. The issue you are speaking > of is that when the pattern includes a date via %d we will not delete files > from previous dates. > > > > If the pattern only contains %d and does not contain %i then the files > will never be deleted by Log4j. > > > > Ralph > > > > > >> On Jul 30, 2014, at 3:44 PM, Remko Popma wrote: > >> > >> The built-in rollover only moves/renames but never deletes files. (We > actually have outstanding enhancement request to add some auto-delete > function.) > >> > >> Sent from my iPhone > >> > >>> On 2014/07/31, at 4:30, Mariano Gonzalez > wrote: > >>> > >>> Hello, > >>> > >>> I'm configuring a RollingFileAppender programatically, using a > >>> TimeBasedTriggeringPolicy in tandem with a DefaultRooloverStrategy. So > far > >>> so good, but is there a way to provide a RolloeverStrategy which > renames > >>> the files but never deletes them? > >>> > >>> Thanks! > >> > >> - > >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > >> For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > > > - > > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > - > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > For additional commands, e-mail: log4j-user-h...@logging.apache.org > >
DefaultRolloeverStrategy
Hello, I'm configuring a RollingFileAppender programatically, using a TimeBasedTriggeringPolicy in tandem with a DefaultRooloverStrategy. So far so good, but is there a way to provide a RolloeverStrategy which renames the files but never deletes them? Thanks!
Re: Disruptor 3.2.x
great. Thank you all! On Tue, Jul 22, 2014 at 8:01 PM, Gary Gregory wrote: > Please see https://issues.apache.org/jira/browse/LOG4J2-732 > > Mariano, please checkout and build trunk and see if you like using the LMAX > Disruptor 3.3.0 :-) > > This update required no code changes. > > Feel free to close the ticket if your case runs OK. > > Gary > > > On Tue, Jul 22, 2014 at 5:56 PM, Remko Popma > wrote: > > > Mariano, the site says async loggers require 3.0.0 or higher. So they'll > > work with 3.0 and better. We've tested with 3.0. 3.1, 3.2 and 3.2.1. > > > > We haven't tested with 3.3 yet. > > > > Gary, the Disruptor 3.3 release notes mention a breaking api change, so > > let's keep it at 3.2.1 until I've had a chance to look at it. > > > > Sent from my iPhone > > > > > On 2014/07/23, at 3:13, Gary Gregory wrote: > > > > > > It looks like 3.3.0 is the latest. > > > > > > Feel free to submit a patch. > > > > > > Gary > > > > > > Original message From: Mariano > Gonzalez > > Date:07/22/2014 11:18 > > (GMT-05:00) To: Log4J Users List < > > log4j-user@logging.apache.org> Subject: Disruptor 3.2.x > > > > > Hello, > > > > > > According to the manual, async loggers require Disruptor 3.0. However, > > the > > > 3.2.x series of disruptor is available. Have you tried this version? > Are > > > you recommending 3.0 because you found issues with other versions or is > > it > > > just in your TODO list to give 3.2 a ride and/or update the manual? > > > > > > Thanks > > > > - > > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > > > > -- > E-Mail: garydgreg...@gmail.com | ggreg...@apache.org > Java Persistence with Hibernate, Second Edition > <http://www.manning.com/bauer3/> > JUnit in Action, Second Edition <http://www.manning.com/tahchiev/> > Spring Batch in Action <http://www.manning.com/templier/> > Blog: http://garygregory.wordpress.com > Home: http://garygregory.com/ > Tweet! http://twitter.com/GaryGregory >
Disruptor 3.2.x
Hello, According to the manual, async loggers require Disruptor 3.0. However, the 3.2.x series of disruptor is available. Have you tried this version? Are you recommending 3.0 because you found issues with other versions or is it just in your TODO list to give 3.2 a ride and/or update the manual? Thanks
Re: why is AsyncLoggerContextSelector faster?
Thanks Remko and Matt for your comments. I will take your suggestions and let you know how it goes. Just out of curiosity (I don't like to use libraries blindly and way too curious), where are the loggers instantiated/configured?? I followed the code of XmlConfiguration but it seems to only parse configuration items, I couldn't find where it parses and configures the loggers. I admit I didn't look that hard, but if you could provide a pointer to better understand how this all works that'd be great. Thanks! On Mon, Jul 21, 2014 at 12:12 PM, Remko Popma wrote: > Mariano, to clarify, I would recommend that you keep your custom context > selector simple and only use it for the log separation that you mentioned. > I would not recommend that you also somehow try to merge the functionality > of AsyncLoggerContextSelector in your context selector. From a performance > perspective that should not be necessary. If you use and > in your configuration then I doubt that logging will be a > performance bottleneck. > > > On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma > wrote: > > > Matt, not sure what you mean. Mariano needs to develop his own > > ContextSelector to achieve log separation. > > So he cannot use the AsyncLoggerContextSelector. > > > > Mariano, I would not be overly concerned with the performance results > > mentioned on the async logger manual page. Bear in mind that in that > > performance test, _all_ that the program does is logging. In the real > world > > (your application) that will obviously not be true: your application will > > do many other things. > > > > Also, mixed async loggers performed much, much better than > AsyncAppenders. > > > > So, unless you have actual measurement numbers for _your_ application, > not > > the theoretical numbers from the async logger manual page, I would keep > it > > simple at first, just get your custom selector to work well, and then > start > > measuring performance. > > > > If you are getting strange results, like AsyncAppenders performing better > > than AsyncLoggers, then let us know and we'll try to help figure out what > > the problem is. > > > > > > > > On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker wrote: > > > >> Well, the way to go that's similar to the async version is to use the > >> BasicContextSelector which contains a singleton LoggerContext. > Otherwise, > >> you'll have to keep your own registry that can be looked up dynamically > >> such as through reflection of the caller stack and other fun design > >> problems. :) > >> > >> > >> On 21 July 2014 09:35, Mariano Gonzalez > >> wrote: > >> > >> > Hello Remko, > >> > > >> > I'm still a couple of days away from starting my own performance > >> testing. > >> > I'm taking about the difference in the async loggers manual page, more > >> > specifically, the charts that compare sync loggers, to mixed async > >> loggers > >> > against purely async loggers. Since I need to build my own selector, > I'm > >> > trying to be clear on how this works internally in order to implement > >> the > >> > less latency possible selector and try to minimize the performance > >> testing > >> > effort. > >> > > >> > Thanks for the clarifications! > >> > > >> > > >> > > >> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma > >> > wrote: > >> > > >> > > Hi, > >> > > No that is incorrect. > >> > > If you do not specify AsyncLoggerContextSelector but instead > configure > >> > with > >> > > and loggers, you _do_ need the disruptor > >> jar on > >> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender > is > >> > > completely separate from Async Loggers. Async Loggers (mixed or all > >> > async) > >> > > use the disruptor and both need the disruptor jar. > >> > > > >> > > You keep mentioning a performance difference. I was assuming you > were > >> > > talking about the performance test results mentioned on the Async > >> Logger > >> > > manual page, but perhaps I was wrong? Are you experiencing a > >> performance > >> > > difference between the two flavors of Async Loggers in your > >> application? > >> > > > >> > > > >> > > > >&g
Re: why is AsyncLoggerContextSelector faster?
Hello Remko, I'm still a couple of days away from starting my own performance testing. I'm taking about the difference in the async loggers manual page, more specifically, the charts that compare sync loggers, to mixed async loggers against purely async loggers. Since I need to build my own selector, I'm trying to be clear on how this works internally in order to implement the less latency possible selector and try to minimize the performance testing effort. Thanks for the clarifications! On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma wrote: > Hi, > No that is incorrect. > If you do not specify AsyncLoggerContextSelector but instead configure with > and loggers, you _do_ need the disruptor jar on > the classpath and this does _not_ use AsyncAppender. AsyncAppender is > completely separate from Async Loggers. Async Loggers (mixed or all async) > use the disruptor and both need the disruptor jar. > > You keep mentioning a performance difference. I was assuming you were > talking about the performance test results mentioned on the Async Logger > manual page, but perhaps I was wrong? Are you experiencing a performance > difference between the two flavors of Async Loggers in your application? > > > > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez < > mariano.l.gonza...@gmail.com> wrote: > > > Hello Remko, > > > > I think I found the difference. AsyncLoggerContextSelector always returns > > the same instance of AsyncLoggerContext, which in turns always returns > > instances of AsyncLogger, which uses disruptor to handle concurrency. > > > > However, with any other selector, a standard Logger instance is returned > > and parallelism is achieved through an AsyncAppender. AsyncAppender use a > > standard blocking queue instead of using disruptor which explains the > > performance difference (there's also the fact that > > AsyncLoggerContextSelector always returns the same context instance and > > does not spend cycles in the lookup, but I think that is not a > significant > > cost once everything was warmed out). > > > > http://logging.apache.org/log4j/2.x/manual/async.html says that when > using > > mixed type loggers disruptor is needed on the classpath. That seems to be > > an error. For what I see disruptor is only used when setting all loggers > > asynchronous. > > > > Does this make sense? Anyway around this? Do you have a disruptor > appender > > somewhere? > > > > Thanks! > > > > > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma > > wrote: > > > > > To be honest, I haven't investigated in detail the reason for the > > > difference in throughput in the performance test. > > > > > > Are you measuring the performance of your application container, and > can > > > you see an improvement when using Async Loggers? > > > Do you see a large difference in performance _in your application_ > > between > > > making all loggers Asynchronous and using mixed synchronous and > > > Asynchronous loggers? > > > > > > > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez < > > > mariano.l.gonza...@gmail.com> wrote: > > > > > > > Hello Remko, > > > > > > > > Thanks for the insight. I guess my case falls into the wrong end of > the > > > > pareto law. My project is a low latency application container, so I > > need > > > to > > > > have: > > > > > > > > > > > >- low latency > > > >- log separation (I actually had to implement my own context > > selector > > > >because my logic is more complicated than the standard > > > >ClassLoaderContextSelector case) > > > >- I want async loggers by default, but deployed apps need to be > able > > > to > > > >specify sync loggers > > > > > > > > > > > > Right now I'm kinda meeting those requirements using config file, > > > AsyncRoot > > > > and my custom selector, but it'd be really great to achieve a > > performance > > > > level like the one that AsyncContextSelector promises. > > > > > > > > Is there a way of doing that? For what I see on the code, the > > > > AsyncLoggerContextSelector's secret sauce is just to always return an > > > > AsyncLogger on the newInstance() method. Why is that so much faster > > than > > > > what ClassLoaderLoggerContextSelector does? > > > > > > > > Thanks! >
Re: why is AsyncLoggerContextSelector faster?
Hello Remko, I think I found the difference. AsyncLoggerContextSelector always returns the same instance of AsyncLoggerContext, which in turns always returns instances of AsyncLogger, which uses disruptor to handle concurrency. However, with any other selector, a standard Logger instance is returned and parallelism is achieved through an AsyncAppender. AsyncAppender use a standard blocking queue instead of using disruptor which explains the performance difference (there's also the fact that AsyncLoggerContextSelector always returns the same context instance and does not spend cycles in the lookup, but I think that is not a significant cost once everything was warmed out). http://logging.apache.org/log4j/2.x/manual/async.html says that when using mixed type loggers disruptor is needed on the classpath. That seems to be an error. For what I see disruptor is only used when setting all loggers asynchronous. Does this make sense? Anyway around this? Do you have a disruptor appender somewhere? Thanks! On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma wrote: > To be honest, I haven't investigated in detail the reason for the > difference in throughput in the performance test. > > Are you measuring the performance of your application container, and can > you see an improvement when using Async Loggers? > Do you see a large difference in performance _in your application_ between > making all loggers Asynchronous and using mixed synchronous and > Asynchronous loggers? > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez < > mariano.l.gonza...@gmail.com> wrote: > > > Hello Remko, > > > > Thanks for the insight. I guess my case falls into the wrong end of the > > pareto law. My project is a low latency application container, so I need > to > > have: > > > > > >- low latency > >- log separation (I actually had to implement my own context selector > >because my logic is more complicated than the standard > >ClassLoaderContextSelector case) > >- I want async loggers by default, but deployed apps need to be able > to > >specify sync loggers > > > > > > Right now I'm kinda meeting those requirements using config file, > AsyncRoot > > and my custom selector, but it'd be really great to achieve a performance > > level like the one that AsyncContextSelector promises. > > > > Is there a way of doing that? For what I see on the code, the > > AsyncLoggerContextSelector's secret sauce is just to always return an > > AsyncLogger on the newInstance() method. Why is that so much faster than > > what ClassLoaderLoggerContextSelector does? > > > > Thanks! > > > > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma > > wrote: > > > > > The Async Loggers created with the context selector have a slightly > > > different mechanism. One of the differences is that LogEvent objects > are > > > re-used. > > > > > > However, unless your application is in the low-latency space, I would > not > > > worry too much about the performance difference. Both flavors of Async > > > Loggers are much faster than the alternative (Async Appenders). > > > > > > Your point is valid though. I've been thinking about an alternative way > > to > > > configure Async Loggers than via system properties. The work in > progress > > is > > > tracked in Jira ticket LOG4J2-321 > > > <https://issues.apache.org/jira/browse/LOG4J2-321>. This is still in > the > > > concept phase though. Meanwhile your best option is probably to use > > > ClassLoaderContextSelector and configure with and > > > . > > > > > > > > > > > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez < > > > mariano.l.gonza...@gmail.com> wrote: > > > > > > > Hello, > > > > > > > > According to the performance charts in the documentation, log4j2 has > a > > > > significantly higher throughput when using AsyncLoggerContextSelector > > > than > > > > when using all async loggers with any different selector. > > > > > > > > Why is that? Is it just because the same context is always reused and > > > > there's no lookup like in the ClassLoaderContextSelector case? > > > > > > > > If I need functionality similar to ClassLoaderContextSelector, is > there > > > any > > > > way to get a throughput similar to AsyncLoggerContextSelector? > > > > > > > > Thanks! > > > > > > > > > >
Re: why is AsyncLoggerContextSelector faster?
Hello Remko, Thanks for the insight. I guess my case falls into the wrong end of the pareto law. My project is a low latency application container, so I need to have: - low latency - log separation (I actually had to implement my own context selector because my logic is more complicated than the standard ClassLoaderContextSelector case) - I want async loggers by default, but deployed apps need to be able to specify sync loggers Right now I'm kinda meeting those requirements using config file, AsyncRoot and my custom selector, but it'd be really great to achieve a performance level like the one that AsyncContextSelector promises. Is there a way of doing that? For what I see on the code, the AsyncLoggerContextSelector's secret sauce is just to always return an AsyncLogger on the newInstance() method. Why is that so much faster than what ClassLoaderLoggerContextSelector does? Thanks! On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma wrote: > The Async Loggers created with the context selector have a slightly > different mechanism. One of the differences is that LogEvent objects are > re-used. > > However, unless your application is in the low-latency space, I would not > worry too much about the performance difference. Both flavors of Async > Loggers are much faster than the alternative (Async Appenders). > > Your point is valid though. I've been thinking about an alternative way to > configure Async Loggers than via system properties. The work in progress is > tracked in Jira ticket LOG4J2-321 > <https://issues.apache.org/jira/browse/LOG4J2-321>. This is still in the > concept phase though. Meanwhile your best option is probably to use > ClassLoaderContextSelector and configure with and > . > > > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez < > mariano.l.gonza...@gmail.com> wrote: > > > Hello, > > > > According to the performance charts in the documentation, log4j2 has a > > significantly higher throughput when using AsyncLoggerContextSelector > than > > when using all async loggers with any different selector. > > > > Why is that? Is it just because the same context is always reused and > > there's no lookup like in the ClassLoaderContextSelector case? > > > > If I need functionality similar to ClassLoaderContextSelector, is there > any > > way to get a throughput similar to AsyncLoggerContextSelector? > > > > Thanks! > > >
why is AsyncLoggerContextSelector faster?
Hello, According to the performance charts in the documentation, log4j2 has a significantly higher throughput when using AsyncLoggerContextSelector than when using all async loggers with any different selector. Why is that? Is it just because the same context is always reused and there's no lookup like in the ClassLoaderContextSelector case? If I need functionality similar to ClassLoaderContextSelector, is there any way to get a throughput similar to AsyncLoggerContextSelector? Thanks!
Any backwards compatibility tool for old log4j.properties?
Hello all, I'm part of the development team of a popular middleware product that acts as an application container. I'm working in moving from log4j 1.2 to 2.0 for our next release. We noticed that most of our users use a log4j.properties file to configure logging in their applications. To make migration easier, I would like to build a custom ConfigurationFactory which is able to parse that .properties file and create the configuration. Do you already have something like that? Anyone had this migration issue already and has some tool for it? Is anyone considering adding something like this to the product? Thanks!
