Mariano, to clarify, I would recommend that you keep your custom context selector simple and only use it for the log separation that you mentioned. I would not recommend that you also somehow try to merge the functionality of AsyncLoggerContextSelector in your context selector. From a performance perspective that should not be necessary. If you use <AsyncRoot> and <AsyncLogger> in your configuration then I doubt that logging will be a performance bottleneck.
On Mon, Jul 21, 2014 at 11:47 PM, Remko Popma <[email protected]> wrote: > Matt, not sure what you mean. Mariano needs to develop his own > ContextSelector to achieve log separation. > So he cannot use the AsyncLoggerContextSelector. > > Mariano, I would not be overly concerned with the performance results > mentioned on the async logger manual page. Bear in mind that in that > performance test, _all_ that the program does is logging. In the real world > (your application) that will obviously not be true: your application will > do many other things. > > Also, mixed async loggers performed much, much better than AsyncAppenders. > > So, unless you have actual measurement numbers for _your_ application, not > the theoretical numbers from the async logger manual page, I would keep it > simple at first, just get your custom selector to work well, and then start > measuring performance. > > If you are getting strange results, like AsyncAppenders performing better > than AsyncLoggers, then let us know and we'll try to help figure out what > the problem is. > > > > On Mon, Jul 21, 2014 at 11:38 PM, Matt Sicker <[email protected]> wrote: > >> Well, the way to go that's similar to the async version is to use the >> BasicContextSelector which contains a singleton LoggerContext. Otherwise, >> you'll have to keep your own registry that can be looked up dynamically >> such as through reflection of the caller stack and other fun design >> problems. :) >> >> >> On 21 July 2014 09:35, Mariano Gonzalez <[email protected]> >> wrote: >> >> > Hello Remko, >> > >> > I'm still a couple of days away from starting my own performance >> testing. >> > I'm taking about the difference in the async loggers manual page, more >> > specifically, the charts that compare sync loggers, to mixed async >> loggers >> > against purely async loggers. Since I need to build my own selector, I'm >> > trying to be clear on how this works internally in order to implement >> the >> > less latency possible selector and try to minimize the performance >> testing >> > effort. >> > >> > Thanks for the clarifications! >> > >> > >> > >> > On Mon, Jul 21, 2014 at 11:22 AM, Remko Popma <[email protected]> >> > wrote: >> > >> > > Hi, >> > > No that is incorrect. >> > > If you do not specify AsyncLoggerContextSelector but instead configure >> > with >> > > <AsyncRoot> and <AsyncLogger> loggers, you _do_ need the disruptor >> jar on >> > > the classpath and this does _not_ use AsyncAppender. AsyncAppender is >> > > completely separate from Async Loggers. Async Loggers (mixed or all >> > async) >> > > use the disruptor and both need the disruptor jar. >> > > >> > > You keep mentioning a performance difference. I was assuming you were >> > > talking about the performance test results mentioned on the Async >> Logger >> > > manual page, but perhaps I was wrong? Are you experiencing a >> performance >> > > difference between the two flavors of Async Loggers in your >> application? >> > > >> > > >> > > >> > > >> > > On Mon, Jul 21, 2014 at 11:10 PM, Mariano Gonzalez < >> > > [email protected]> wrote: >> > > >> > > > Hello Remko, >> > > > >> > > > I think I found the difference. AsyncLoggerContextSelector always >> > returns >> > > > the same instance of AsyncLoggerContext, which in turns always >> returns >> > > > instances of AsyncLogger, which uses disruptor to handle >> concurrency. >> > > > >> > > > However, with any other selector, a standard Logger instance is >> > returned >> > > > and parallelism is achieved through an AsyncAppender. AsyncAppender >> > use a >> > > > standard blocking queue instead of using disruptor which explains >> the >> > > > performance difference (there's also the fact that >> > > > AsyncLoggerContextSelector always returns the same context instance >> and >> > > > does not spend cycles in the lookup, but I think that is not a >> > > significant >> > > > cost once everything was warmed out). >> > > > >> > > > http://logging.apache.org/log4j/2.x/manual/async.html says that >> when >> > > using >> > > > mixed type loggers disruptor is needed on the classpath. That seems >> to >> > be >> > > > an error. For what I see disruptor is only used when setting all >> > loggers >> > > > asynchronous. >> > > > >> > > > Does this make sense? Anyway around this? Do you have a disruptor >> > > appender >> > > > somewhere? >> > > > >> > > > Thanks! >> > > > >> > > > >> > > > On Sat, Jul 19, 2014 at 11:55 PM, Remko Popma < >> [email protected]> >> > > > wrote: >> > > > >> > > > > To be honest, I haven't investigated in detail the reason for the >> > > > > difference in throughput in the performance test. >> > > > > >> > > > > Are you measuring the performance of your application container, >> and >> > > can >> > > > > you see an improvement when using Async Loggers? >> > > > > Do you see a large difference in performance _in your application_ >> > > > between >> > > > > making all loggers Asynchronous and using mixed synchronous and >> > > > > Asynchronous loggers? >> > > > > >> > > > > >> > > > > On Sun, Jul 20, 2014 at 7:45 AM, Mariano Gonzalez < >> > > > > [email protected]> wrote: >> > > > > >> > > > > > Hello Remko, >> > > > > > >> > > > > > Thanks for the insight. I guess my case falls into the wrong >> end of >> > > the >> > > > > > pareto law. My project is a low latency application container, >> so I >> > > > need >> > > > > to >> > > > > > have: >> > > > > > >> > > > > > >> > > > > > - low latency >> > > > > > - log separation (I actually had to implement my own context >> > > > selector >> > > > > > because my logic is more complicated than the standard >> > > > > > ClassLoaderContextSelector case) >> > > > > > - I want async loggers by default, but deployed apps need to >> be >> > > able >> > > > > to >> > > > > > specify sync loggers >> > > > > > >> > > > > > >> > > > > > Right now I'm kinda meeting those requirements using config >> file, >> > > > > AsyncRoot >> > > > > > and my custom selector, but it'd be really great to achieve a >> > > > performance >> > > > > > level like the one that AsyncContextSelector promises. >> > > > > > >> > > > > > Is there a way of doing that? For what I see on the code, the >> > > > > > AsyncLoggerContextSelector's secret sauce is just to always >> return >> > an >> > > > > > AsyncLogger on the newInstance() method. Why is that so much >> faster >> > > > than >> > > > > > what ClassLoaderLoggerContextSelector does? >> > > > > > >> > > > > > Thanks! >> > > > > > >> > > > > > >> > > > > > On Fri, Jul 18, 2014 at 1:20 PM, Remko Popma < >> > [email protected]> >> > > > > > wrote: >> > > > > > >> > > > > > > The Async Loggers created with the context selector have a >> > slightly >> > > > > > > different mechanism. One of the differences is that LogEvent >> > > objects >> > > > > are >> > > > > > > re-used. >> > > > > > > >> > > > > > > However, unless your application is in the low-latency space, >> I >> > > would >> > > > > not >> > > > > > > worry too much about the performance difference. Both flavors >> of >> > > > Async >> > > > > > > Loggers are much faster than the alternative (Async >> Appenders). >> > > > > > > >> > > > > > > Your point is valid though. I've been thinking about an >> > alternative >> > > > way >> > > > > > to >> > > > > > > configure Async Loggers than via system properties. The work >> in >> > > > > progress >> > > > > > is >> > > > > > > tracked in Jira ticket LOG4J2-321 >> > > > > > > <https://issues.apache.org/jira/browse/LOG4J2-321>. This is >> > still >> > > in >> > > > > the >> > > > > > > concept phase though. Meanwhile your best option is probably >> to >> > use >> > > > > > > ClassLoaderContextSelector and configure with <AsyncRoot> and >> > > > > > > <AsyncLogger>. >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > On Fri, Jul 18, 2014 at 10:57 PM, Mariano Gonzalez < >> > > > > > > [email protected]> wrote: >> > > > > > > >> > > > > > > > Hello, >> > > > > > > > >> > > > > > > > According to the performance charts in the documentation, >> > log4j2 >> > > > has >> > > > > a >> > > > > > > > significantly higher throughput when using >> > > > AsyncLoggerContextSelector >> > > > > > > than >> > > > > > > > when using all async loggers with any different selector. >> > > > > > > > >> > > > > > > > Why is that? Is it just because the same context is always >> > reused >> > > > and >> > > > > > > > there's no lookup like in the ClassLoaderContextSelector >> case? >> > > > > > > > >> > > > > > > > If I need functionality similar to >> ClassLoaderContextSelector, >> > is >> > > > > there >> > > > > > > any >> > > > > > > > way to get a throughput similar to >> AsyncLoggerContextSelector? >> > > > > > > > >> > > > > > > > Thanks! >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > >> > >> >> >> >> -- >> Matt Sicker <[email protected]> >> > >
