Thanks Remko for the valuable insight on queue size and ring buffer size in log4j and logback. I will make adjustments to the settings and run the test again.
On Tue, Oct 1, 2013 at 10:02 AM, Remko Popma <remko.po...@gmail.com> wrote: > Michael, > > I finally got around to looking at the test results you posted on Github ( > https://github.com/michaelzhou999/logging-profiling). > > Very extensive testing, I can tell you put a lot of work into it! > > I was initially a bit surprised by your Log4J2 finding that at ten threads > or more, synchronous logging with RandomAccessFile is even faster than > async logging with RandomAccessFile. But I believe this is because the > performance test logs 500,000 events, and the async logger ring buffer by > default only has 256*1024 slots. When the ring buffer is full the test can > only put new log events into the ring buffer when previous events have been > taken out, so essentially it will be running at the speed of the > RandomAccessFile appender with the additional overhead of going through the > ring buffer. > > To be honest I am not sure why this effect does not occur with 1 - 5 > threads. > > Also interesting to see a similar phenomenon with LogBack: > async appender is faster with 1 - 5 threads, but from 10 threads and up, > synchronous logging is faster than asynchrous logging. > > In this case, the LogBack default queue size is only 256, and it is a > fixed-size queue, so it is basically full all the way through the test. So > I think what we're seeing here is the throughput of the underlying appender > plus the contention of the multiple threads trying to add to the queue. > Again unsure about the difference between 10+ threads and 1 - 5 threads... > (I noticed you correctly set the "discardingThreshold" to zero for LogBack > so it won't start dropping trace/debug/info events when the queue is 80% > full. Well-spotted! Don't allow LogBack to cheat! :-) ) > > But to make the test fair the LogBack queue size should really be the same > as the Log4J2 async logger queue size (256*1024). > > Actually, I would recommend making the queues large enough to contain all > events. If you want to log 500,000 in the test, the queues should be at > least that size. Otherwise the async test will actually be measuring a > partially async and partially synchronous logging. > > One final note: are you aware that Log4J2 natively supports SLF4J-style > formatting? e.g. logger.info("hi {}", "world!"); > > Overall, I was happy to see that your results are similar to the results I > reported in the performance section of the Async Loggers page. > > Best regards, > Remko > > > > > On Sun, Sep 29, 2013 at 9:59 PM, Michael Zhou <michael.z...@gmail.com > >wrote: > > > Hi Remko, > > > > You can find the micro-benchmarking tool I wrote on Github: > > https://github.com/michaelzhou999/logging-profiling > > > > Drop me a message if you have any questions. Thanks > > > > Michael > > > > > > On Sat, Sep 28, 2013 at 9:34 PM, Remko Popma <remko.po...@gmail.com> > > wrote: > > > > > Michael, > > > > > > I'd be interested to see your benchmarking code and results. Is it > > > possible for you to share them? > > > > > > Remko > > > > > > Sent from my iPhone > > > > > > > On 2013/09/29, at 7:58, Michael Zhou <michael.z...@gmail.com> wrote: > > > > > > > > The micro-benchmark testing I wrote to compare Log4j 1 and 2 shows > > > > considerable performance gain in favor of Log4j 2. I have only one > > > request > > > > for your consideration: simplifying the way to get all loggers in the > > > > system and set/reset logging levels for certain loggers. This is > very > > > easy > > > > to do in Log4j 1, but thanks to the rewrite of API and implementation > > in > > > > Log4j 2, it becomes complicated. I did search and read previous > > > > discussions (in August and May) on this topic, and consider this a > much > > > > needed feature request. Thanks > > > > > > > > > > > > On Sat, Sep 28, 2013 at 2:44 AM, Ralph Goers < > > ralph.go...@dslextreme.com > > > >wrote: > > > > > > > >> I'd prefer to turn this around and ask a couple of questions: > > > >> > > > >> 1. Why do you need to know the "official release" date? Is > something > > > >> blocking you from using Log4j 2? > > > >> 2. Are you currently using the beta releases? Are you satisfied > that > > it > > > >> is production ready in your environment? > > > >> > > > >> The answers to the questions above are important for us as that is > the > > > >> basis for us labeling the release 2.0 GA vs 2.0 beta10 (or 2.0 RC1). > > > >> > > > >> Ralph > > > >> > > > >>> On Sep 27, 2013, at 8:07 AM, Jingdong Sun wrote: > > > >>> > > > >>> Can anyone tell me when the log4j 2 official release will be > > available? > > > >>> > > > >>> Thanks. > > > >>> Jingdong Sun > > > >>> InfoSphere Streams Development > > > >>> Phone 507 253-5958 (T/L 553-5958) > > > >>> jind...@us.ibm.com > > > >> > > > >> > > > >> > --------------------------------------------------------------------- > > > >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > > >> For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > > > > > > > > > -- > > > > Michael > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org > > > For additional commands, e-mail: log4j-user-h...@logging.apache.org > > > > > > > > > > > > -- > > Michael > > > -- Michael
