[
https://issues.apache.org/jira/browse/LOG4J2-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207371#comment-15207371
]
Ron Gonzalez edited comment on LOG4J2-1324 at 3/22/16 9:47 PM:
---------------------------------------------------------------
we dug into the heap dump and found that the original asyncloggerconfig thread
had exited, but the ring buffer and other objects were still instantiated. We
have reached a preliminary conclusion that the root cause is that an exception
is being raised that is aborting the consumer thread (which is why a new
AsyncLoggerConfig-2 is being instantiated) however, when the default disruptor
strategy is set as blocking, there's no chance for the disruptor to release
it's "blockingwaitstrategy" lock, so the new async logger config thread is
unable to start servicing the ring buffer again (or some such variation of this
interaction) as we haven't quite pieced it all together yet.
In log4j2-1221 the conclusion was reached that a "signal was missed" in
solaris, however if you look at the ticket
https://github.com/LMAX-Exchange/disruptor/issues/138 you will notice that
others have reproduced with linux, and that the maintainer has not been able to
reproduce it with the disruptor in isolation.
I believe the core issue is release of the lock when the thread aborts in the
blocking wait strategy, but this might be something needing to be handled in
log4j instead of in the disruptor.
was (Author: rong):
we dug into the heap dump and found that the original asyncloggerconfig thread
had exited, but the ring buffer and other objects were still instantiated. We
have reached a preliminary conclusion that the root cause is that an exception
is being raised that is aborting the consumer thread (which is why a new
AsyncLoggerConfig-2 is being instantiated) however, when the default disruptor
strategy is set as blocking, there's no chance for the disruptor to release
it's "blockingwaitstrategy" lock, so the new async logger config thread is
unable to start servicing the ring buffer again (or some such variation of this
interaction) as we haven't quite pieced it all together yet.
> Async Logger - Consumer thread dying - new thread unable to start
> -----------------------------------------------------------------
>
> Key: LOG4J2-1324
> URL: https://issues.apache.org/jira/browse/LOG4J2-1324
> Project: Log4j 2
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.2
> Environment: LOG4J CORE Release Version 2.2
> Disruptor Bundle-Version 3.3.2
> ORACLE jdk1.8.0_45
> Linux 2.6.32-573.el6.x86_64, amd64
> Reporter: Ron Gonzalez
> Attachments: 2016-03-18_17-44-06.jpg, 2016-03-22_8-32-37.jpg,
> BatchEventProcessor.png, log4j2_config.xml
>
>
> We are seeing a situation where the consumer thread
> "AsyncLoggerConfig-1" is apparently dying.
> We do see a new consumer thread trying to start up, but it is blocked
> waiting for a lock, so no logging is happening.
> Is this a defect in log4j?
> Is the original consumer thread dying due to perhaps an unhandled exception ?
> Is that original consumer thread not terminating gracefully and
> releasing the locked object so the new consumer thread can start?
> Please see attached screenshot. Top of screen is normal
> asyncloggerconfig thread (consumer).
> Bottom is thread trace when logging stops, we no longer see an
> "asyncloggerconfig-1" thread, instead a new thread is trying to start
> but never does "asyncloggerconfig-2".
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
