Re: porting log4j2 to .NET

[Mikael Ståldal]

Just to make things clear, Log4j is a logging framework for the JVM
platform, and it is agnostic to the underlying OS. It it well tested on (at
least) both Linux and Windows.

On Tue, Oct 18, 2016 at 2:33 AM, Nicholas Duane  wrote:

> Figured I would send this question out to the log4j side.  I have already
> had some email exchanges with the log4net mailing list regarding porting
> log4j2 to .NET.  My suggestion was that the apache logging framework be a
> single architecture design which is platform agnostic and then teams which
> port to the different platforms.  It seems log4net was a port of log4j and
> may be going off in its own direction from that initial port.  My viewpoint
> is that's a bad idea as one of the benefits I saw was that log4net was
> similar to log4j2 and we're looking for logging frameworks for our
> enterprise.  We have applications on both Windows/.NET and Linux/Java so
> having a logging framework for Windows/.NET which is similar to a logging
> framework for Linux/Java was a big plus.
>
>
> While I have no doubt the effort to port log4j2 to .NET is considerable,
> it would be a port and thus I'm not spending time figuring out design and
> algorithms.  Would anyone want to venture a guess at what that effort might
> be?
>
>
> Thanks,
>
> Nick
>



-- 
[image: MagineTV]

*Mikael Ståldal*
Senior software developer

*Magine TV*
mikael.stal...@magine.com
Grev Turegatan 3  | 114 46 Stockholm, Sweden  |   www.magine.com

Privileged and/or Confidential Information may be contained in this
message. If you are not the addressee indicated in this message
(or responsible for delivery of the message to such a person), you may not
copy or deliver this message to anyone. In such case,
you should destroy this message and kindly notify the sender by reply
email.

Re: approach for defining loggers

[Matt Sicker]

What about event logging? <
https://logging.apache.org/log4j/2.x/manual/eventlogging.html>

This sounds pretty similar to what you're asking about. You define a map
message essentially, plus your other requirements seem to be met here.

On 17 October 2016 at 21:46, Gary Gregory  wrote:

> On Mon, Oct 17, 2016 at 4:27 PM, Nicholas Duane  wrote:
>
> > Sorry to revive this old thread.  However, we're in the process of adding
> > support for other "categories" of events and thus I wanted to first take
> a
> > step back and try to ensure we're not convoluting things.
> >
> >
> > There was a requirement to log a "compliance" event under certain
> > conditions.  We did not want to write our own logging framework and
> instead
> > decided to use existing off-the-shelf logging frameworks.  We have
> > applications on both Linux/Java, Windows/.NET and Windows/Java.
> Initially
> > we chose log4net for Windows/.NET and log4j2 for Windows/Java and
> > Linux/Java.  For these logging frameworks we wrote artifacts, appenders
> > basically, to help facilitate getting these events to our system.  By the
> > way, our system will get the events centrally, possibly put them into a
> > relational database and also hand them off to another system which will
> get
> > them eventually to an HDFS backend.  We also exposed methods for creating
> > this compliance event.  The compliance event is basically a map.  We
> chose
> > a map so that the event could also be extended by the application team in
> > case they needed to add additional properties which made sense for them.
> >
> >
> > We chose to create a custom level for this "compliance" event such that
> we
> > could filter out only these events and get them into our system.  The
> > configuration example we created had our custom unix domain socket
> appender
> > added to the root logger.  It also contained a filter which filtered out
> > any events that weren't compliance events.  The level we chose for
> > "compliance" was less critical than off and more critical than fatal as
> we
> > wanted to ensure that as long as logging wasn't turned off altogether our
> > events would get logged.
> >
> >
> > I want to go over a few suggestions that were made and explain why I
> > didn't make use of those suggestions.
> >
> >
> > 1. Since our "compliance" level didn't fit within the "vernacular" of the
> > existing levels we should not define this custom level.  Instead we
> should
> > look at using markers.
> >
>
> Yes, this is a use case for markers. The level should be used to note how
> important is each compliance event.
>
>
> >
> > I am not that familiar with markers but did look into them when they were
> > suggested.  While I don't have anything against markers in general there
> > were some downsides as I saw it.
> >
> >
> > a. Markers are not available, as far as I know, in log4net so we'd still
> > have to figure out something there.
> >
>
> Indeed, we really need a port of Log4j 2 to .NET.
>
>
> >
> > b. A bigger problem, at least I thought it was a bigger problem, was that
> > there would be confusion about what level to log the event at.  I would
> > certainly not want to give an example as follows:
> >
> >
> > logger.debug(COMPLIANCE_MARKER, evnt);
> >
> >
> > or
> >
> >
> > logger.info(COMPLIANCE_MARKER, evnt);
> >
> >
> > or
> >
> >
> > logger.error(COMPLIANCE_MARKER, evnt);
> >
> >
> > ...
> >
>
> Think about: How important is this event? Are there different level of
> importance to the audience?
>
>
> >
> >
> > That just screams confusion to me.
> >
> >
> > 2. Use a dedicated logger to log all compliance events.
> >
> >
> > There were, as far as I could tell, a couple problems with this approach
> > also.
> >
> >
> > a. If everyone is using a single "well known" logger to log a specific
> > event category then I lose the logger "context" of who's logging the
> > event.  As it stands now we're copying the logger name into a property we
> > call "eventSource".
> >
>
> A practice is to use one logger per class. Another is to use a higher-level
> logger to represent higher-level abstractions like a module.
>
>
> >
> >
> > b. You cannot turn on/off logging for a specific set of code.  If it
> turns
> > out that we have some errant code which is using this well known logger
> > then we can't just turn off that code from logging as turning off the
> well
> > know logger would turn it off for everyone using it.
> >
> >
> > I did look into the EventLogger and initially that seemed promising as I
> > guess it logs any event you give it at the "all" level.  However, as a
> well
> > known logger it suffers from the same issues above.
> >
> >
> > Now we're looking to add Business events.  My initial thinking is that I
> > can do the same thing we did before.  Add an additional custom level
> called
> > "Business" and expose methods for creating a business event.
>
>
> I would NOT create a custom level. Instead, I would use a Logger called
> "Business".
>
>
> > Though unlike the 

Re: approach for defining loggers

[Gary Gregory]

On Mon, Oct 17, 2016 at 4:27 PM, Nicholas Duane  wrote:

> Sorry to revive this old thread.  However, we're in the process of adding
> support for other "categories" of events and thus I wanted to first take a
> step back and try to ensure we're not convoluting things.
>
>
> There was a requirement to log a "compliance" event under certain
> conditions.  We did not want to write our own logging framework and instead
> decided to use existing off-the-shelf logging frameworks.  We have
> applications on both Linux/Java, Windows/.NET and Windows/Java.  Initially
> we chose log4net for Windows/.NET and log4j2 for Windows/Java and
> Linux/Java.  For these logging frameworks we wrote artifacts, appenders
> basically, to help facilitate getting these events to our system.  By the
> way, our system will get the events centrally, possibly put them into a
> relational database and also hand them off to another system which will get
> them eventually to an HDFS backend.  We also exposed methods for creating
> this compliance event.  The compliance event is basically a map.  We chose
> a map so that the event could also be extended by the application team in
> case they needed to add additional properties which made sense for them.
>
>
> We chose to create a custom level for this "compliance" event such that we
> could filter out only these events and get them into our system.  The
> configuration example we created had our custom unix domain socket appender
> added to the root logger.  It also contained a filter which filtered out
> any events that weren't compliance events.  The level we chose for
> "compliance" was less critical than off and more critical than fatal as we
> wanted to ensure that as long as logging wasn't turned off altogether our
> events would get logged.
>
>
> I want to go over a few suggestions that were made and explain why I
> didn't make use of those suggestions.
>
>
> 1. Since our "compliance" level didn't fit within the "vernacular" of the
> existing levels we should not define this custom level.  Instead we should
> look at using markers.
>

Yes, this is a use case for markers. The level should be used to note how
important is each compliance event.


>
> I am not that familiar with markers but did look into them when they were
> suggested.  While I don't have anything against markers in general there
> were some downsides as I saw it.
>
>
> a. Markers are not available, as far as I know, in log4net so we'd still
> have to figure out something there.
>

Indeed, we really need a port of Log4j 2 to .NET.


>
> b. A bigger problem, at least I thought it was a bigger problem, was that
> there would be confusion about what level to log the event at.  I would
> certainly not want to give an example as follows:
>
>
> logger.debug(COMPLIANCE_MARKER, evnt);
>
>
> or
>
>
> logger.info(COMPLIANCE_MARKER, evnt);
>
>
> or
>
>
> logger.error(COMPLIANCE_MARKER, evnt);
>
>
> ...
>

Think about: How important is this event? Are there different level of
importance to the audience?


>
>
> That just screams confusion to me.
>
>
> 2. Use a dedicated logger to log all compliance events.
>
>
> There were, as far as I could tell, a couple problems with this approach
> also.
>
>
> a. If everyone is using a single "well known" logger to log a specific
> event category then I lose the logger "context" of who's logging the
> event.  As it stands now we're copying the logger name into a property we
> call "eventSource".
>

A practice is to use one logger per class. Another is to use a higher-level
logger to represent higher-level abstractions like a module.


>
>
> b. You cannot turn on/off logging for a specific set of code.  If it turns
> out that we have some errant code which is using this well known logger
> then we can't just turn off that code from logging as turning off the well
> know logger would turn it off for everyone using it.
>
>
> I did look into the EventLogger and initially that seemed promising as I
> guess it logs any event you give it at the "all" level.  However, as a well
> known logger it suffers from the same issues above.
>
>
> Now we're looking to add Business events.  My initial thinking is that I
> can do the same thing we did before.  Add an additional custom level called
> "Business" and expose methods for creating a business event.


I would NOT create a custom level. Instead, I would use a Logger called
"Business".


> Though unlike the compliance event, the application teams would be
> defining the schema more so than our framework team.  Thus any method we
> expose would just be used as a starting point for setting the common
> properties.  You would use another instance of our unix domain socket
> appender for these business events and forward them to a different location
> as business events would most likely have a different retention period than
> compliance events.  Plus you might also want them in a different store as
> you may never need to query for both categories of events

Re: porting log4j2 to .NET

[Nicholas Duane]

Thanks for the guess.  I assume there is enough "separate" big enough pieces 
that a small group of people could work on different sections without stepping 
on each others toes such that you'd get some linear scaling.  For instance, it 
might take 3 people 2 months?  Even if we double your estimate, 3 people taking 
4 months?  Doesn't sound too bad.


Thanks,

Nick


From: Gary Gregory 
Sent: Monday, October 17, 2016 8:54 PM
To: Log4J Users List
Subject: Re: porting log4j2 to .NET

Random guesstimate for a complete port, including tests (what about docs):
6 man-months. You can shorten things up by reducing appenders and
configuration formats.

Gary

On Mon, Oct 17, 2016 at 5:33 PM, Nicholas Duane  wrote:

> Figured I would send this question out to the log4j side.  I have already
> had some email exchanges with the log4net mailing list regarding porting
> log4j2 to .NET.  My suggestion was that the apache logging framework be a
> single architecture design which is platform agnostic and then teams which
> port to the different platforms.  It seems log4net was a port of log4j and
> may be going off in its own direction from that initial port.  My viewpoint
> is that's a bad idea as one of the benefits I saw was that log4net was
> similar to log4j2 and we're looking for logging frameworks for our
> enterprise.  We have applications on both Windows/.NET and Linux/Java so
> having a logging framework for Windows/.NET which is similar to a logging
> framework for Linux/Java was a big plus.
>
>
> While I have no doubt the effort to port log4j2 to .NET is considerable,
> it would be a port and thus I'm not spending time figuring out design and
> algorithms.  Would anyone want to venture a guess at what that effort might
> be?
>
>
> Thanks,
>
> Nick
>



--
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition



JUnit in Action, Second Edition

JUnit in Action, Second Edition: Petar Tahchiev, Felipe Leme, Vincent Massol, 
Gary Gregory: 9781935182023: Amazon.com: 
Books
www.amazon.com
JUnit in Action, Second Edition [Petar Tahchiev, Felipe Leme, Vincent Massol, 
Gary Gregory] on Amazon.com. *FREE* shipping on qualifying offers.
When JUnit was first introduced a decade ago by Kent Beck and Erich Gamma, the 
Agile movement was in its infancy




Spring Batch in Action

Spring Batch in Action: Arnaud Cogoluegnes, Thierry Templier, Gary Gregory, 
Olivier Bazoud: 9781935182955: Amazon.com: 
Books
www.amazon.com
Spring Batch in Action [Arnaud Cogoluegnes, Thierry Templier, Gary Gregory, 
Olivier Bazoud] on Amazon.com. *FREE* shipping on qualifying offers.
Summary Spring Batch in Action is an in-depth guide to writing batch 
applications using Spring Batch. Written for developers who have basic 
knowledge of Java and the Spring lightweight container



Blog: http://garygregory.wordpress.com
[https://s0.wp.com/i/blank.jpg]

Gary Gregory
garygregory.wordpress.com
Software construction, the web, and other techs


Home: http://garygregory.com/
Gary Gregory
garygregory.com
Rocket | Seagull . I am a Software Architect for Seagull Software, a division 
of Rocket Software. Rocket Seagull specializes in tools and expertise to 
modernize ...


Tweet! http://twitter.com/GaryGregory

Re: porting log4j2 to .NET

[Gary Gregory]

Random guesstimate for a complete port, including tests (what about docs):
6 man-months. You can shorten things up by reducing appenders and
configuration formats.

Gary

On Mon, Oct 17, 2016 at 5:33 PM, Nicholas Duane  wrote:

> Figured I would send this question out to the log4j side.  I have already
> had some email exchanges with the log4net mailing list regarding porting
> log4j2 to .NET.  My suggestion was that the apache logging framework be a
> single architecture design which is platform agnostic and then teams which
> port to the different platforms.  It seems log4net was a port of log4j and
> may be going off in its own direction from that initial port.  My viewpoint
> is that's a bad idea as one of the benefits I saw was that log4net was
> similar to log4j2 and we're looking for logging frameworks for our
> enterprise.  We have applications on both Windows/.NET and Linux/Java so
> having a logging framework for Windows/.NET which is similar to a logging
> framework for Linux/Java was a big plus.
>
>
> While I have no doubt the effort to port log4j2 to .NET is considerable,
> it would be a port and thus I'm not spending time figuring out design and
> algorithms.  Would anyone want to venture a guess at what that effort might
> be?
>
>
> Thanks,
>
> Nick
>



-- 
E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
Java Persistence with Hibernate, Second Edition



JUnit in Action, Second Edition



Spring Batch in Action


Blog: http://garygregory.wordpress.com
Home: http://garygregory.com/
Tweet! http://twitter.com/GaryGregory

porting log4j2 to .NET

[Nicholas Duane]

Figured I would send this question out to the log4j side.  I have already had 
some email exchanges with the log4net mailing list regarding porting log4j2 to 
.NET.  My suggestion was that the apache logging framework be a single 
architecture design which is platform agnostic and then teams which port to the 
different platforms.  It seems log4net was a port of log4j and may be going off 
in its own direction from that initial port.  My viewpoint is that's a bad idea 
as one of the benefits I saw was that log4net was similar to log4j2 and we're 
looking for logging frameworks for our enterprise.  We have applications on 
both Windows/.NET and Linux/Java so having a logging framework for Windows/.NET 
which is similar to a logging framework for Linux/Java was a big plus.


While I have no doubt the effort to port log4j2 to .NET is considerable, it 
would be a port and thus I'm not spending time figuring out design and 
algorithms.  Would anyone want to venture a guess at what that effort might be?


Thanks,

Nick

Re: approach for defining loggers

[Nicholas Duane]

Sorry to revive this old thread.  However, we're in the process of adding 
support for other "categories" of events and thus I wanted to first take a step 
back and try to ensure we're not convoluting things.


There was a requirement to log a "compliance" event under certain conditions.  
We did not want to write our own logging framework and instead decided to use 
existing off-the-shelf logging frameworks.  We have applications on both 
Linux/Java, Windows/.NET and Windows/Java.  Initially we chose log4net for 
Windows/.NET and log4j2 for Windows/Java and Linux/Java.  For these logging 
frameworks we wrote artifacts, appenders basically, to help facilitate getting 
these events to our system.  By the way, our system will get the events 
centrally, possibly put them into a relational database and also hand them off 
to another system which will get them eventually to an HDFS backend.  We also 
exposed methods for creating this compliance event.  The compliance event is 
basically a map.  We chose a map so that the event could also be extended by 
the application team in case they needed to add additional properties which 
made sense for them.


We chose to create a custom level for this "compliance" event such that we 
could filter out only these events and get them into our system.  The 
configuration example we created had our custom unix domain socket appender 
added to the root logger.  It also contained a filter which filtered out any 
events that weren't compliance events.  The level we chose for "compliance" was 
less critical than off and more critical than fatal as we wanted to ensure that 
as long as logging wasn't turned off altogether our events would get logged.


I want to go over a few suggestions that were made and explain why I didn't 
make use of those suggestions.


1. Since our "compliance" level didn't fit within the "vernacular" of the 
existing levels we should not define this custom level.  Instead we should look 
at using markers.


I am not that familiar with markers but did look into them when they were 
suggested.  While I don't have anything against markers in general there were 
some downsides as I saw it.


a. Markers are not available, as far as I know, in log4net so we'd still have 
to figure out something there.

b. A bigger problem, at least I thought it was a bigger problem, was that there 
would be confusion about what level to log the event at.  I would certainly not 
want to give an example as follows:


logger.debug(COMPLIANCE_MARKER, evnt);


or


logger.info(COMPLIANCE_MARKER, evnt);


or


logger.error(COMPLIANCE_MARKER, evnt);


...


That just screams confusion to me.


2. Use a dedicated logger to log all compliance events.


There were, as far as I could tell, a couple problems with this approach also.


a. If everyone is using a single "well known" logger to log a specific event 
category then I lose the logger "context" of who's logging the event.  As it 
stands now we're copying the logger name into a property we call "eventSource".


b. You cannot turn on/off logging for a specific set of code.  If it turns out 
that we have some errant code which is using this well known logger then we 
can't just turn off that code from logging as turning off the well know logger 
would turn it off for everyone using it.


I did look into the EventLogger and initially that seemed promising as I guess 
it logs any event you give it at the "all" level.  However, as a well known 
logger it suffers from the same issues above.


Now we're looking to add Business events.  My initial thinking is that I can do 
the same thing we did before.  Add an additional custom level called "Business" 
and expose methods for creating a business event.  Though unlike the compliance 
event, the application teams would be defining the schema more so than our 
framework team.  Thus any method we expose would just be used as a starting 
point for setting the common properties.  You would use another instance of our 
unix domain socket appender for these business events and forward them to a 
different location as business events would most likely have a different 
retention period than compliance events.  Plus you might also want them in a 
different store as you may never need to query for both categories of events 
and thus no need to query against a larger set of data.


In addition we're planning to capture centrally what we refer to as diagnostic 
events: error, info, warn, debug, trace, etc.  However, we may need to separate 
these out into two different categories: critical-diagnostic and 
noncritical-diagnostic.  The reason is that we don't want the potential of a 
critical diagnostic event, let's say an error, queued up behind potentially 
thousands of non-critical diagnostic events.  So you see, the category also 
defines aspects on how we handle events at the source.   We separate at the 
source based on category as it seems a reasonable place to do so.  Also, you 
may want differen

Re: Log4j2 Kafka appender NullPointerException when put the related jars in tomcat lib

[Mikael Ståldal]

Ralph, are you going to work on this issue?

On Mon, Oct 17, 2016 at 2:04 PM, Mikael Ståldal 
wrote:

> So this issue is triggered by the fact that the Kafka client library we
> use itself does logging via SLF4J during initialization. And in this
> particular web configuration, org.slf4j.LoggerFactory.getLogger(Class)
> return null during Configuration.start().
>
> On Sat, Aug 20, 2016 at 12:13 AM, Bill Okara  wrote:
>
>> Ralph,
>>
>> thanks! created jira:
>> https://issues.apache.org/jira/browse/LOG4J2-1535
>>
>> I guess I still wanted the ClassLoaderContextSelector, as I wanted
>> each webapp to have it's own config (log to different target etc), but
>> just shared same jars. so the intention is to have all log related
>> jars in tomcat/lib, and have log4j2.xml in WEB-INF/classes.
>>
>> Another reason to have log related jars in tomcat/lib is to have some
>> common jars to log using log4j2 too...
>>
>> Thanks,
>> Bill
>>
>>
>> On Fri, Aug 19, 2016 at 3:17 PM, Ralph Goers 
>> wrote:
>> > So they are being kicked off by the webapp initialization. That follows
>> what I was thinking. I have a suspicion I may have to put in some code to
>> check if the logging classes are in the same ClassLoader as the current
>> ClassLoader. If they aren’t we probably need to kick off logging
>> initialization there first and then continue with the current
>> initialization. I need to look at the code again and try a few things
>> though.
>> >
>> > Is it possible you could create a Jira for this and attach a sample
>> project that demonstrates the problem?
>> >
>> > Another workaround to this is to not use the
>> ClassLoaderContextSelector. If you really want a common configuration
>> across all your web apps then you can use the BasicContextSelector.
>> However, that may cause problems if you try to use hot deployment.
>> >
>> > Ralph
>> >
>> >
>> >> On Aug 19, 2016, at 1:52 PM, Bill Okara  wrote:
>> >>
>> >> Hi Gary,
>> >>
>> >> actually the full stack was attached in that email already, please let
>> >> me know if you can't find them...
>> >> (Attachments: stacktrace_webinf.txt (5 kb)   stacktrace_tomcatlib.txt
>> (7 kb) )
>> >>
>> >> Yeah, that's kinda my understanding (or guess) too. That is, by
>> >> putting the log related jars in tomcat/lib, when the initializing the
>> >> log4j2.xml, it's trying to initialize the KafkaAppender, whose jars
>> >> are in tomcat/lib, thus it's trying to initialize the log4j config for
>> >> that context.
>> >>
>> >> I was able to hack/workaround the issue by adding a log4j2-test.xml in
>> >> WEB-INF/classes, and not using the KafkaAppender in the
>> >> log4j2-test.xml. In that case, when initializing for tomcat/lib, the
>> >> log4j2-test.xml got picked up before the log4j2.xml, and that kinda
>> >> avoided the problem.
>> >>
>> >> But the caveat is that no longer can use KafkaAppender for anything
>> >> logged by jars in tomcat/lib...
>> >>
>> >> I guess KafkaAppender is a special case that it's using a jar that
>> >> also uses SLF4j/log4j logs.
>> >>
>> >> By having the jars in tomcat/lib seems like pretty clean for webapps
>> >> to share same logging implementation and use their specific config.
>> >> but not sure how to better workaround / resolve the issue without
>> >> limiting all jars in tomcat/lib to not using KafkaAppender in this
>> >> case.
>> >>
>> >> comments?
>> >>
>> >>
>> >> Thanks!
>> >> Bill
>> >>
>> >>
>> >> On Fri, Aug 19, 2016 at 1:07 PM, Ralph Goers <
>> ralph.go...@dslextreme.com > wrote:
>> >>> I have a suspicion the key part of the stack trace isn’t included.  I
>> am guessing that the code that causes the first call to the Kafka appender
>> is in your web app.  This causes the ClassLoaderContextSelector to
>> initialize Log4j for the web app ClassLoader. Kafka itself is in
>> tomcat/lib. When it logs it causes Log4j to initialize for the tomcat/lib
>> ClassLoader.  That is why you see the second LoggerContext start.
>> >>>
>> >>> However, I would really like to see the stack frames just below where
>> you stopped to see what is causing the initial Log4j initialization.
>> >>>
>> >>> Ralph
>> >>>
>> >>>
>> >>>
>>  On Aug 19, 2016, at 11:17 AM, Bill Okara > > wrote:
>> 
>>  thanks for guiding me through this, Gary!
>> 
>>  Attached are 2 stack dumps, one is with the log related jars packaged
>>  in WEB-INF/lib, the other is having the jars provided in tomcat/lib,
>>  and below are what I noticed; that is,
>>  in WEB-INF/lib case, when initializing the
>>  private static final Logger log = LoggerFactory.getLogger(KafkaP
>> roducer.class);
>> 
>>  the getLogger seemed returned a logger right the way, vs in
>> tomcat/lib
>>  case, it's trying to go through the getContext etc init stack, which
>>  in turn hit the KafkaProducer.java:188 again and had the NULL log at
>>  that case...
>> 
>>  comments?
>> 
>>  In WEB-INF/lib:
>>  at

Re: Log4j2 Kafka appender NullPointerException when put the related jars in tomcat lib

[Mikael Ståldal]

So this issue is triggered by the fact that the Kafka client library we use
itself does logging via SLF4J during initialization. And in this particular
web configuration, org.slf4j.LoggerFactory.getLogger(Class) return
null during Configuration.start().

On Sat, Aug 20, 2016 at 12:13 AM, Bill Okara  wrote:

> Ralph,
>
> thanks! created jira:
> https://issues.apache.org/jira/browse/LOG4J2-1535
>
> I guess I still wanted the ClassLoaderContextSelector, as I wanted
> each webapp to have it's own config (log to different target etc), but
> just shared same jars. so the intention is to have all log related
> jars in tomcat/lib, and have log4j2.xml in WEB-INF/classes.
>
> Another reason to have log related jars in tomcat/lib is to have some
> common jars to log using log4j2 too...
>
> Thanks,
> Bill
>
>
> On Fri, Aug 19, 2016 at 3:17 PM, Ralph Goers 
> wrote:
> > So they are being kicked off by the webapp initialization. That follows
> what I was thinking. I have a suspicion I may have to put in some code to
> check if the logging classes are in the same ClassLoader as the current
> ClassLoader. If they aren’t we probably need to kick off logging
> initialization there first and then continue with the current
> initialization. I need to look at the code again and try a few things
> though.
> >
> > Is it possible you could create a Jira for this and attach a sample
> project that demonstrates the problem?
> >
> > Another workaround to this is to not use the ClassLoaderContextSelector.
> If you really want a common configuration across all your web apps then you
> can use the BasicContextSelector. However, that may cause problems if you
> try to use hot deployment.
> >
> > Ralph
> >
> >
> >> On Aug 19, 2016, at 1:52 PM, Bill Okara  wrote:
> >>
> >> Hi Gary,
> >>
> >> actually the full stack was attached in that email already, please let
> >> me know if you can't find them...
> >> (Attachments: stacktrace_webinf.txt (5 kb)   stacktrace_tomcatlib.txt
> (7 kb) )
> >>
> >> Yeah, that's kinda my understanding (or guess) too. That is, by
> >> putting the log related jars in tomcat/lib, when the initializing the
> >> log4j2.xml, it's trying to initialize the KafkaAppender, whose jars
> >> are in tomcat/lib, thus it's trying to initialize the log4j config for
> >> that context.
> >>
> >> I was able to hack/workaround the issue by adding a log4j2-test.xml in
> >> WEB-INF/classes, and not using the KafkaAppender in the
> >> log4j2-test.xml. In that case, when initializing for tomcat/lib, the
> >> log4j2-test.xml got picked up before the log4j2.xml, and that kinda
> >> avoided the problem.
> >>
> >> But the caveat is that no longer can use KafkaAppender for anything
> >> logged by jars in tomcat/lib...
> >>
> >> I guess KafkaAppender is a special case that it's using a jar that
> >> also uses SLF4j/log4j logs.
> >>
> >> By having the jars in tomcat/lib seems like pretty clean for webapps
> >> to share same logging implementation and use their specific config.
> >> but not sure how to better workaround / resolve the issue without
> >> limiting all jars in tomcat/lib to not using KafkaAppender in this
> >> case.
> >>
> >> comments?
> >>
> >>
> >> Thanks!
> >> Bill
> >>
> >>
> >> On Fri, Aug 19, 2016 at 1:07 PM, Ralph Goers <
> ralph.go...@dslextreme.com > wrote:
> >>> I have a suspicion the key part of the stack trace isn’t included.  I
> am guessing that the code that causes the first call to the Kafka appender
> is in your web app.  This causes the ClassLoaderContextSelector to
> initialize Log4j for the web app ClassLoader. Kafka itself is in
> tomcat/lib. When it logs it causes Log4j to initialize for the tomcat/lib
> ClassLoader.  That is why you see the second LoggerContext start.
> >>>
> >>> However, I would really like to see the stack frames just below where
> you stopped to see what is causing the initial Log4j initialization.
> >>>
> >>> Ralph
> >>>
> >>>
> >>>
>  On Aug 19, 2016, at 11:17 AM, Bill Okara  > wrote:
> 
>  thanks for guiding me through this, Gary!
> 
>  Attached are 2 stack dumps, one is with the log related jars packaged
>  in WEB-INF/lib, the other is having the jars provided in tomcat/lib,
>  and below are what I noticed; that is,
>  in WEB-INF/lib case, when initializing the
>  private static final Logger log = LoggerFactory.getLogger(
> KafkaProducer.class);
> 
>  the getLogger seemed returned a logger right the way, vs in tomcat/lib
>  case, it's trying to go through the getContext etc init stack, which
>  in turn hit the KafkaProducer.java:188 again and had the NULL log at
>  that case...
> 
>  comments?
> 
>  In WEB-INF/lib:
>  at org.apache.kafka.common.config.ConfigDef.define(
> ConfigDef.java:346)
>  at org.apache.kafka.clients.producer.ProducerConfig.<
> clinit>(ProducerConfig.java:220)
>  at org.apache.kafka.clients.producer.KafkaProducer.(
> Ka