Re: Log entries written to wrong file

[Remko Popma]

What happens if you don't disable the "Use an isolated class loader for
this shared library" option? Is there an error message?

To go back to your original question:
it should be possible to have the log4j jars in the web container's shared
lib and let each web app have its own log4j2.xml config file in
WEB-INF/classes. If you are seeing log entries written to the wrong file
this may be a bug. Is this easy to reproduce?

Unfortunately there are currently a number of memory leaks in versions
2.4.1 and earlier. I have addressed these and the fixes will be in the next
2.5 release. Any help identifying remaining issues (like the issue you
describe) would be great.




On Thu, Nov 12, 2015 at 4:10 AM, Chinh Do  wrote:

> I got log separation to work by copying the log4j-core*.jar and
> log4j-api-*.jar to the WAR's lib folder. We are using a Shared Library and
> I also had to disable the option "Use an isolated class loader for this
> shared library" for it to work.
>
> However this is not ideal as disabling "Use an isolated class loader" can
> use up more memory unnecessarily. So I am still trying to get the "Using
> Context Selectors" approach, specifically this part from the manual:
>
> Follow the instructions to initialize Log4j 2 in a web application
>  and set the
> system
> property or servlet context parameter Log4jContextSelector to
> org.apache.logging.log4j.core.selector.JndiContextSelector. This will cause
> the container to use JNDI to locate each web application's LoggerContext.
> Be sure to set the isLog4jContextSelectorNamed context parameter to true
> and also set the log4jContextName and log4jConfiguration context
> parameters.
>
> I am a little lost on exactly what to do though. I tried to define a JVM
> Custom Property named Log4jContextSelector and set it to
> org.apache.logging.log4j.core.selector.JndiContextSelector,
> just to see what happens. After I restarted WAS, this error showed up in
> SystemErr.log: SystemErr R ERROR StatusLogger Unable to lookup
> java:comp/env/log4j/context-name javax.naming.NameNotFoundException: Name
> comp/env/log4j not found in context "java:". So I guess I need to set
> isLog4jContextSelectorNamed, log4jContextName, and log4jConfiguration
> context parameters. But how exactly do you do that for a SpringBoot app
> running in WebSphere 8.5.5?
>
>
> In the process I also upgraded to log4j 2.4.1 to see if it helps with this.
> Don't think the new version makes a difference.
>
> I've been on this for more than 1 week now so any help is greatly
> appreciated.
>
> Chinh
>
>
> On Tue, Nov 10, 2015 at 10:10 AM, Chinh Do  wrote:
>
> > The manual (http://logging.apache.org/log4j/2.x/manual/logsep.html) says
> > you can follow step (3) below to get log separation. I am not clear on
> what
> > values to set for log4jContextName and log4jConfiguration, or how exactly
> > to do it on WebSphere 8.5 Has anyone done this? Really appreciate any
> help
> > in advance.
> > Using Context Selectors
> >
> > There are a few patterns for achieving the desired state of logging
> > separation using ContextSelectors:
> >
> >1. Place the logging jars in the container's classpath and set the
> >system property "Log4jContextSelector" to
> >"org.apache.logging.log4j.core.selector.BasicContextSelector". This
> will
> >create a single LoggerContext using a single configuration that will
> be
> >shared across all applications.
> >2. Place the logging jars in the container's classpath and use the
> >default ClassLoaderContextSelector. Follow the instructions to
> >initialize Log4j 2 in a web application
> >. Each
> >application can be configured to share the same configuration used at
> the
> >container or can be individually configured. If status logging is set
> to
> >debug in the configuration there will be output from when logging is
> >initialized in the container and then again in each web application.
> >3. Follow the instructions to initialize Log4j 2 in a web application
> > and set the
> >system property or servlet context parameter Log4jContextSelector to
> >org.apache.logging.log4j.core.selector.JndiContextSelector. This will
> >cause the container to use JNDI to locate each web application's
> >LoggerContext. Be sure to set the isLog4jContextSelectorNamed context
> >parameter to true and also set the log4jContextName and
> >log4jConfiguration context parameters.
> >
> > The exact method for setting system properties depends on the container.
> > For Tomcat, edit $CATALINA_HOME/conf/catalina.properties. Consult the
> > documentation for other web containers.
> >
> > More info:
> > - log4j 2.1 with slf4j 1.7.12
> > - Not using Async Loggers right now (planning to use it later)
> > - WebSphere 8.5
> >

Re: Log entries written to wrong file

[Chinh Do]

I got log separation to work by copying the log4j-core*.jar and
log4j-api-*.jar to the WAR's lib folder. We are using a Shared Library and
I also had to disable the option "Use an isolated class loader for this
shared library" for it to work.

However this is not ideal as disabling "Use an isolated class loader" can
use up more memory unnecessarily. So I am still trying to get the "Using
Context Selectors" approach, specifically this part from the manual:

Follow the instructions to initialize Log4j 2 in a web application
 and set the system
property or servlet context parameter Log4jContextSelector to
org.apache.logging.log4j.core.selector.JndiContextSelector. This will cause
the container to use JNDI to locate each web application's LoggerContext.
Be sure to set the isLog4jContextSelectorNamed context parameter to true
and also set the log4jContextName and log4jConfiguration context
parameters.

I am a little lost on exactly what to do though. I tried to define a JVM
Custom Property named Log4jContextSelector and set it to
org.apache.logging.log4j.core.selector.JndiContextSelector,
just to see what happens. After I restarted WAS, this error showed up in
SystemErr.log: SystemErr R ERROR StatusLogger Unable to lookup
java:comp/env/log4j/context-name javax.naming.NameNotFoundException: Name
comp/env/log4j not found in context "java:". So I guess I need to set
isLog4jContextSelectorNamed, log4jContextName, and log4jConfiguration
context parameters. But how exactly do you do that for a SpringBoot app
running in WebSphere 8.5.5?


In the process I also upgraded to log4j 2.4.1 to see if it helps with this.
Don't think the new version makes a difference.

I've been on this for more than 1 week now so any help is greatly
appreciated.

Chinh


On Tue, Nov 10, 2015 at 10:10 AM, Chinh Do  wrote:

> The manual (http://logging.apache.org/log4j/2.x/manual/logsep.html) says
> you can follow step (3) below to get log separation. I am not clear on what
> values to set for log4jContextName and log4jConfiguration, or how exactly
> to do it on WebSphere 8.5 Has anyone done this? Really appreciate any help
> in advance.
> Using Context Selectors
>
> There are a few patterns for achieving the desired state of logging
> separation using ContextSelectors:
>
>1. Place the logging jars in the container's classpath and set the
>system property "Log4jContextSelector" to
>"org.apache.logging.log4j.core.selector.BasicContextSelector". This will
>create a single LoggerContext using a single configuration that will be
>shared across all applications.
>2. Place the logging jars in the container's classpath and use the
>default ClassLoaderContextSelector. Follow the instructions to
>initialize Log4j 2 in a web application
>. Each
>application can be configured to share the same configuration used at the
>container or can be individually configured. If status logging is set to
>debug in the configuration there will be output from when logging is
>initialized in the container and then again in each web application.
>3. Follow the instructions to initialize Log4j 2 in a web application
> and set the
>system property or servlet context parameter Log4jContextSelector to
>org.apache.logging.log4j.core.selector.JndiContextSelector. This will
>cause the container to use JNDI to locate each web application's
>LoggerContext. Be sure to set the isLog4jContextSelectorNamed context
>parameter to true and also set the log4jContextName and
>log4jConfiguration context parameters.
>
> The exact method for setting system properties depends on the container.
> For Tomcat, edit $CATALINA_HOME/conf/catalina.properties. Consult the
> documentation for other web containers.
>
> More info:
> - log4j 2.1 with slf4j 1.7.12
> - Not using Async Loggers right now (planning to use it later)
> - WebSphere 8.5
>
> On Mon, Nov 9, 2015 at 11:15 AM, Chinh Do  wrote:
>
>> The manual (http://logging.apache.org/log4j/2.x/manual/logsep.html) says
>> you can follow step (3) below to get log separation. I am not clear on what
>> values to set for log4jContextName and log4jConfiguration. Has anyone done
>> this?
>> Using Context Selectors
>>
>> There are a few patterns for achieving the desired state of logging
>> separation using ContextSelectors:
>>
>>1. Place the logging jars in the container's classpath and set the
>>system property "Log4jContextSelector" to
>>"org.apache.logging.log4j.core.selector.BasicContextSelector". This will
>>create a single LoggerContext using a single configuration that will be
>>shared across all applications.
>>2. Place the logging jars in the container's classpath and use the
>>default ClassLoaderContextSelector. Follow the instructions 

Re: RollingFileAppender maxDelayMillis?

[Benjamin Jaton]

Thanks a lot for the explanation, much clearer now.

Here is the JIRA: https://issues.apache.org/jira/browse/LOG4J2-1197

On Tue, Nov 10, 2015 at 11:39 PM, Remko Popma  wrote:

> On Wed, Nov 11, 2015 at 2:34 PM, Benjamin Jaton 
> wrote:
>
> > On Tue, Nov 10, 2015 at 6:13 PM, Remko Popma 
> > wrote:
> >
> > > To check my understanding: the problem is that with bufferedIO=true the
> > > buffer contains log events that you want to see, but the buffer is only
> > > flushed to disk when it is full (which is not frequent enough); is that
> > > correct?
> > >
> >
> > Yes that is correct.
> > Typical scenario is when the server is not in production, so almost no
> > traffic except for the request manually triggered by the operator,
> usually
> > in order to debug something.
> >
> >
> > > If you use AsyncAppender or Async Loggers, the buffer will be flushed
> > when
> > > the queue is empty. This results in efficient IO (lumping together IO
> > > operations where possible), but at the same time your log file always
> > > reflects the most recent message.
> > >
> >
> > So I have a dumb question. What's the difference between bufferedIO and
> > AsyncAppender?
> > Isn't bufferedIO a kind of asynchronous treatment of the log messages?
> I'm
> > sorry I'm a bit confused.
> >
>
> No problem at all. BufferedIO is not asynchronous. The difference between
> bufferedIO and Async Appenders/Loggers is that the async stuff deals with a
> queue of log event objects and happens further upstream, where bufferedIO
> deals with a byte[] array buffer and happens further downstream.
>
> When you use Async Appenders/Loggers, each time your application logs
> something a Log Event object is added to a queue. A background thread takes
> these events off the queue, asks the Layout to turn the event into a byte[]
> array and uses the Appender to send these bytes to their destination.
>
> Without Async Appenders/Loggers, there is no queue, and this work happens
> in your application thread (the Layout turns the event into a byte[] array
> and the Appender sends these bytes to their destination).
>
> When the configuration has bufferedIO="true", the Appender will save up the
> byte[] arrays for multiple log events together in an in-memory buffer first
> instead of writing each byte[] array to its destination immediately,
> to reduce the number of expensive I/O operations. As you noticed, this
> sometimes results in data sitting in this in-memory buffer for a long time.
>
> Now, with Async Appenders/Loggers, we can use what Martin Thompson calls
> "smart
> batching
> ":
> when
> the background thread processes a log event, it checks if the queue is now
> empty (no further log events are in the pipeline to be processed) and if it
> is, it will also call Appender.flush() in addition to the above.
>
>
>
> > >
> > > For synchronous buffered logging, I can see how a smarter flushing
> > > mechanism would be useful.
> > >
> > > Question is, what would trigger the check (whether the buffer needs to
> be
> > > flushed)?
> > >
> > > Easiest is to check with each log event, but if no new event arrives
> you
> > > still end up with a stale buffer. Better than nothing, but not great.
> > >
> > > An alternative is to check periodically from a background thread and
> > flush
> > > if the buffer is non-empty. This probably gives the best results but is
> > > fairly complex to implement.
> > >
> >
> > That was what I had in mind but I don't know the complexity.
> >
> >
> > >
> > > I need to think about this a bit more, but for now I would suggest
> using
> > > async logging. It may be a good idea to create a Jira ticket for this.
> > >
> >
> > OK I will create a Jira for this.
> >
> >
> > >
> > > Remko
> > >
> > > > On 2015/11/11, at 5:16, Gary Gregory  wrote:
> > > >
> > > > And more generally, can all appenders benefit from such a setting?
> > > >
> > > > Gary
> > > >
> > > > On Tue, Nov 10, 2015 at 11:04 AM, Benjamin Jaton <
> > > benjamin.ja...@gmail.com>
> > > > wrote:
> > > >
> > > >> Hi,
> > > >>
> > > >> I am wondering if it would make sense to have a maxDelayMillis
> > parameter
> > > >> for the RollingFileAppender (like FlumeAppender).
> > > >>
> > > >> Use case:
> > > >> You have a web server access log, you produce an HTTP request and
> > > expect to
> > > >> see it (and the response) in the access logs right away-ish (or at
> > least
> > > >> control how much you have to wait).
> > > >>
> > > >> Thoughts?
> > > >> Ben
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > E-Mail: garydgreg...@gmail.com | ggreg...@apache.org
> > > > Java Persistence with Hibernate, Second Edition
> > > > 
> > > > JUnit in Action, Second Edition 
> > > > Spring Batch in Action 
> > > > Blog: