Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Christopher Schultz
Ronald, On 2/13/17 2:29 PM, DiFrango, Ronald wrote: > Yeah, I’m aware and agreed, but this is done in the Apache HTTP Client > Component’s Code, not ours and existed as-is with log4j2 in ALL the versions > we’ve been using. > > That said, I think this is a red herring and it’s still true that

RE: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
They were all tied to log4j2 itself, though this seemed to primarily occur with 2.6.2 and went away in 2.7 Ron DiFrango From: Christopher Schultz Sent: Monday, February 13, 2017 3:12:34 PM To: Log4J Users List Subject: Re: CPU

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Apache
The only threads in Log4j that I can think of that are short lived are the threads that compress the files at rollover. That would only consume a lot of CPU if rollover is happening a lot. Ralph > On Feb 13, 2017, at 1:12 PM, Christopher Schultz > wrote: > >

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Apache
If you are seeing lots of threads can you do a jstack on the process so we can see what they are? Ralph > On Feb 13, 2017, at 3:50 PM, DiFrango, Ronald > wrote: > > Our policy generally looks like the following where max log files is 2 and > size is 10MB and

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Remko Popma
A value of max=2 means that the RolloverAppender ensures no more than 2 rolled over log files are kept. But all files in the directory are inspected. If the directory contains many files, the scan can take a while. An alternative is the action.

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Remko Popma
I've seen cases where the "max" in caused issues because the directory contained many files that all needed to be scanned to see if they matched old log files to delete. That said, it would be good if you can share the results of jstack and do a run with Java Flight Recorder. Sent from my

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Apache
With a max of 2 I wouldn’t expect any problems. It does mean that you are only going to keep the last 2 files each day, and the last one might not be full as it might not be full when it rolls over at midnight. If maxFiles was 100 and you had more than 100 files per day then the time it would

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Remko Popma
Ron, We haven't heard of any issues like you describe. Have you tried running your application with Java Flight Recorder ? This should help diagnose what is going on. Remko On Mon, Feb 13, 2017 at 9:59 PM,

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Matt Sicker
What server environment are you running this in? On 13 February 2017 at 09:19, Remko Popma wrote: > Ron, > > We haven't heard of any issues like you describe. > Have you tried running your application with Java Flight Recorder >

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
This is running in Tomcat 8.0.33 in a Docker Container via AWS’s ECS which is identical to before with log4j2 2.5. We’ve run the application with Visual VM and the one thing for sure that we’ve seen is that in 2.6.2 it created tons of threads, something like 50+ but on 2.6 or 2.7 it was only

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Matt Sicker
Are you logging that info? On 13 February 2017 at 11:46, DiFrango, Ronald < ronald.difra...@capitalone.com> wrote: > All, > > One thing we just noticed is that we are using Apache’s HTTP connection > pooling for our downstream calls and in our latest performance run is that > the READ operation

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
No we are not and since it’s a log.debug() it should just get dropped on the floor before any sort of copying occurs, right? Ron DiFrango On 2/13/17, 12:50 PM, "Matt Sicker" wrote: Are you logging that info? On 13 February 2017 at 11:46, DiFrango, Ronald <

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
All, One thing we just noticed is that we are using Apache’s HTTP connection pooling for our downstream calls and in our latest performance run is that the READ operation on org.apache.http.impl.conn.LoggingInputStream seems to be taking a bulk of the time. If I look at the code, once the

Re: Broken downloads

2017-02-13 Thread Mikael Ståldal
The .zip and .tar.gz links on https://logging.apache.org/log4j/2.x/download.html actually goes to another HTML pages with the actual download links on. Maybe the URLs to the mirrors page should not have .zip or .tar.gz extension to avoid confusion? On Mon, Feb 13, 2017 at 11:39 AM, Aron

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
Ralph, Yeah, I’m aware and agreed, but this is done in the Apache HTTP Client Component’s Code, not ours and existed as-is with log4j2 in ALL the versions we’ve been using. That said, I think this is a red herring and it’s still true that at least for our application any version of log4j2

Re: CPU Utilization: log4j 2.6 and great

2017-02-13 Thread Apache
The log.debug call will exit quickly, but if it is constructing a large message that will happen anyway unless one of the standard techniques (like wrapping in a logger.isDebugEnabled call) is used to avoid it. Constructing debug Strings can be expensive. Ralph > On Feb 13, 2017, at 10:58 AM,

CPU Utilization: log4j 2.6 and great

2017-02-13 Thread DiFrango, Ronald
All, We recently upgrade to 2.6 and noticed a dramatic increase in CPU and Thread utilization that seems to be tied to the new “garbage free” mode of log4j 2.6. Here’s some of the baseline numbers: · Log4j 2.5: CPU typically ran around 25% · Log4j 2.6: CPU typically ran