Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I do, but since our code was blatantly copied from Quartz I am not sure how easy it will be to do. If this support is added it should also be submitted to Quartz. I would look and see how Jenkins. Jenkins seems to be under the MIT license so that should be no problem. Ralph > On Mar 27,

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

That's probably why I wasn't able to figure out how you can do this with CronTrigerringPolicy :) Do you think that adding this random support in cron expression is a better solution ? Regards, Anthony 2017-03-24 18:47 GMT+01:00 Ralph Goers : > I thought

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I thought CronTriggeringPolicy would handle this but it is based on Quartz’ CronExpression which doesn’t support randomizing fields. I was confused because Jenkins supports this and I had thought it was using Quartz. Apparently not. Ralph > On Mar 24, 2017, at 10:15 AM, Anthony Maire

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Basically we are hosting distributed systems for our clients. So we have several dozens of JVM per physical host, and several dozens of physical hosts. Moreover some clients are sometimes moved from one machine to another so having a single configuration file is a very huge requirement. Having

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Actually, there is already something like this because Log4j2 configuration has some support for scripting. I would prefer to enhance that over building conditional constructs in the configuration. However, having to specify the exact rollover time for individual servers would result in a much

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

The problem is that they want the files to roll over exactly at midnight so that they contain that day’s log files. They just don’t want the compression to happen all at the same time. If that wasn’t the requirement the existing CronTriggeringPolicy would already solve the problem. Ralph >

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

What if a configuration could contain conditional statements? For instance: Cheers On 2017-03-24 11:08, Remko Popma wrote: I

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I see what you are saying, but the use case is to have a single configuration, that is what drives the request. Sent from my iPhone > On Mar 24, 2017, at 17:22, Dominik Psenner wrote: > > Hi there, > > Cron proved itself very to be stable and usable over the past years.

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Hi there, Cron proved itself very to be stable and usable over the past years. FWIW, I would not recommend to introduce a randomization algorithm. An application that does things random means that the application does things when nobody expects it to do so. Further it does not solve the

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

https://issues.apache.org/jira/browse/LOG4J2-1855 https://github.com/apache/logging-log4j2/pull/68 Let me know if you want me to do some changes Regards, Anthony 2017-03-23 13:15 GMT+01:00 Anthony Maire : > Ok, I will open a jira ticket and provide a PR. > > Thanks

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Ok, I will open a jira ticket and provide a PR. Thanks for your input. Le 23 mars 2017 13:08, "Remko Popma" a écrit : > I see what you mean now. I agree it's better to keep the rollover concept > to mean file rename and compression that happen in sequence together. So >

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I see what you mean now. I agree it's better to keep the rollover concept to mean file rename and compression that happen in sequence together. So the randomization affects when the _sequence_ is triggered, not just one part of the sequence. Makes sense. Sent from my iPhone > On Mar 23,

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Hi Remko My first idea was to have the rolling that triggers at the expected time, and the compression that will be delayed. That's why I wanted the delayed compression to occur before shutdown since the rolling already occurred. But I think that's a bad idea. First, it will lead to "fancy code"

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

> On Mar 23, 2017, at 1:06, Anthony Maire wrote: > > Thanks for these answers > > @Ralph : that was the kind of idea I had in mind : changing the > RollingFileManager.asyncExecutor to be a ScheduledThreadPoolExecutor, and > based on some configuration, submitting task

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

Thanks for these answers @Ralph : that was the kind of idea I had in mind : changing the RollingFileManager.asyncExecutor to be a ScheduledThreadPoolExecutor, and based on some configuration, submitting task to be executed after a random delay. However with this kind of approach, special

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

These are separate JVMs, so having a single executor would be of no help. I believe the only way to do what you are asking for is to add configuration so that the asynchronous thread has a semi-random delay when it starts. Ralph > On Mar 22, 2017, at 7:58 AM, Greg Thomas

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

>One common issue we have with that framework (and I assume we can have the >same with log4j2) is that all of our JVMs (we can have more than 50 JVMs on >the same server in production) roll their file at midnight. > >When this happens, the system became often not usable for a few seconds >because

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I think something like the following is possible Means that you specify that the rollover should happen at 12:00 but the seconds are kind of randomized via Property Substitution or so. Not sure if date:seconds works but alternatively you could use system or env variables for it i guess. Br,

Re: Compression of rolled files : performance issue when lots of JVMs are using timed based policy

I'm not sure if the feature is in log4j, but that reminds me of the jenkins feature where you can do the same. I'd take a look at the cron triggering policy docs first to see what's already possible. You can also combine that with some scripts, so there's certainly a way to do it using current