https://issues.apache.org/jira/browse/LOG4J2-1855 https://github.com/apache/logging-log4j2/pull/68
Let me know if you want me to do some changes Regards, Anthony 2017-03-23 13:15 GMT+01:00 Anthony Maire <maire.anth...@gmail.com>: > Ok, I will open a jira ticket and provide a PR. > > Thanks for your input. > > Le 23 mars 2017 13:08, "Remko Popma" <remko.po...@gmail.com> a écrit : > >> I see what you mean now. I agree it's better to keep the rollover concept >> to mean file rename and compression that happen in sequence together. So >> the randomization affects when the _sequence_ is triggered, not just one >> part of the sequence. Makes sense. >> >> Sent from my iPhone >> >> > On Mar 23, 2017, at 16:28, Anthony Maire <maire.anth...@gmail.com> >> wrote: >> > >> > Hi Remko >> > >> > My first idea was to have the rolling that triggers at the expected >> time, >> > and the compression that will be delayed. That's why I wanted the >> delayed >> > compression to occur before shutdown since the rolling already occurred. >> > >> > But I think that's a bad idea. First, it will lead to "fancy code" and I >> > would like to avoid it too. But the main issue is that this behavior >> should >> > impact only the time based triggering when combining several policy. So >> the >> > code should be related to the triggering policy and not to the rolling >> > strategy. >> > >> > So the best thing to do is to add some property on the timed base >> > triggering policy and let that class handle all the logic and delay the >> > triggering itself instead of the compression. >> > >> > Are you OK with that? >> > >> > Anthony >> > >> > Le 23 mars 2017 00:24, "Remko Popma" <remko.po...@gmail.com> a écrit : >> > >> > >> >> On Mar 23, 2017, at 1:06, Anthony Maire <maire.anth...@gmail.com> >> wrote: >> >> >> >> Thanks for these answers >> >> >> >> @Ralph : that was the kind of idea I had in mind : changing the >> >> RollingFileManager.asyncExecutor to be a ScheduledThreadPoolExecutor, >> and >> >> based on some configuration, submitting task to be executed after a >> random >> >> delay. However with this kind of approach, special treatment should be >> > made >> >> if the manager is stopped with some pending delayed tasks in it. >> > >> > I'm okay with randomization except for this last bit about "special >> > treatment...". Let's not make it too fancy. If the manager is stopped >> > before it rolled over, then it didn't roll over, just like it works >> > currently. I don't see the point of adding extra logic to trigger a >> > rollover when the manager is stopped within the randomized time window. >> > >> >> >> >> @Matt : Cron policy can be a solution, but I don't know how to inject >> some >> >> random element in this to make the file roll at midnight + X random >> >> seconds. Since there is a lot of JVM to manage and some of them can be >> >> moved from a machine to another, I need to have a single log4j2.xml >> file >> >> for all environments. Moreover, our system administrators are >> reluctant to >> >> have something based on a shell-specific feature (such has the $RANDOM >> >> variable from bash) >> >> >> >> Anthony >> >> >> >> 2017-03-22 16:31 GMT+01:00 Ralph Goers <ralph.go...@dslextreme.com>: >> >> >> >>> These are separate JVMs, so having a single executor would be of no >> help. >> >>> >> >>> I believe the only way to do what you are asking for is to add >> >>> configuration so that the asynchronous thread has a semi-random delay >> > when >> >>> it starts. >> >>> >> >>> Ralph >> >>> >> >>>> On Mar 22, 2017, at 7:58 AM, Greg Thomas <greg.d.tho...@gmail.com> >> >>> wrote: >> >>>> >> >>>>> One common issue we have with that framework (and I assume we can >> have >> >>> the >> >>>>> same with log4j2) is that all of our JVMs (we can have more than 50 >> >>> JVMs on >> >>>>> the same server in production) roll their file at midnight. >> >>>>> >> >>>>> When this happens, the system became often not usable for a few >> seconds >> >>>>> because of the simultaneous zipping of all the rolled files that >> >>> overload >> >>>>> the CPU (although zipping is done in a specific background thread). >> >>>> >> >>>> ISTR that with the most recent versions of log4j, these threads are >> in a >> >>>> thread pool so that they are properly shutdown at the right time. I >> >>> wonder >> >>>> if it's possible (or could be possible) to somehow inject a thread >> pool >> >>> in >> >>>> to log4j for this rollover, so that for you use case you could >> inject a >> >>>> single thread executor, so only one thread is ever compressing at a >> > time. >> >>>> >> >>>> Just a thought, anyway, >> >>>> >> >>>> Greg >> >>>> >> >>>> On 22 March 2017 at 13:47, Anthony Maire <maire.anth...@gmail.com> >> >>> wrote: >> >>>> >> >>>>> Hi >> >>>>> >> >>>>> We are currently using another logging framework in production, but >> I'm >> >>>>> pushing to change it for log4j2. >> >>>>> >> >>>>> One common issue we have with that framework (and I assume we can >> have >> >>> the >> >>>>> same with log4j2) is that all of our JVMs (we can have more than 50 >> >>> JVMs on >> >>>>> the same server in production) roll their file at midnight. >> >>>>> >> >>>>> When this happens, the system became often not usable for a few >> seconds >> >>>>> because of the simultaneous zipping of all the rolled files that >> >>> overload >> >>>>> the CPU (although zipping is done in a specific background thread). >> To >> >>>>> reduce this effect, we are combining a time based rolling policy >> with a >> >>>>> sized based policy to zip smaller files, but this is not enough to >> make >> >>> the >> >>>>> system fully responsive at midnight. >> >>>>> >> >>>>> A pretty cool feature for us to avoid this issue is to have the >> >>> possibility >> >>>>> when a rolling is triggered because of a time based policy to change >> >>> file >> >>>>> immediately, but to wait for a random amount of time (within a >> >>> configurable >> >>>>> limit) before starting the compression. This random delay should >> help a >> >>> lot >> >>>>> to avoid contention on CPU cycles. >> >>>>> >> >>>>> Does log4j2 have something to solve this kind of issue ? If not, >> would >> >>> you >> >>>>> accept a pull request for this (I will open a Jira if needed) ? >> >>>>> >> >>>>> Regards, >> >>>>> Anthony >> >>>>> >> >>> >> >>> >> >>> >> >>> --------------------------------------------------------------------- >> >>> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org >> >>> For additional commands, e-mail: log4j-user-h...@logging.apache.org >> >>> >> >>> >> > >> > --------------------------------------------------------------------- >> > To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org >> > For additional commands, e-mail: log4j-user-h...@logging.apache.org >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org >> For additional commands, e-mail: log4j-user-h...@logging.apache.org >> >>
