Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Anshu Garg1]

Hi,

Due to limited time left now for us to complete our logging application, 
today we finally achieved valid JSON files with Log4j1 itself, of course 
it is not without few undesired restrictions we put on format.

We implemented an Action class to open a log file and write footer to it. 
This action instance is being passed to RolloverDescriptionImpl()

  public RolloverDescription rollover(final String currentLogFileName) {

WriteFooterAction action = new 
WriteFooterAction(currentLogFileName, footerText);
String newFileName = "newLogFileName";

return new RolloverDescriptionImpl(newFileName, false, null, 
action);
}

Gary, I created a JIRA for providing a way to customize header and footer 
in JSONLayout format https://issues.apache.org/jira/browse/LOG4J2-1244

I'll test the https://issues.apache.org/jira/browse/LOG4J2-908 as soon as 
possible and update this thread.

-Thanks & Regards,
Anshu Garg
Platform Technology Center, India

Phone: 8800096760
E-mail: anshug...@in.ibm.com 



From:   Remko Popma 
To: Log4J Users List 
Date:   01/05/2016 07:19 PM
Subject:Re: Using log4j1.2.17 foter is not getting added to each 
log file in rollingfileappender



FWIW, decoupling the filePattern from the rollover frequency is one of the
things I had in mind with

   1. LOG4J2-1198 .


On Tuesday, 5 January 2016, Gary Gregory  wrote:

> On Mon, Jan 4, 2016 at 11:04 PM, Anshu Garg1  > wrote:
>
> > Hi Ralph,
> >
> > We  have actually tried log4j2, but unfortunately with that also we 
could
> > not meet our requirements
> >
> >  We are trying a JSON log using rolling file appender based on both 
time
> > and size but we are facing below challenges
> >
> > 1. Apache log4j2 default roll over implementation adds time stamp to 
log
> > file name  while saving and closing, while our requirement is to add 
time
> > stamp to file name at the time of file creation. For this we 
implemented
> > our own rollover strategy. But that was not a complete solution since
> > first file name was saved in rolling file appender without time stamp 
and
> > to change that we extended "AbstractOutputStreamAppender" to create 
our
> > own RollingFileAppender (but most of code in it is from Apache, except
> > what we had to modify to set first file name with time stamp).
> >
> >
> 
http://stackoverflow.com/questions/30059333/log4j2-rolling-appender-filename-sliding-according-to-pattern

> >
> >
> >
> 
http://stackoverflow.com/questions/14862770/log4j2-assigning-file-appender-filename-at-runtime

> >
> >
> > 2. Above lead to mismatch between  the type of parameters in various
> other
> > classes' methods where default Apache classes' types  were expected 
but
> > we were using our own implementation. So we had to use source code for
> > such classes as is and just replace parameters' types from default to 
our
> > implemented classes.
> >
> > 3. By default JSON Layout from Apache log4j2 has a non resolved defect
> > which does not write a valid JSON. Log entries do not have a comma
> between
> > then which results in invalid JSON. To rectify this we implemented our
> own
> > layout. This again has most of the code from Apache but few changes 
here
> > and there to get comma at start of each entry except first.
> > https://issues.apache.org/jira/browse/LOG4J2-908
>
>
> Hello Anshu,
>
> LOG4J2-908  is fixed 
in
> Git master. You can build from Git and install locally or point your 
Maven
> build to our SNAPSHOT repository here:
>
> https://repository.apache.org/content/repositories/snapshots/
>
>
> >
> > 4. We had to customize header and footer values as per our 
requirement,
> > since default in Apache default header is "[" and footer is "]". While 
we
> > needed more information in header section.
> >
>
> If you want to make the "[" and "]" customizable for the JSON layout,
> please create a JIRA. Can you describe the use case please? That would 
make
> for a better story in the documentation.
>
>
> > 5. With Apache log4j2's default behavior for time based rollover
> >  duration's unit was determined by file pattern. We had to make 
duration
> > time unit independent of file pattern because with default behavior if 
a
> > user wanted time stamp in file name till mili seconds then even file
> > rollover would happen in mili seconds which is not desired. And if 
user
> > wanted hourly or daily roll over then file stamp could only be used 
till
> > date or hour. To be able to specify duration in a definite unit 
(minutes
> > in our case) we had to implement our own PatternProcessor.
> > https://issues.apache.org/jira/browse/LOG4J2-573
> >
> > 6. We needed to add some custom log fields in JSON logs for which we 
had
> > to write our own LogEntry and MessageFactory.
> >
>
> Can you be more 

Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Ralph Goers]

Remember, you can now use the CronTriggeringPolicy which does not use the 
filePattern.

Ralph

> On Jan 5, 2016, at 6:49 AM, Remko Popma  wrote:
> 
> FWIW, decoupling the filePattern from the rollover frequency is one of the
> things I had in mind with
> 
>   1. LOG4J2-1198  >.
> 
> 
> On Tuesday, 5 January 2016, Gary Gregory  > wrote:
> 
>> On Mon, Jan 4, 2016 at 11:04 PM, Anshu Garg1 > > wrote:
>> 
>>> Hi Ralph,
>>> 
>>> We  have actually tried log4j2, but unfortunately with that also we could
>>> not meet our requirements
>>> 
>>> We are trying a JSON log using rolling file appender based on both time
>>> and size but we are facing below challenges
>>> 
>>> 1. Apache log4j2 default roll over implementation adds time stamp to log
>>> file name  while saving and closing, while our requirement is to add time
>>> stamp to file name at the time of file creation. For this we implemented
>>> our own rollover strategy. But that was not a complete solution since
>>> first file name was saved in rolling file appender without time stamp and
>>> to change that we extended "AbstractOutputStreamAppender" to create our
>>> own RollingFileAppender (but most of code in it is from Apache, except
>>> what we had to modify to set first file name with time stamp).
>>> 
>>> 
>> http://stackoverflow.com/questions/30059333/log4j2-rolling-appender-filename-sliding-according-to-pattern
>>> 
>>> 
>>> 
>> http://stackoverflow.com/questions/14862770/log4j2-assigning-file-appender-filename-at-runtime
>>> 
>>> 
>>> 2. Above lead to mismatch between  the type of parameters in various
>> other
>>> classes' methods where default Apache classes' types  were expected  but
>>> we were using our own implementation. So we had to use source code for
>>> such classes as is and just replace parameters' types from default to our
>>> implemented classes.
>>> 
>>> 3. By default JSON Layout from Apache log4j2 has a non resolved defect
>>> which does not write a valid JSON. Log entries do not have a comma
>> between
>>> then which results in invalid JSON. To rectify this we implemented our
>> own
>>> layout. This again has most of the code from Apache but few changes here
>>> and there to get comma at start of each entry except first.
>>> https://issues.apache.org/jira/browse/LOG4J2-908
>> 
>> 
>> Hello Anshu,
>> 
>> LOG4J2-908  is fixed in
>> Git master. You can build from Git and install locally or point your Maven
>> build to our SNAPSHOT repository here:
>> 
>> https://repository.apache.org/content/repositories/snapshots/
>> 
>> 
>>> 
>>> 4. We had to customize header and footer values as per our requirement,
>>> since default in Apache default header is "[" and footer is "]". While we
>>> needed more information in header section.
>>> 
>> 
>> If you want to make the "[" and "]" customizable for the JSON layout,
>> please create a JIRA. Can you describe the use case please? That would make
>> for a better story in the documentation.
>> 
>> 
>>> 5. With Apache log4j2's default behavior for time based rollover
>>> duration's unit was determined by file pattern. We had to make duration
>>> time unit independent of file pattern because with default behavior  if a
>>> user wanted time stamp in file name till mili seconds then even file
>>> rollover would happen in mili seconds which is not desired. And if user
>>> wanted hourly or daily roll over then file stamp could only be used till
>>> date or hour. To be able to specify duration in a definite unit (minutes
>>> in our case) we had to implement our own PatternProcessor.
>>> https://issues.apache.org/jira/browse/LOG4J2-573
>>> 
>>> 6. We needed to add some custom log fields in JSON logs for which we had
>>> to write our own LogEntry and MessageFactory.
>>> 
>> 
>> Can you be more specific please?
>> 
>> 
>>> 
>>> All above needed to write own classes with most functionality copied from
>>> Apache which means we could not achieve above with just extending the
>>> Apache classes. We had to make source code changes which we want to
>> avoid.
>>> Do you have any suggestion to overcome above without having to fork code
>>> of Log4j2 ?
>>> 
>> 
>> You've come to the right place. Let's start with your validation that
>> https://issues.apache.org/jira/browse/LOG4J2-908 has been fixed for your
>> use case.
>> 
>> Gary
>> 
>> 
>>> We did not find any solution to above so we came back to Log4j1. Is there
>>> a way round to get footer in Log4j1 or Log4j2,  with above requirements,
>>> without having to change source code ?
>>> 
>>> 
>>> -Thanks & Regards,
>>> Anshu Garg
>>> Platform Technology Center, India
>>> 
>>> 
>>> 
>>> 
>>> 
>>> From:   Ralph Goers >
>>> To: Log4J Users List 

Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Remko Popma]

FWIW, decoupling the filePattern from the rollover frequency is one of the
things I had in mind with

   1. LOG4J2-1198 .


On Tuesday, 5 January 2016, Gary Gregory  wrote:

> On Mon, Jan 4, 2016 at 11:04 PM, Anshu Garg1  > wrote:
>
> > Hi Ralph,
> >
> > We  have actually tried log4j2, but unfortunately with that also we could
> > not meet our requirements
> >
> >  We are trying a JSON log using rolling file appender based on both time
> > and size but we are facing below challenges
> >
> > 1. Apache log4j2 default roll over implementation adds time stamp to log
> > file name  while saving and closing, while our requirement is to add time
> > stamp to file name at the time of file creation. For this we implemented
> > our own rollover strategy. But that was not a complete solution since
> > first file name was saved in rolling file appender without time stamp and
> > to change that we extended "AbstractOutputStreamAppender" to create our
> > own RollingFileAppender (but most of code in it is from Apache, except
> > what we had to modify to set first file name with time stamp).
> >
> >
> http://stackoverflow.com/questions/30059333/log4j2-rolling-appender-filename-sliding-according-to-pattern
> >
> >
> >
> http://stackoverflow.com/questions/14862770/log4j2-assigning-file-appender-filename-at-runtime
> >
> >
> > 2. Above lead to mismatch between  the type of parameters in various
> other
> > classes' methods where default Apache classes' types  were expected  but
> > we were using our own implementation. So we had to use source code for
> > such classes as is and just replace parameters' types from default to our
> > implemented classes.
> >
> > 3. By default JSON Layout from Apache log4j2 has a non resolved defect
> > which does not write a valid JSON. Log entries do not have a comma
> between
> > then which results in invalid JSON. To rectify this we implemented our
> own
> > layout. This again has most of the code from Apache but few changes here
> > and there to get comma at start of each entry except first.
> > https://issues.apache.org/jira/browse/LOG4J2-908
>
>
> Hello Anshu,
>
> LOG4J2-908  is fixed in
> Git master. You can build from Git and install locally or point your Maven
> build to our SNAPSHOT repository here:
>
> https://repository.apache.org/content/repositories/snapshots/
>
>
> >
> > 4. We had to customize header and footer values as per our requirement,
> > since default in Apache default header is "[" and footer is "]". While we
> > needed more information in header section.
> >
>
> If you want to make the "[" and "]" customizable for the JSON layout,
> please create a JIRA. Can you describe the use case please? That would make
> for a better story in the documentation.
>
>
> > 5. With Apache log4j2's default behavior for time based rollover
> >  duration's unit was determined by file pattern. We had to make duration
> > time unit independent of file pattern because with default behavior  if a
> > user wanted time stamp in file name till mili seconds then even file
> > rollover would happen in mili seconds which is not desired. And if user
> > wanted hourly or daily roll over then file stamp could only be used till
> > date or hour. To be able to specify duration in a definite unit (minutes
> > in our case) we had to implement our own PatternProcessor.
> > https://issues.apache.org/jira/browse/LOG4J2-573
> >
> > 6. We needed to add some custom log fields in JSON logs for which we had
> > to write our own LogEntry and MessageFactory.
> >
>
> Can you be more specific please?
>
>
> >
> > All above needed to write own classes with most functionality copied from
> > Apache which means we could not achieve above with just extending the
> > Apache classes. We had to make source code changes which we want to
> avoid.
> > Do you have any suggestion to overcome above without having to fork code
> > of Log4j2 ?
> >
>
> You've come to the right place. Let's start with your validation that
> https://issues.apache.org/jira/browse/LOG4J2-908 has been fixed for your
> use case.
>
> Gary
>
>
> > We did not find any solution to above so we came back to Log4j1. Is there
> > a way round to get footer in Log4j1 or Log4j2,  with above requirements,
> > without having to change source code ?
> >
> >
> > -Thanks & Regards,
> > Anshu Garg
> > Platform Technology Center, India
> >
> >
> >
> >
> >
> > From:   Ralph Goers >
> > To: Log4J Users List >
> > Date:   01/05/2016 12:04 PM
> > Subject:Re: Using log4j1.2.17 foter is not getting added to each
> > log file in rollingfileappender
> >
> >
> >
> > Please see
> >
> >
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> > <
> >
> >
> 

Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Simon Park]

FWIW, if you're stuck on Log4J 1.2.17, you could use the 
TimeAndSizeRollingAppender available from www.simonsite.org.uk to solve this 
use case.

Best Regards,

Simon

Sent from my iPad

> On 5 Jan 2016, at 06:14, Anshu Garg1  wrote:
> 
> Hi,
> 
> We have extended patternlayout to set custom footer and header. Now when 
> we use this pattern in rolling file appender footer is only added to last 
> log file generated while header is added to all files. Our requirement is 
> to have footer also added to files created. Any suggestion? Here is the 
> snippet. 
> 
> Logger logger = Logger.getLogger("myLogger");
> CustomPattern pattern = new CustomPattern();
> pattern.setHeader("HeaderText");
> pattern.setFooter("FooterText");
> rollingFileAppender.setLayout(pattern);
> 
> -Thanks & Regards,
> Anshu Garg
> Platform Technology Center, India
> 
> Phone: 8800096760
> E-mail: anshug...@in.ibm.com 
> 

-
To unsubscribe, e-mail: log4j-user-unsubscr...@logging.apache.org
For additional commands, e-mail: log4j-user-h...@logging.apache.org

Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Gary Gregory]

On Tue, Jan 5, 2016 at 2:51 PM, Remko Popma  wrote:

> Thanks for raising these issues. In addition to Gary's response, can I ask
> some more questions to clarify?
>
> 1. Does LOG4J2-1101 
> capture
> your first requirement for a date in the pre-rollover file name?
> 3. I believe you are already verifying the fix for LOG4J2-908
> , so that's in progress.
> 4. I commented on your ticket LOG4J2-1244
>  for the custom
> header/footer, let us know if that won't work.
>

This one is done (in Git master).

Gary


> 5. Decoupling file pattern from rollover frequency can be achieved with
> CronTriggerPolicy
> <
> http://logging.apache.org/log4j/2.x/manual/appenders.html#TriggeringPolicies
> >
> since 2.5. Does this do what you have in mind? (You mention LOG4J2-573
> , but it looks like that
> ticket is about something else, unless I misunderstand).
> 6. Custom log fields, are these items that cannot be covered with Lookups
>  or ThreadContext
> (MDC) 
> values? Like Gary, I'd like to understand better.
>
>
> On Tue, Jan 5, 2016 at 4:04 PM, Anshu Garg1  wrote:
>
> > Hi Ralph,
> >
> > We  have actually tried log4j2, but unfortunately with that also we could
> > not meet our requirements
> >
> >  We are trying a JSON log using rolling file appender based on both time
> > and size but we are facing below challenges
> >
> > 1. Apache log4j2 default roll over implementation adds time stamp to log
> > file name  while saving and closing, while our requirement is to add time
> > stamp to file name at the time of file creation. For this we implemented
> > our own rollover strategy. But that was not a complete solution since
> > first file name was saved in rolling file appender without time stamp and
> > to change that we extended "AbstractOutputStreamAppender" to create our
> > own RollingFileAppender (but most of code in it is from Apache, except
> > what we had to modify to set first file name with time stamp).
> >
> >
> http://stackoverflow.com/questions/30059333/log4j2-rolling-appender-filename-sliding-according-to-pattern
> >
> >
> >
> http://stackoverflow.com/questions/14862770/log4j2-assigning-file-appender-filename-at-runtime
> >
> >
> > 2. Above lead to mismatch between  the type of parameters in various
> other
> > classes' methods where default Apache classes' types  were expected  but
> > we were using our own implementation. So we had to use source code for
> > such classes as is and just replace parameters' types from default to our
> > implemented classes.
> >
> > 3. By default JSON Layout from Apache log4j2 has a non resolved defect
> > which does not write a valid JSON. Log entries do not have a comma
> between
> > then which results in invalid JSON. To rectify this we implemented our
> own
> > layout. This again has most of the code from Apache but few changes here
> > and there to get comma at start of each entry except first.
> > https://issues.apache.org/jira/browse/LOG4J2-908
> >
> > 4. We had to customize header and footer values as per our requirement,
> > since default in Apache default header is "[" and footer is "]". While we
> > needed more information in header section.
> >
> > 5. With Apache log4j2's default behavior for time based rollover
> >  duration's unit was determined by file pattern. We had to make duration
> > time unit independent of file pattern because with default behavior  if a
> > user wanted time stamp in file name till mili seconds then even file
> > rollover would happen in mili seconds which is not desired. And if user
> > wanted hourly or daily roll over then file stamp could only be used till
> > date or hour. To be able to specify duration in a definite unit (minutes
> > in our case) we had to implement our own PatternProcessor.
> > https://issues.apache.org/jira/browse/LOG4J2-573
> >
> > 6. We needed to add some custom log fields in JSON logs for which we had
> > to write our own LogEntry and MessageFactory.
> >
> > All above needed to write own classes with most functionality copied from
> > Apache which means we could not achieve above with just extending the
> > Apache classes. We had to make source code changes which we want to
> avoid.
> > Do you have any suggestion to overcome above without having to fork code
> > of Log4j2 ?
> >
> > We did not find any solution to above so we came back to Log4j1. Is there
> > a way round to get footer in Log4j1 or Log4j2,  with above requirements,
> > without having to change source code ?
> >
> >
> > -Thanks & Regards,
> > Anshu Garg
> > Platform Technology Center, India
> >
> >
> >
> >
> >
> > From:   Ralph Goers 
> > To: Log4J Users 

Re: Using log4j1.2.17 foter is not getting added to each log file in rollingfileappender

[Remko Popma]

Thanks for raising these issues. In addition to Gary's response, can I ask
some more questions to clarify?

1. Does LOG4J2-1101  capture
your first requirement for a date in the pre-rollover file name?
3. I believe you are already verifying the fix for LOG4J2-908
, so that's in progress.
4. I commented on your ticket LOG4J2-1244
 for the custom
header/footer, let us know if that won't work.
5. Decoupling file pattern from rollover frequency can be achieved with
CronTriggerPolicy

since 2.5. Does this do what you have in mind? (You mention LOG4J2-573
, but it looks like that
ticket is about something else, unless I misunderstand).
6. Custom log fields, are these items that cannot be covered with Lookups
 or ThreadContext
(MDC) 
values? Like Gary, I'd like to understand better.


On Tue, Jan 5, 2016 at 4:04 PM, Anshu Garg1  wrote:

> Hi Ralph,
>
> We  have actually tried log4j2, but unfortunately with that also we could
> not meet our requirements
>
>  We are trying a JSON log using rolling file appender based on both time
> and size but we are facing below challenges
>
> 1. Apache log4j2 default roll over implementation adds time stamp to log
> file name  while saving and closing, while our requirement is to add time
> stamp to file name at the time of file creation. For this we implemented
> our own rollover strategy. But that was not a complete solution since
> first file name was saved in rolling file appender without time stamp and
> to change that we extended "AbstractOutputStreamAppender" to create our
> own RollingFileAppender (but most of code in it is from Apache, except
> what we had to modify to set first file name with time stamp).
>
> http://stackoverflow.com/questions/30059333/log4j2-rolling-appender-filename-sliding-according-to-pattern
>
>
> http://stackoverflow.com/questions/14862770/log4j2-assigning-file-appender-filename-at-runtime
>
>
> 2. Above lead to mismatch between  the type of parameters in various other
> classes' methods where default Apache classes' types  were expected  but
> we were using our own implementation. So we had to use source code for
> such classes as is and just replace parameters' types from default to our
> implemented classes.
>
> 3. By default JSON Layout from Apache log4j2 has a non resolved defect
> which does not write a valid JSON. Log entries do not have a comma between
> then which results in invalid JSON. To rectify this we implemented our own
> layout. This again has most of the code from Apache but few changes here
> and there to get comma at start of each entry except first.
> https://issues.apache.org/jira/browse/LOG4J2-908
>
> 4. We had to customize header and footer values as per our requirement,
> since default in Apache default header is "[" and footer is "]". While we
> needed more information in header section.
>
> 5. With Apache log4j2's default behavior for time based rollover
>  duration's unit was determined by file pattern. We had to make duration
> time unit independent of file pattern because with default behavior  if a
> user wanted time stamp in file name till mili seconds then even file
> rollover would happen in mili seconds which is not desired. And if user
> wanted hourly or daily roll over then file stamp could only be used till
> date or hour. To be able to specify duration in a definite unit (minutes
> in our case) we had to implement our own PatternProcessor.
> https://issues.apache.org/jira/browse/LOG4J2-573
>
> 6. We needed to add some custom log fields in JSON logs for which we had
> to write our own LogEntry and MessageFactory.
>
> All above needed to write own classes with most functionality copied from
> Apache which means we could not achieve above with just extending the
> Apache classes. We had to make source code changes which we want to avoid.
> Do you have any suggestion to overcome above without having to fork code
> of Log4j2 ?
>
> We did not find any solution to above so we came back to Log4j1. Is there
> a way round to get footer in Log4j1 or Log4j2,  with above requirements,
> without having to change source code ?
>
>
> -Thanks & Regards,
> Anshu Garg
> Platform Technology Center, India
>
>
>
>
>
> From:   Ralph Goers 
> To: Log4J Users List 
> Date:   01/05/2016 12:04 PM
> Subject:Re: Using log4j1.2.17 foter is not getting added to each
> log file in rollingfileappender
>
>
>
> Please see
>
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> <
>
>