Re: Automatically compressing localhost_access_log after rotation

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Martin,

On 8/4/17 5:08 AM, Martin Knoblauch wrote:
> On Thu, Aug 3, 2017 at 7:16 PM, Christopher Schultz < 
> ch...@christopherschultz.net> wrote:
> 
>> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
>> 
>> Martin,
>> 
>> On 8/3/17 5:47 AM, Martin Knoblauch wrote:
>>> is there a way to compress the localhost_access_log.#.txt
>>> file automatically after rotation?
>> 
>> Not really. The file is rotated *during* log events, and stalling
>> to compress a log file is probably not a great solution.
>> 
>> 
> Hmm. Smart way would be to rotate the log, continue logging to the
> new file and compress the old one "in the background". That way
> stalling would not be an issue.

Sure. But that requires more code, of course, and therefore more
complexity.

>> Alternatively/preferably is there a way to put the access
>> logging
>>> under "log4j"?
>> 
>> Also not really, but if you are willing to write code, you can do
>> it. The AccessLogValve handles its own logging to a file, but if
>> you were to subclass AccessLogValve and override the "open"
>> method and assign a value to the AccessLogValve.writer member
>> that writes to a log4j logger, then I think you could probably do
>> this.
>> 
> 
> Writing my own class is clearly an option, but I wanted to first
> check the already available  options.
> 
> 
>> I believe that log4j will stall your access log during the 
>> compression, though, so you might want to think about whether or
>> not you want to implement it this way.
>> 
>> 
> Need to check how they are doing it. We use rolling with
> compression, but not on a high volume log like the access log.

You'd have to check to see how log4j actually does things, but I think
compression might happen in-line with the logging thread. I'm sure if
varies greatly depending upon your log4j version.

- -chris
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJZhO6hAAoJEBzwKT+lPKRY9OgP/iG+YKDbbG3w7SaKyvOSMlzK
Y9yxQqgK0fpGO7XgMXKM0IuPh8G5EWwiNPUc5k0AbQNjCkNcTmTISzj8TUTu43HO
eyPlVOWjZRfdvetL99AxhkEDaw1j7HdsL5fi+zDi440t8sS4z4RETeT5a8DE2Vti
v80ER+EdAMJeydvOxMMGgk8nHal379aW95m1JHJQppC8V6wcbWnYr7TGVUjl+gDR
SL9fiiymrgLYvC1WhSblKQaOEpzyfhoFm4IOTwG/M5jh/coPsFj7KGdpxels69MM
B4dLdKwbECZbVJ2348rScrxjMevjRLkVlq38zw2kchk6gav9DjfF1i1nMd8piOoD
FgpXgfjn7elq9C2jaKgGodfBdc/cw04ysoZgTRss71vfcL69WW/yQ6rHMUJH0XsM
fqDmZWfXzwBBH6I4fGZuiH+Z+bzdsrSYBgWcJxlGtDE3Vf07oCljvb3W4GCXVdzq
u0qdB+YWyU0daWEErdGl4AlwCmE21HSzzwz/4RdvMJHgxgMwTs2KfF6/KCP2S1PX
Hpv5dGBSb8OWOZ8Y6I6N2Qo1VSME9WfGh4mU8X/r2rMPeBjRoOwVnG6OYPtCYSnl
TKyPpGTHf/m30A6k3uGucYaSocyNvqmP9MqCfeHEAyG/eY3ig/BY1HW65Oel5vAM
FzNKelDMwl4zqyYwDIr+
=5o3t
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Automatically compressing localhost_access_log after rotation

[Mark H. Wood]

On 8/3/17 5:47 AM, Martin Knoblauch wrote:
> is there a way to compress the localhost_access_log.#.txt file 
> automatically after rotation?

There is, but maybe not one built into Tomcat.  I have a daily cron
job to run a script which looks for older logs and ZIPs them:

LOGDIR=/var/log/tomcat-7

ZIP_OPTS='-9mou'

# localhost
for LOG in $(find ${LOGDIR} -mtime +7 -name localhost.-??-??.log | sort); do
  YEAR=$(echo ${LOG} | cut -d. -f2 | cut -d- -f1)
  zip ${ZIP_OPTS} /var/biglogs/tomcat/localhost.log.${YEAR}.zip ${LOG}
done

I have a bunch of these blocks in the script, so I ought to make one
into a function and just call it N times with various arguments, but
this works for me.

-- 
Mark H. Wood
Lead Technology Analyst

University Library
Indiana University - Purdue University Indianapolis
755 W. Michigan Street
Indianapolis, IN 46202
317-274-0749
www.ulib.iupui.edu


signature.asc
Description: PGP signature

Re: Automatically compressing localhost_access_log after rotation

[Martin Knoblauch]

On Thu, Aug 3, 2017 at 9:43 PM, Leon Rosenberg 
wrote:

> On Thu, Aug 3, 2017 at 8:16 PM, Christopher Schultz <
> ch...@christopherschultz.net> wrote:
>
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA256
> >
> > Martin,
> >
> > On 8/3/17 5:47 AM, Martin Knoblauch wrote:
> > > is there a way to compress the localhost_access_log.#.txt file
> > > automatically after rotation?
> >
> > Not really. The file is rotated *during* log events, and stalling to
> > compress a log file is probably not a great solution.
> >
> > > Alternatively/preferably is there a way to put the access logging
> > > under "log4j"?
> >
> > Also not really, but if you are willing to write code, you can do it.
> > The AccessLogValve handles its own logging to a file, but if you were
> > to subclass AccessLogValve and override the "open" method and assign a
> > value to the AccessLogValve.writer member that writes to a log4j
> > logger, then I think you could probably do this.
> >
> > I believe that log4j will stall your access log during the
> > compression, though, so you might want to think about whether or not
> > you want to implement it this way.
> >
> > I think at least logback is performing file operations asynchronously to
> log events so maybe using slf4j over logback would be a more reliable way.
>
> regards
> Leon
>

Hi Leon,

 thanks for the hint. Will look into it.

Cheers
Martin

-- 
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de

Re: Automatically compressing localhost_access_log after rotation

[Martin Knoblauch]

On Thu, Aug 3, 2017 at 7:16 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Martin,
>
> On 8/3/17 5:47 AM, Martin Knoblauch wrote:
> > is there a way to compress the localhost_access_log.#.txt file
> > automatically after rotation?
>
> Not really. The file is rotated *during* log events, and stalling to
> compress a log file is probably not a great solution.
>
>
 Hmm. Smart way would be to rotate the log, continue logging to the new
file and compress the old one "in the background". That way stalling would
not be an issue.

> Alternatively/preferably is there a way to put the access logging
> > under "log4j"?
>
> Also not really, but if you are willing to write code, you can do it.
> The AccessLogValve handles its own logging to a file, but if you were
> to subclass AccessLogValve and override the "open" method and assign a
> value to the AccessLogValve.writer member that writes to a log4j
> logger, then I think you could probably do this.
>

Writing my own class is clearly an option, but I wanted to first check the
already available  options.


> I believe that log4j will stall your access log during the
> compression, though, so you might want to think about whether or not
> you want to implement it this way.
>
>
Need to check how they are doing it. We use rolling with compression, but
not on a high volume log like the access log.


> What is your operating system? If it were me, I think I'd write a cron
> job to check the directory for uncompressed, rotated log files and
> compress them in a separate process.
>
>
Linux. Sure, we can deploy Cron to do the compression. It just adds another
external process to an already complex setup. But I guess nothing in life
is free :-)


> > I am on Tomcat 7.0.62 (no, do not tell me to update. I know. I want
> > to. But I am not allowed to. So, do not tell me. Please :-)
>
> You should upgrade :)
>

I told you not to tell me :-)

Thanks
Martin


-- 
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de

Re: Automatically compressing localhost_access_log after rotation

[Leon Rosenberg]

On Thu, Aug 3, 2017 at 8:16 PM, Christopher Schultz <
ch...@christopherschultz.net> wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
>
> Martin,
>
> On 8/3/17 5:47 AM, Martin Knoblauch wrote:
> > is there a way to compress the localhost_access_log.#.txt file
> > automatically after rotation?
>
> Not really. The file is rotated *during* log events, and stalling to
> compress a log file is probably not a great solution.
>
> > Alternatively/preferably is there a way to put the access logging
> > under "log4j"?
>
> Also not really, but if you are willing to write code, you can do it.
> The AccessLogValve handles its own logging to a file, but if you were
> to subclass AccessLogValve and override the "open" method and assign a
> value to the AccessLogValve.writer member that writes to a log4j
> logger, then I think you could probably do this.
>
> I believe that log4j will stall your access log during the
> compression, though, so you might want to think about whether or not
> you want to implement it this way.
>
> I think at least logback is performing file operations asynchronously to
log events so maybe using slf4j over logback would be a more reliable way.

regards
Leon

Re: Automatically compressing localhost_access_log after rotation

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Martin,

On 8/3/17 5:47 AM, Martin Knoblauch wrote:
> is there a way to compress the localhost_access_log.#.txt file 
> automatically after rotation?

Not really. The file is rotated *during* log events, and stalling to
compress a log file is probably not a great solution.

> Alternatively/preferably is there a way to put the access logging
> under "log4j"?

Also not really, but if you are willing to write code, you can do it.
The AccessLogValve handles its own logging to a file, but if you were
to subclass AccessLogValve and override the "open" method and assign a
value to the AccessLogValve.writer member that writes to a log4j
logger, then I think you could probably do this.

I believe that log4j will stall your access log during the
compression, though, so you might want to think about whether or not
you want to implement it this way.

What is your operating system? If it were me, I think I'd write a cron
job to check the directory for uncompressed, rotated log files and
compress them in a separate process.

> I am on Tomcat 7.0.62 (no, do not tell me to update. I know. I want
> to. But I am not allowed to. So, do not tell me. Please :-)

You should upgrade :)

Actually, 7.0.62 is pretty good. Most people say "I'm using Tomcat
7.0.13" or whatever. Your version is at least younger than my kids.

You might want to make your decision makers aware of several important
security issues with Tomcat 7.0.x that have been patched after the
version you are using. Have a look at the changelog[1] specifically
for versions 7.0.75 and 7.0.77. Depending upon your use of Tomcat's
various features, versions 7.0.68, 7.0.70, 7.0.73, and 7.0.78 might be
concerning to you.

- -chris

[1] http://tomcat.apache.org/security-7.html
-BEGIN PGP SIGNATURE-
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBCAAGBQJZg1phAAoJEBzwKT+lPKRYLy0P/jzNNe/5dCzCZSpwkkKobamt
/eroSyMkR+8CiU2i6AbnGrXmhqDUsif3UQJgBik76wscK9V+BqCatcLEMxY2R9Yu
1EOFAexBwC3mhjwXJLjxITMiECmIEhvwxykb+n7EbXQ+OP7xA7b6MHoGAKjXKc8v
KFnnSKUr90D9tRYjGoKqPdDYaaVjo7qZVx6kHwLCl3aw+NGlrDMlzQYkY/mQzaBc
IDO3YhbQss39GDLGrdF02kNuDkK6xwbikSvObwS7dbA2suGyUUyfweU5AA8BkUi0
hrvwgfMuHJFxKF++LiR2AJDga+qpT+3cYt06wzTc4bNjk7FLrnF5s6Y624nddoPV
9kzfuhSoTWA4xWlX8pIOMkx9B7TrDZT/Y2hnnUZuvBDQLpvgM4qtWgRS/CR+4l+0
0ScxLLwzTHeZGLnjut7bN1JLjDUopt4Su+loPeO11AfFbkhhEbrk42RM68Q4FzfP
2cnHK+lKGHhwhrFIiw5l7cpir4UJyuFjM1Bz9AJ98xnpROqbYXTeSfwMpSZfLAf2
pANppcqc0HczZvSJY9BfxY9UCgoLVGc9UeUd1NV5ecb57GkEcElXriHBAPOonrbQ
0xCNLp1f6gLIWzpwrrBSMPe61x6zJMxl0pb3c6jpKnDUQnCeoP4YK9rERYv2giw1
fUEQn+EFIqRfNuKmKt5j
=fZ5s
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Automatically compressing localhost_access_log after rotation

[Martin Knoblauch]

Hi,

 is there a way to compress the localhost_access_log.#.txt file
automatically after rotation? Alternatively/preferably is there a way to
put the access logging under "log4j"?

 I am on Tomcat 7.0.62 (no, do not tell me to update. I know. I want to.
But I am not allowed to. So, do not tell me. Please :-)

Thanks
Martin
-- 
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de