Re: Novice Tomcat Admin Question - Filtering log output

[Darryl Philip Baker]

>   > The second reason is we use Splunk as a log aggregator. In Splunk

>> it is easy to filter these out when looking at the log but having

>> all these almost useless messages significantly adds to the

>> activity of the Splunk forwarder on these systems.

>I'm surprised Splunk doesn't have a "drop records matching pattern" or

>something like that, so you can just never ingest them. Maybe that

>would be a feature too easy to exploit.



Chris, that is a great idea. I don't control the aggregator and that may be 
where a filter might be configured. I will check.



Darryl Baker, GSEC  (he/him/his)

Sr. System Administrator

Distributed Application Platform Services

Northwestern University

1800 Sherman Ave.

Suite 6-600 – Box #39

Evanston, IL  60201-3715

darryl.ba...@northwestern.edu

(847) 467-6674

Re: Novice Tomcat Admin Question - Filtering log output

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256


Darryl,

On 2/21/20 12:49, Darryl Philip Baker wrote:
> On 2/21/20, 11:36 AM, "Christopher Schultz"
>  wrote:
>
> -BEGIN PGP SIGNED MESSAGE- Hash: SHA256
>
> Darryl,
>
> On 2/21/20 12:15, Darryl Philip Baker wrote:
>> I have taken over the administration of several Tomcat
>> instances. A number of these are load balanced using an F5
>> appliance.  The org.apache.catalina.values.AccessLogValve log
>> file is filled with the F5 polls to see if the application is
>> alive. Under almost all circumstances these are useless, I would
>> like to stop logging just these requests.
> Dumb question: why bother removing them?
>
> Not so dumb a question, I have 2 reasons. First when looking at
> the raw file on the server these poles from the load balancers make
> it hard to find the useful entries.

grep?

> The second reason is we use Splunk as a log aggregator. In Splunk
> it is easy to filter these out when looking at the log but having
> all these almost useless messages significantly adds to the
> activity of the Splunk forwarder on these systems.
I'm surprised Splunk doesn't have a "drop records matching pattern" or
something like that, so you can just never ingest them. Maybe that
would be a feature too easy to exploit.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl5T6dEACgkQHPApP6U8
pFg1SA/9Gl05+KgkrmgwKcbcGKf+KgFCaWALn9ui29CdtwBwES9dMVbxDaKppOMg
ElQKuGIJt0xjqoL/kBRSaNXPWhB2rst9dv1cJI3XrIKQRIM0TNeGSHByTp76HAZD
6xFIoOjuWb8g5keon8TEBlRa+BRvhM6vvBb7AhJqIdQcXajpPkmNH/kYVDPK6+0I
o3ts5gwRtLgf4wErFUZjYR39nk8t9vYgxKJwbWYLeQP9OQrw0ikumzxiadj0YRqk
ETuX7SF8MVznL/Aeb4QY0ifS6u2a4E7JU+KEnwnoPEJfus2Wz/y45RxZAFXE6rD1
7LDmKc0abvzclEjUXBZsxYPFPqDaSys76X41/ZrpyeCVcWEfaMb+SLnPZYbi9HMW
lgkXRrVsbzYZ7o7yMLMlX1QmnbY/i7eg/xSSoq/8D5ZoRnX0luyT3XwgR4hH7hDP
NUdYzMUjZ5yVgC3T7hOAQQ0r908YUaVGTLUVjiVBTv300ntksAQ4fubaoRNEOhlm
wLHCBBtd4I2Ff0TbwTWJd2nVVHGhmz7uiEk8V7WVX4wvy319700BSizrclBb4ybm
T0GBuRilqa/j8cO13LFFp8sJyz4PnFylkLJC/OC9CFfyG5dTd5TRFE+VT5r4mpfo
TFIO+cCPUcUsWnuZzMzyZvQ2veVFYXTAadNNoQE/EZL00KF3bkY=
=2I07
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Novice Tomcat Admin Question - Filtering log output

[tomcat/perl]

On 21.02.2020 18:49, Darryl Philip Baker wrote:

On 2/21/20, 11:36 AM, "Christopher Schultz"  
wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256
 
 Darryl,
 
 On 2/21/20 12:15, Darryl Philip Baker wrote:

 > I have taken over the administration of several Tomcat instances.
 > A number of these are load balanced using an F5 appliance.  The
 > org.apache.catalina.values.AccessLogValve log file is filled with
 > the F5 polls to see if the application is alive. Under almost all
 > circumstances these are useless, I would like to stop logging just
 > these requests.
 Dumb question: why bother removing them?

Not so dumb a question, I have 2 reasons. First when looking at the raw file on 
the server these poles from the load balancers make it hard to find the useful 
entries. The second reason is we use Splunk as a log aggregator. In Splunk it 
is easy to filter these out when looking at the log but having all these almost 
useless messages significantly adds to the activity of the Splunk forwarder on 
these systems.



Apart from everything else (I had a few dumb questions/remarks of my own),
See : http://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Access_Log_Valve
attribute : conditionUnless
So you could use this, along with a Filter which applies only to requests with URI 
"/MySite/isAlive.jsp" and sets an attribute "junk" for these; and bingo! they would not be 
logged anymore.


This page even seems to show how you could do that without a Filter, directly in your own 
"isAlive.jsp" :

https://www.roseindia.net/jsp/requestsetattribute.shtml
(No guarantee though, I'm not really the expert here)


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Novice Tomcat Admin Question - Filtering log output

[Darryl Philip Baker]

On 2/21/20, 11:36 AM, "Christopher Schultz"  
wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Darryl,

On 2/21/20 12:15, Darryl Philip Baker wrote:
> I have taken over the administration of several Tomcat instances.
> A number of these are load balanced using an F5 appliance.  The
> org.apache.catalina.values.AccessLogValve log file is filled with
> the F5 polls to see if the application is alive. Under almost all
> circumstances these are useless, I would like to stop logging just
> these requests.
Dumb question: why bother removing them?

Not so dumb a question, I have 2 reasons. First when looking at the raw file on 
the server these poles from the load balancers make it hard to find the useful 
entries. The second reason is we use Splunk as a log aggregator. In Splunk it 
is easy to filter these out when looking at the log but having all these almost 
useless messages significantly adds to the activity of the Splunk forwarder on 
these systems.

Darryl Baker, GSEC  (he/him/his)
Sr. System Administrator
Distributed Application Platform Services
Northwestern University
1800 Sherman Ave.
Suite 6-600 – Box #39
Evanston, IL  60201-3715
darryl.ba...@northwestern.edu
(847) 467-6674
 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Novice Tomcat Admin Question - Filtering log output

[Darryl Philip Baker]

On 2/21/20, 11:23 AM, "M. Manna"  wrote:

Hey Darryl,

 I may be mistaken, but It seems you are probably trying to make the
logging coarser. You can take a look at conf/logging.properties for your
tomcat instances to do the adjustments of log levels.

Regards,

Thank you for your reply but no I do not want to reduce the logging level. I 
want every other access logged just not the specific target used by the load 
balancers.


Darryl Baker, GSEC  (he/him/his)
Sr. System Administrator
Distributed Application Platform Services
Northwestern University
1800 Sherman Ave.
Suite 6-600 – Box #39
Evanston, IL  60201-3715
darryl.ba...@northwestern.edu
(847) 467-6674
 

 


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Novice Tomcat Admin Question - Filtering log output

[Christopher Schultz]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Darryl,

On 2/21/20 12:15, Darryl Philip Baker wrote:
> I have taken over the administration of several Tomcat instances.
> A number of these are load balanced using an F5 appliance.  The
> org.apache.catalina.values.AccessLogValve log file is filled with
> the F5 polls to see if the application is alive. Under almost all
> circumstances these are useless, I would like to stop logging just
> these requests.
Dumb question: why bother removing them?

> What is the best way to stop these entries being written?  I’ve
> included a sample of the log entries.
>
> 10.0.171.163 - - [21/Feb/2020:09:04:11 -0600] "GET
> /MySite/isAlive.jsp " 200 112 10.0.171.162 - -
> [21/Feb/2020:09:04:16 -0600] "GET /MySite/isAlive.jsp " 200 112
> 10.0.171.163 - - [21/Feb/2020:09:04:16 -0600] "GET
> /MySite/isAlive.jsp " 200 112 10.0.171.162 - -
> [21/Feb/2020:09:04:21 -0600] "GET /MySite/isAlive.jsp " 200 112
> 10.0.171.163 - - [21/Feb/2020:09:04:21 -0600] "GET
> /MySite/isAlive.jsp " 200 112 10.0.171.162 - -
> [21/Feb/2020:09:04:26 -0600] "GET /MySite/isAlive.jsp " 200 112
>
> The entry for the log file in server.xml is  className="org.apache.catalina.valves.AccessLogValve"
> directory="logs" prefix="localhost_access_log" suffix=".txt"
> pattern="%h %l %u %t %r %s %b" />

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl5QFR8ACgkQHPApP6U8
pFjAVRAAqwD1gGJXijn2XlahCEJwh69F4xAAwx8HNo7xPN++QWzbsb2Bk8i+9sdZ
b8zOtumE6lbVfyOAIGwXcy6TnzQbrWSlCA3CpsseADiHC43ZLiO9MFxUyMyxOKV9
CZtaHr8Q0SCo5Lg8U2u529F9Dbbp/MMMiqEqSASUcy86LPpO4BBUJ2mqdCfNEkXM
+jTcOAhiDqq90PdCmonIWG3aNvxsBxgS85HQxyZK2vhcSD1Yy/GbQyqeuZ7ExMj6
umGRQCtwHLjI2zqkhDvYuF4RLLYcZw0uatoANmPvJ/PfjHen3I6MDzktR1jwkWoH
PQuVqCQ/WyM97M1gCml2Gwzwv7eYz74A2I84GPkXx1mOVILweqm9EKOBIKXKJ8Lp
oKd4slic6AGK7SrIogIBuvljFh4HsBE2NvBM0+DJ7s3duy4QCbmHdsGe0Ey1TnKL
6Ib69+QhLbDWl3bWtcjPHN+SperxOh9Y+0ob9jYmGgQB0gSjvujs1eCW9SMkWjoh
GczJaRE+27vYTrkjcg+Ejljl8x+H/2JlZnaYGbqbUX/YUa9jxX0/gMMTxGC7Ha4c
8wGWnSRy1D8EgxgiFq4b/P4bmEwFXrSuwSNPVQ1cHUCNLVkJVOrl+FG25OkoXeAH
whGn9BPi7uX+3uDSVoyx2DkKLxc+bAoJGC2W3lUn21kvs+MzxHE=
=uSo6
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Novice Tomcat Admin Question - Filtering log output

[M. Manna]

Hey Darryl,

On Fri, 21 Feb 2020 at 17:15, Darryl Philip Baker <
darryl.ba...@northwestern.edu> wrote:

> I have taken over the administration of several Tomcat instances. A number
> of these are load balanced using an F5 appliance.  The
> org.apache.catalina.values.AccessLogValve log file is filled with the F5
> polls to see if the application is alive. Under almost all circumstances
> these are useless, I would like to stop logging just these requests. What
> is the best way to stop these entries being written?  I’ve included a
> sample of the log entries.
>
> 10.0.171.163 - - [21/Feb/2020:09:04:11 -0600] "GET /MySite/isAlive.jsp "
> 200 112
> 10.0.171.162 - - [21/Feb/2020:09:04:16 -0600] "GET /MySite/isAlive.jsp "
> 200 112
> 10.0.171.163 - - [21/Feb/2020:09:04:16 -0600] "GET /MySite/isAlive.jsp "
> 200 112
> 10.0.171.162 - - [21/Feb/2020:09:04:21 -0600] "GET /MySite/isAlive.jsp "
> 200 112
> 10.0.171.163 - - [21/Feb/2020:09:04:21 -0600] "GET /MySite/isAlive.jsp "
> 200 112
> 10.0.171.162 - - [21/Feb/2020:09:04:26 -0600] "GET /MySite/isAlive.jsp "
> 200 112
>
> The entry for the log file in server.xml is
>  directory="logs"
>prefix="localhost_access_log" suffix=".txt"
>pattern="%h %l %u %t %r %s %b" />
>
>
> Darryl Baker
> Northwestern University
> darryl.ba...@northwestern.edu
>

 I may be mistaken, but It seems you are probably trying to make the
logging coarser. You can take a look at conf/logging.properties for your
tomcat instances to do the adjustments of log levels.

Regards,

>
>