Re: How to configure logfile and options for mod_jk2
So, how do I shut that up? Any help is appreciated. I already tried stuff like JKLogFile or logger.file, but no real success :-( AFAIK jk2 isn't actively developed any more. Instead, the use of jk is recommended. Docs ar here: http://tomcat.apache.org/connectors-doc/index.html The HowTo contains a configuration example: http://tomcat.apache.org/connectors-doc/howto/apache.html Regards mks thanks a lot. For reasons that I do not understand I always thought that mod_jk2 was the successor to mod_jk. Apparently this is no longer true. The docs are not very clear about the relationships :-( Anyway, mod_jk 1.2.14 works great and does what I want :-) Cheers Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Tomcat 6 Clustering vs. Tomcat 5.5 clustering
Hi, while moving an application from Tomcat 5.5 to Tomcat 6 I found some differences in clustering, that I could not resolve myself: e. G. What happened to replicationMode, ackTimeout waitForAck? In TC 5.5 I had: Sender className=org.apache.catalina.cluster.tcp.ReplicationTransmitter replicationMode=fastasyncqueue ackTimeout=15000 waitForAck=true/ Are they still available with org.apache.catalina.tribes.transport.ReplicationTransmitter? Thanks Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat 6 Clustering vs. Tomcat 5.5 clustering
Hi, while moving an application from Tomcat 5.5 to Tomcat 6 I found some differences in clustering, that I could not resolve myself: e. G. What happened to replicationMode, ackTimeout waitForAck? In TC 5.5 I had: Sender className=org.apache.catalina.cluster.tcp.ReplicationTransmitter replicationMode=fastasyncqueue ackTimeout=15000 waitForAck=true/ Are they still available with org.apache.catalina.tribes.transport.ReplicationTransmitter? Also, can I assume the bindAddress from org.apache.catalina.cluster.mcast.McastService is the same as bind from org.apache.catalina.tribes.membership.McastService? Thanks Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Weird problerm accessing request headers from tomcat
Hi,
sorry for the crosspost, but I am not sure where to ask. I am trying to
understand a weird problem accessing HTTP request headers from a jsf page.
The setup is as follows:
apache1 - apache2 - mod_jk - tomcat
Apache1 is accessible from the Internet and forwards requests to my application
to apache2. I have no direct control over the setup of apache1 and I cannot
look
at the configuration. Apache2 (2.0.63 on Linux) is answering requests from the
intranet and forwards them via mod_jk (1.2.30) to two loadbanced tomcats
(Linux,
6.0.32).
Apache1 inserts the following variables into the requests it forwards to
Apache1. Apache1 can see them, I have checked that using cgi-bin/printenv (some
values anonymized):
HTTP_X_FORWARDED_FOR=aa.bb.cc.dd
HTTP_X_FORWARDED_HOST=xxx.yyy.net
HTTP_X_FORWARDED_PORT=443
HTTP_X_FORWARDED_PROTOCOL=https
HTTP_X_FORWARDED_SERVER=aaa.bbb.ccc
If I try to read those variables from the tomcat application using this
facelett code:
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_HOST: #{header['X_FORWARDED_HOST']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_PROTOCOL: #{header['X_FORWARDED_PROTOCOL']}
/
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_PORT: #{header['X_FORWARDED_PORT']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_FOR: #{header['X_FORWARDED_FOR']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_SERVER: #{header['X_FORWARDED_SERVER']} /
br /
Only X_FORWARDED_PORT and X_FORWARDED_PROTOCOL are set. The other three are
empty/null. It seem that somewhere they get dropped between Apache2, mod_jk and
Tomcat.
Now, I know that _PORT and _PROTOCOL are inserted at Apache1 using
'RequestHeader set X_FORWARDED_PROTOCOL https', while I do not know how the
other three are generated. If I add 'RequestHeader set X_FORWARDED_HOST
xxx.yyy.net' to the configuration from apache2, the application sees that
header as well. Confused? so am I :-(
Happy weekend
Martin
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Weird problerm accessing request headers from tomcat
Hi Martin,
the reverse proxy (gateway) in my case would be apache1 me thinks. apache2
definitely does not use mod_proxy/ProxyPass. It is just loadbalancing two
tomcat
instances using mod_jk.
My problem is (maybe I was not clear) that apache2 does see the
X-Forwarded-For, X-Forwarded-Host and X-Forwarded-Server headers apparently
added by apache1, but that those are not visible in the requests reaching
tomcat. If, just as an experiment, I use RequestHeader set in the apache2
ssl configuration, tomcat sees them. Same is true for the _PORT and
_PROTOCOL headers that are added on apache1 via RequestHeader set.
Did I express myself better this time? There must be some handling differences
between headers added from mod_proxy (if that is used by apache1 I will not
find
out before some time next week) and those added by mod_headers.
Thanks
Martin :-)
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
- Original Message
From: Martin Gainty mgai...@hotmail.com
To: Tomcat Users List users@tomcat.apache.org
Sent: Fri, May 27, 2011 3:29:32 PM
Subject: RE: Weird problerm accessing request headers from tomcat
when your Apache2 is configured as reverse-proxy you are fowarding
IP,RequestedHost and Proxy-Server specifically:
When acting in a reverse-proxy mode (using the ProxyPass directive, for
example),
mod_proxy_http adds several request headers in
order to pass information to the origin server. These headers
are:
X-Forwarded-ForThe IP address of the client.X-Forwarded-HostThe original
host requested by the client in the Host
HTTP request header.X-Forwarded-ServerThe hostname of the proxy
server.
http://httpd.apache.org/docs/2.2/mod/mod_proxy.html
is this not the case?
Martin
__
Date: Fri, 27 May 2011 05:53:14 -0700
From: spamt...@knobisoft.de
Subject: Weird problerm accessing request headers from tomcat
To: users@tomcat.apache.org; us...@httpd.apache.org
Hi,
sorry for the crosspost, but I am not sure where to ask. I am trying to
understand a weird problem accessing HTTP request headers from a jsf page.
The setup is as follows:
apache1 - apache2 - mod_jk - tomcat
Apache1 is accessible from the Internet and forwards requests to my
application
to apache2. I have no direct control over the setup of apache1 and I
cannot
look
at the configuration. Apache2 (2.0.63 on Linux) is answering requests from
the
intranet and forwards them via mod_jk (1.2.30) to two loadbanced tomcats
(Linux,
6.0.32).
Apache1 inserts the following variables into the requests it forwards to
Apache1. Apache1 can see them, I have checked that using cgi-bin/printenv
(some
values anonymized):
HTTP_X_FORWARDED_FOR=aa.bb.cc.dd
HTTP_X_FORWARDED_HOST=xxx.yyy.net
HTTP_X_FORWARDED_PORT=443
HTTP_X_FORWARDED_PROTOCOL=https
HTTP_X_FORWARDED_SERVER=aaa.bbb.ccc
If I try to read those variables from the tomcat application using this
facelett code:
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_HOST: #{header['X_FORWARDED_HOST']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_PROTOCOL:
#{header['X_FORWARDED_PROTOCOL']}
/
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_PORT: #{header['X_FORWARDED_PORT']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_FOR: #{header['X_FORWARDED_FOR']} /
br /
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_SERVER:
#{header['X_FORWARDED_SERVER']}
/
br /
Only X_FORWARDED_PORT and X_FORWARDED_PROTOCOL are set. The other three
are
empty/null. It seem that somewhere they get dropped between Apache2,
mod_jk
and
Tomcat.
Now, I know that _PORT and _PROTOCOL are inserted at Apache1 using
'RequestHeader set X_FORWARDED_PROTOCOL https', while I do not know how
the
other three are generated. If I add 'RequestHeader set X_FORWARDED_HOST
xxx.yyy.net' to the configuration from apache2, the application sees
that
header as well. Confused? so am I :-(
Happy weekend
Martin
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Weird problerm accessing request headers from tomcat
Hi Andre,
see below. You are not going to hear me slapping my face, but definitely doing
a! :-)
Cheers
Martin
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
- Original Message
From: André Warnier a...@ice-sa.com
To: Tomcat Users List users@tomcat.apache.org
Sent: Fri, May 27, 2011 6:01:28 PM
Subject: Re: Weird problerm accessing request headers from tomcat
Hi.
I believe that you are making the often-made confusion between environment
values (or variables), and HTTP headers content.
In particular, here :
seems you are right.
Apache1 inserts the following variables into the requests it forwards to
Apache1 (I suppose you meant Apache2 here)
No. It does not do that. It adds some HTTP headers. This is different, see
below.
. Apache1 (I suppose you meant Apache2 here) can see them, I have checked
that
using cgi-bin/printenv (some
values anonymized):
HTTP_X_FORWARDED_FOR=aa.bb.cc.dd
HTTP_X_FORWARDED_HOST=xxx.yyy.net
HTTP_X_FORWARDED_PORT=443
HTTP_X_FORWARDED_PROTOCOL=https
HTTP_X_FORWARDED_SERVER=aaa.bbb.ccc
Your check does not show that at all. It shows something that is just
confusing enough to get you confused as to what you are seeing. ;-)
But you have excuses for your confusion, because the Apache documentation
itself is very confusing as to environment variables.
Indeed, the documentation leaves this pretty diffuse.
What the cgi-bin script sees, are indeed environment values.
These are set by the Apache process (Apache2), just before it executes the
cgi-bin script. So the cgi-bin script sees them in its environment when it
runs.
(like with $ENV{'HTTP_X_FORWARDED_PORT'})
But there is no one-to-one relationship between what Apache finds in HTTP
request headers, and the environment values which it sets for the cgi-bin
scripts that it runs.
Apache does convert some of the request HTTP header values into cgi-bin
environment variables, but :
- the name of the environment variable may be different from the
corresponding
HTTP header label (you see this yourself above : a HTTP header named
X-forwarded-for: has been passed to the cgi-bin script as the environment
value named HTTP_X_FORWARDED_FOR)
- not all HTTP headers are converted and passed that way
- some environment values passed to the cgi-bin script are not, and never
were, HTTP headers of the request (for example, the cgi-bin environment
values
QUERY_STRING, or SCRIPT_FILENAME)
Ok, that definitely explains what I am seeing.
On the other hand :
When a HTTP proxy server forwards a HTTP request to another HTTP server via
the HTTP protocol, it forwards *all* the request headers and request content
to
this other server, as a HTTP request (otherwise, it would not be a valid HTTP
proxy server). But it cannot forward environment values, because there is
no
defined way of doing this over the HTTP protocol. (*)
But now I see your second post, and your problem is in fact much simpler.
By doing this :
h:outputText style=font: bold 14px sans-serif;
value=X_FORWARDED_HOST: #{header['X_FORWARDED_HOST']} /
what you are actually trying to retrieve, is the content of the HTTP request
header
X_FORWARDED_HOST: (I guess), but this HTTP header does not exist in the
request.
What you are giving as a HTTP header name, is actually what the cgi-bin
environment value name was for your cgi-bin.
Which, as I try to explain at long length above, is not the same thing.
So you get back a null, and you think that the header was not there.
But it is there, only under its real HTTP header name.
Try something like
value=X_FORWARDED_HOST: #{header['X-Forwarded-for']} /
instead.
Yup, using X-Forewarded-Host works as expected. Thanks a big lot.
(Noise of self-slap on face ?).
As I said above: no self slapping. Just amazement on how much there is still to
learn after all these years :-)
(*) However, when the proxy protocol used is AJP (as it is between Apache
and
Tomcat when using the mod_jk connector, or the mod_proxy_ajp connector),
/then/
some additional values /can/ be passed along with the request (because the
AJP
protocol allows that). On the Tomcat side, these then appear as request
attributes which the webapp can retrieve (via request.getAttribute(name)),
but
not as environment values of the Tomcat process for example.
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6 Clustering vs. Tomcat 5.5 clustering
Hi, I almost forgot about this one :-) - Original Message From: Filip Hanik - Dev Lists devli...@hanik.com To: Tomcat Users List users@tomcat.apache.org Sent: Tue, June 7, 2011 1:07:20 AM Subject: Re: Tomcat 6 Clustering vs. Tomcat 5.5 clustering On 5/23/2011 6:50 AM, Martin Knoblauch wrote: Hi, while moving an application from Tomcat 5.5 to Tomcat 6 I found some differences in clustering, that I could not resolve myself: e. G. What happened to replicationMode, ackTimeout waitForAck? In TC 5.5 I had: Sender className=org.apache.catalina.cluster.tcp.ReplicationTransmitter replicationMode=fastasyncqueue ackTimeout=15000 waitForAck=true/ channelSendOptions http://tomcat.apache.org/tomcat-6.0-doc/config/cluster.html I know those pages. They are not that much helpful (IMO) in the transition. What I am looking for is a kind of side-by-side comparison of what changed between 5.5 and 6.0. Any pointers are welcome. Are they still available with org.apache.catalina.tribes.transport.ReplicationTransmitter? Also, can I assume the bindAddress from org.apache.catalina.cluster.mcast.McastService is the same as bind from org.apache.catalina.tribes.membership.McastService? yes http://tomcat.apache.org/tomcat-6.0-doc/config/cluster-membership.html OK, I figured that out. Cheers Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: jk 1.2.36 throwing 503/sendfull/cping errors
On Fri, May 25, 2012 at 6:45 AM, Mladen Turk mt...@apache.org wrote: On 05/24/2012 09:40 PM, Anthony J. Biacco wrote: I'm still puzzled as to why this behavior just changed between .35 and .36 OK, but if you follow the recommended configuration by making sure that workers which are members of lb are not listed inside worker.list, does it works? The question is, is this a recommendation or a restriction that should be enforced? Maybe a warning about this, apparently common, [mis-]configuration should be issued at module configuration time? Cheers Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: jk 1.2.36 throwing 503/sendfull/cping errors
On Fri, May 25, 2012 at 1:29 PM, Mladen Turk mt...@apache.org wrote: On 05/25/2012 11:28 AM, Martin Knoblauch wrote: On Fri, May 25, 2012 at 6:45 AM, Mladen Turkmt...@apache.org wrote: On 05/24/2012 09:40 PM, Anthony J. Biacco wrote: I'm still puzzled as to why this behavior just changed between .35 and .36 OK, but if you follow the recommended configuration by making sure that workers which are members of lb are not listed inside worker.list, does it works? The question is, is this a recommendation or a restriction that should be enforced? Maybe a warning about this, apparently common, [mis-]configuration should be issued at module configuration time? Hmm, I still didn't get the response if it works in case member workers are not in the worker.list. I'm not the OP, although I believe he wrote so. But you are right. Before 1.2.36 we just created another worker slot in shared memory. Now it's allocated 'by name' so it can create problems if the same name is declared both as standalone ajp13 worker and load balancer member. This would explain the changed behaviour. As I said, if this is a non-no with 1.2.36+, there should be a watning in the logs, or an outright refusal to load. I'm sure that once we had 'must not be in worker.list', but someone changed that to 'should' inside http://tomcat.apache.org/connectors-doc/reference/workers.html (see balance_workers directive) Probably needs to be changed back, or a strong warning added. Cheers Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat uses 99% of CPU - Something to do with AJP connector
On Mon, Jun 11, 2012 at 10:37 PM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: Dharamshila Khandelwal [mailto:dharmshil...@gmail.com] Subject: Re: Tomcat uses 99% of CPU - Something to do with AJP connector I cannot upgrade Tomcat because we upgraded last year. Now that is a completely bogus reason. From a technical point of view - yes. But from experience I can tell that you there are IT organisations out there that have really weird policies. So upgrade middleware only once a year is not unthinkable :-( Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat uses 99% of CPU - Something to do with AJP connector
On Tue, Jun 12, 2012 at 7:59 AM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: Martin Knoblauch [mailto:knobis...@googlemail.com] Subject: Re: Tomcat uses 99% of CPU - Something to do with AJP connector From a technical point of view - yes. But from experience I can tell that you there are IT organisations out there that have really weird policies. So upgrade middleware only once a year is not unthinkable I didn't say (or imply) that it was unthinkable - just stupid. What I wanted to say is that the original poster may not have full control over the environment s/he is working in. Cheers Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Issue starting catalina.sh on Windows for tomcat 7 version
On Tue, Aug 28, 2012 at 1:17 PM, André Warnier a...@ice-sa.com wrote: irfan pasha wrote: Well i have alternative of starting tomcat based on the OS. But i specifically want to use catalina.sh for both windows and unix operating systems. What you want, and what is possible and/or supported, may be different things. For the past 4 years i am able to use it, but not only with this version. What is the major change that was done with this version. why does catalina.sh does'nt get started with this version on windows. I don't know. But just look at catalina.sh, lines 100-108 fo example. I am not sure what will result there when running this from a MKS bash shell console under Windows. (On my XP laptop under the MKS sh, uname returns Windows_NT). Probably not many people on this list would be sure either, as this is definitely not a common environment to run Tomcat, and probably not something that is ever tested. Maybe the fact that it was running with previous versions was just a lucky coincidence. By the way, if (from a MKS shell command window) I run bin/startup.sh I get the same result as you, but if I run bin/startup.bat, Tomcat starts up fine. (That is with Tomcat 7.0.21, Windows full version). regardless whether the request makes any sense - I would do a diff between the working and non working shell scripts and see where it leads me to :-) Cheers Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Test - please ignore
Just a test after moving from googlemail to gmail. I am kind of afraid that I broke my subscription ... Thanks Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Find out which sessions are connected to a specific Tomcat in a HA/Failover setup
Hi, this is more of a principal question. We operate the following HA/Failover Setup: Frontend: Apache 2.2.21 + mod_jk 1.2.32 Backend: 6 x Tomcat 7.0.32 We have two types of clients. First there are interactive clients that connect via a browser. Second we have batch clients that are run via a queuing system (SGE) on a cluster of Linux servers. Those batch jobs do the dirty work like transferring mass data and other long running stuff that we do not want to run on the backends themselves. Now sometimes the clients get into trouble and we need to find out what is going on. First step is to look into the tomcat log files. For that of course we need to know which of the instances is serving the client (we do session persistence, so the clients stay on their servers unless there is a failover). For the interactive clients this is usually easy. We can have the users look at the jvmroute part of the session cookie. The problem arises on the batch clients. How to ask them? :-) So, the question is whether there is a way of finding out from the outside which tomcat a client-session is talking to? The connections are of course all through the Apache frontend, which does not make it easier. Any insight (or pointer to the right manual) would be highly appreciated. Ah yes, modifying the application to just log the cookie is of course possible, but for reasons, not to be discussable here, political difficult. Thanks in advance Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Find out which sessions are connected to a specific Tomcat in a HA/Failover setup
On Thu, Feb 21, 2013 at 8:16 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Martin, On 2/21/13 8:11 AM, Martin Knoblauch wrote: this is more of a principal question. We operate the following HA/Failover Setup: Frontend: Apache 2.2.21 + mod_jk 1.2.32 Backend: 6 x Tomcat 7.0.32 Sticky sessions or not? Sticky. Thought I had written it below. We can have the users look at the jvmroute part of the session cookie. The problem arises on the batch clients. How to ask them? :-) So, the question is whether there is a way of finding out from the outside which tomcat a client-session is talking to? The connections are of course all through the Apache frontend, which does not make it easier. How can you identify a client in general? Do they maintain sessions? If so, how do they maintain them (e.g. cookie-based, URL-based, or no session state)? Obviously, if there is no session state, then there is no backend serving them because they'll just use whatever backend they get for their individual requests. Lots of session state, cookie based. Due to the stickiness, we can assume that the client/server relationship is stable once the session is established (not counting failover here, but that is not an big issue). Any insight (or pointer to the right manual) would be highly appreciated. Ah yes, modifying the application to just log the cookie is of course possible, but for reasons, not to be discussable here, political difficult. How about modifying the clients? Same not to be discussed here political problem :-( The whole application (server and clients) is closed commercial code. Thanks Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Find out which sessions are connected to a specific Tomcat in a HA/Failover setup
On Fri, Feb 22, 2013 at 10:10 AM, Mark Thomas ma...@apache.org wrote: On 22/02/2013 08:59, Martin Knoblauch wrote: Same not to be discussed here political problem :-( The whole application (server and clients) is closed commercial code. Can you modify anything? For example, you could add client IP and session ID to the access log for httpd. If your batch clients have unique IPs that will give you the jvm route that an IP is using. While this might not be exactly what you need some variation of it might work given that you can log pretty much anything from an HTTP header. Hi Mark, OK. I will try looking at extending the access log. Sounds promising. Thanks Martin - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: mod_jk worker activation not working anymore in 1.2.37?
Any plans when 1.2.38 will be released? Thanks Martin On Mon, Jun 10, 2013 at 10:20 PM, Rainer Jung rainer.j...@kippdata.dewrote: On 10.06.2013 17:29, Konstantin Kolinko wrote: 2013/6/10 David Gubler d...@doodle.com: Hi list, We have recently upgraded our Apache servers from Debian Squeeze to Wheezy (from Apache 2.2.16 with mod_jk 1.2.30 to Apache 2.2.22 with mod_jk 1.2.37). The Tomcat version hasn't changed (7.0.37). We often do rolling releases by disabling (DIS) some worker in jkmanager, waiting for a few minutes for most sessions to go away (we use sticky sessions but not forced), upgrading it, and re-enabling it. This worked perfectly with mod_jk 1.2.30. The server is rather busy (order of 100-200 req/s going to tomcat). However, with mod_jk 1.2.37, the activation state behaves erratically. Say I disable worker1 on all apache servers via jkmanager. When I go back to the jkmanager overview screen, it still shows as active. I hit reload, now it shows as disabled. I can wait for a few seconds or minutes, reload, and suddenly it shows up as active again! It keeps switching back and forth between active and disabled if I reload often enough. Afterwards I usually have to set it to active a few times to make it stick there. This happens on all apache servers independently. And more worringly, the load on the worker does not decrease, not even after waiting for half an hour or longer (with 1.2.30, the load on a worker decreased to about 5% after 5-10 minutes). When I set a worker to stopped, the activation state also switches between active and stopped, the load on the worker goes down slowly, but the requests do not cease completely. With 1.2.30, I could set a worker to stopped and it instantaneously received no more requests. Other than that, mod_jk behaves as expected (e.g. if I shut down one of the tomcats, the requests go to the other; load balancing works fine in normal operation). I have stripped down our workers.properties to the bare minimum that we need, and the problem is still there: ps=/ worker.list=loadbalancer,jkstatus worker.jkstatus.type=status worker.loadbalancer.type=lb worker.loadbalancer.sticky_session=true worker.loadbalancer.balance_workers=worker1,worker2 worker.worker1.type=ajp13 worker.worker1.host=WW.XX.YY.ZZ worker.worker1.port=8009 worker.worker1.connect_timeout=7 worker.worker1.prepost_timeout=7 worker.worker1.socket_timeout=70 worker.worker1.connection_pool_timeout=70 worker.worker1.connection_pool_size=200 worker.worker1.retry_interval=1000 worker.worker1.lbfactor=1 [same for worker2, only difference is the IP address] Rest of the configuration is Debian standard. Apache uses JkAutoAlias, JkMount and a bunch of JkUnMounts, but nothing fancy. The changelog does not really give me any clues as to what change could cause this, and neither does the workers.properties documentation :( Does anyone have an idea what I could be doing wrong? Looking at the current changelog, section name=Changes between 1.2.37 and 1.2.38 ... fix Fix status worker not updating parameters for all members. (mturk) /fix That is http://svn.apache.org/viewvc?view=revisionrevision=1354021 Yes that should be it. If the OP compiles himself, just add the tiny patch http://svn.apache.org/viewvc/tomcat/jk/trunk/native/common/jk_status.c?r1=1354021r2=1354020pathrev=1354021 to your mod_jk source before compiling. Regards, Rainer - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: mod_jk worker activation not working anymore in 1.2.37?
I can confirm that the one liner patch fixes the problem for me as well. Due to policies, I have problems to roll this into the productive environment, so a release would be highly welcome even if this is the only fix (so it seems to me). Cheers Martin On Tue, Jun 11, 2013 at 6:42 AM, Rainer Jung rainer.j...@kippdata.dewrote: On 11.06.2013 00:58, Martin Knoblauch wrote: Any plans when 1.2.38 will be released? Not really, but it is overdue. So IMO we should release it during the next few weeks. Regards, Rainer On Mon, Jun 10, 2013 at 10:20 PM, Rainer Jung rainer.j...@kippdata.de wrote: On 10.06.2013 17:29, Konstantin Kolinko wrote: 2013/6/10 David Gubler d...@doodle.com: Hi list, We have recently upgraded our Apache servers from Debian Squeeze to Wheezy (from Apache 2.2.16 with mod_jk 1.2.30 to Apache 2.2.22 with mod_jk 1.2.37). The Tomcat version hasn't changed (7.0.37). We often do rolling releases by disabling (DIS) some worker in jkmanager, waiting for a few minutes for most sessions to go away (we use sticky sessions but not forced), upgrading it, and re-enabling it. This worked perfectly with mod_jk 1.2.30. The server is rather busy (order of 100-200 req/s going to tomcat). However, with mod_jk 1.2.37, the activation state behaves erratically. Say I disable worker1 on all apache servers via jkmanager. When I go back to the jkmanager overview screen, it still shows as active. I hit reload, now it shows as disabled. I can wait for a few seconds or minutes, reload, and suddenly it shows up as active again! It keeps switching back and forth between active and disabled if I reload often enough. Afterwards I usually have to set it to active a few times to make it stick there. This happens on all apache servers independently. And more worringly, the load on the worker does not decrease, not even after waiting for half an hour or longer (with 1.2.30, the load on a worker decreased to about 5% after 5-10 minutes). When I set a worker to stopped, the activation state also switches between active and stopped, the load on the worker goes down slowly, but the requests do not cease completely. With 1.2.30, I could set a worker to stopped and it instantaneously received no more requests. Other than that, mod_jk behaves as expected (e.g. if I shut down one of the tomcats, the requests go to the other; load balancing works fine in normal operation). I have stripped down our workers.properties to the bare minimum that we need, and the problem is still there: ps=/ worker.list=loadbalancer,jkstatus worker.jkstatus.type=status worker.loadbalancer.type=lb worker.loadbalancer.sticky_session=true worker.loadbalancer.balance_workers=worker1,worker2 worker.worker1.type=ajp13 worker.worker1.host=WW.XX.YY.ZZ worker.worker1.port=8009 worker.worker1.connect_timeout=7 worker.worker1.prepost_timeout=7 worker.worker1.socket_timeout=70 worker.worker1.connection_pool_timeout=70 worker.worker1.connection_pool_size=200 worker.worker1.retry_interval=1000 worker.worker1.lbfactor=1 [same for worker2, only difference is the IP address] Rest of the configuration is Debian standard. Apache uses JkAutoAlias, JkMount and a bunch of JkUnMounts, but nothing fancy. The changelog does not really give me any clues as to what change could cause this, and neither does the workers.properties documentation :( Does anyone have an idea what I could be doing wrong? Looking at the current changelog, section name=Changes between 1.2.37 and 1.2.38 ... fix Fix status worker not updating parameters for all members. (mturk) /fix That is http://svn.apache.org/viewvc?view=revisionrevision=1354021 Yes that should be it. If the OP compiles himself, just add the tiny patch http://svn.apache.org/viewvc/tomcat/jk/trunk/native/common/jk_status.c?r1=1354021r2=1354020pathrev=1354021 to your mod_jk source before compiling. Regards, Rainer - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Status of mod_jk-1.2.38
Hi, happy New year. Should be still early enough for this :-) I have one short question: what is the status of mod_jk-1.2.38? It has been over a year since 1.2.37 and I am just curious. I am specially interested in an official fix for *Bug 53762*https://issues.apache.org/bugzilla/show_bug.cgi?id=53762. The patch mentioned there works, but cannot be used due to an only official releases policy at the customer site. On a related note: is mod_jk still the preferred way for HA and load-balancing tomcats? Are there better ways? Our environment is currently httpd/mod_jk/tomcat on Linux. Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Status of mod_jk-1.2.38
On Wed, Jan 8, 2014 at 8:45 PM, Rainer Jung rainer.j...@kippdata.de wrote: On 08.01.2014 18:25, Martin Knoblauch wrote: Hi, happy New year. Should be still early enough for this :-) I have one short question: what is the status of mod_jk-1.2.38? It has been over a year since 1.2.37 and I am just curious. I am specially interested in an official fix for *Bug 53762*https://issues.apache.org/bugzilla/show_bug.cgi?id=53762. The patch mentioned there works, but cannot be used due to an only official releases policy at the customer site. I did some more changes lately due to a user request which need some more testing. I expect that we'll do the 1.2.38 release during the next 4-8 weeks. I have to ask Mladen about the status of his IPv6 changes though. Cool. Thanks for the info. On a related note: is mod_jk still the preferred way for HA and load-balancing tomcats? Are there better ways? Our environment is currently httpd/mod_jk/tomcat on Linux. Better or worse depend on your requirements. I think mod_jk is still working very well and has some unique advantages. OK, good to know. I have no actual issues using mod_jk. It is working well and does what we need. Just the release frequency is a bit low, which is not really a technical thing. But then maybe mod_jk is perfect :-) anyway, do you (or someone on the list) happen to know a good comparison/discussion of the available options? Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: tomcat 7.0.53 error
On Wed, Apr 16, 2014 at 7:03 AM, Philippe Couas pco...@orange.fr wrote: Hi, I want migrate from tomcat 7.0.52 to 7.0.53. My tomcat is launched in standalone mode from command line (linux level3)Currently i have a servlet that create image to disk and only with last version i have following error messageException in thread http-bio-8080-exec-10 java.lang.InternalError: Can't connect to X11 window server using ':0.0' as the value of the DISPLAY variable.Same servlet running perfect with previous version. RegardsPhil Hi Philippe, seems some [new] component wants to talk to the X server. Can you try to add -Djava.awt.headless=true to the JAVA_OPTS when starting Tomcat? We are using that to avoid dependence on X11 in our application. Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Problems with excessive connection timeouts
Hi,
my environment is:
OS SuSE SLES11/SP2 64-bit
HTTP frontend: Apache2-2.2.21 (64-bit)
mod_jk: 1.2.37
Problem short: excessive connection timeouts with weird socket ids
Problem long: we frontend 4 Tomcats (7.0.42) using Apache2(2.2.21) and
mod_jk(1.2.37). We are seeing a heavy amount of connection timeouts when
the tomcats are loaded. Surprisingly the logs show socket id -1, which
should not happen in my opinion, because most actions are guarded by the
IS_VALID_SOCKET macro. Here is one example. The code emmitting the line
should not be reached due to the check in line 1543:
[Mon Jun 16 17:12:05.405 2014] [7233:140275337774848] [info]
ajp_send_request::jk_ajp_common.c (1562): (lpsdm83-1) failed sending
request, socket -1 prepost cping/cpong failure (errno=110)
1543:while (IS_VALID_SOCKET(ae-sd)) {
int err = JK_FALSE;
if (jk_is_socket_connected(ae-sd, l) == JK_FALSE) {
ae-last_errno = errno;
jk_log(l, JK_LOG_DEBUG,
(%s) failed sending request,
socket %d is not connected any more (errno=%d),
ae-worker-name, ae-sd, ae-last_errno);
ajp_abort_endpoint(ae, JK_FALSE, l);
err = JK_TRUE;
err_conn++;
}
if (ae-worker-prepost_timeout 0 !err) {
/* handle cping/cpong if prepost_timeout is set
* If the socket is disconnected no need to handle
* the cping/cpong
*/
if (ajp_handle_cping_cpong(ae,
ae-worker-prepost_timeout, l) == JK_FALSE) {
1562:jk_log(l, JK_LOG_INFO,
(%s) failed sending request,
socket %d prepost cping/cpong failure (errno=%d),
ae-worker-name, ae-sd, ae-last_errno);
/* XXX: Is there any reason to try other
* connections to the node if one of them fails
* the cping/cpong heartbeat?
* Tomcat can be either too busy or simply dead, so
* there is a chance that all other connections would
* fail as well.
*/
err = JK_TRUE;
err_cping++;
}
}
So, before trying to tell me that I am missing workers.properties and
complete logs, I want to check first whether the -1 socket ids are to be
expected here or whether there is some kind of bug involved.
Thanks in advance
Martin
--
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Re: Problems with excessive connection timeouts
Hi Konstantin,
yeah, found out two hours after posting. Embarassing, isn't it.
Thanks
Martin
On Tue, Jun 17, 2014 at 7:14 PM, Konstantin Kolinko knst.koli...@gmail.com
wrote:
2014-06-16 19:44 GMT+04:00 Martin Knoblauch knobis...@gmail.com:
Hi,
my environment is:
OS SuSE SLES11/SP2 64-bit
HTTP frontend: Apache2-2.2.21 (64-bit)
mod_jk: 1.2.37
Problem short: excessive connection timeouts with weird socket ids
Problem long: we frontend 4 Tomcats (7.0.42) using Apache2(2.2.21) and
mod_jk(1.2.37). We are seeing a heavy amount of connection timeouts when
the tomcats are loaded. Surprisingly the logs show socket id -1, which
should not happen in my opinion, because most actions are guarded by the
IS_VALID_SOCKET macro. Here is one example. The code emmitting the line
should not be reached due to the check in line 1543:
[Mon Jun 16 17:12:05.405 2014] [7233:140275337774848] [info]
ajp_send_request::jk_ajp_common.c (1562): (lpsdm83-1) failed sending
request, socket -1 prepost cping/cpong failure (errno=110)
1543:while (IS_VALID_SOCKET(ae-sd)) {
int err = JK_FALSE;
if (jk_is_socket_connected(ae-sd, l) == JK_FALSE) {
ae-last_errno = errno;
jk_log(l, JK_LOG_DEBUG,
(%s) failed sending request,
socket %d is not connected any more (errno=%d),
ae-worker-name, ae-sd, ae-last_errno);
ajp_abort_endpoint(ae, JK_FALSE, l);
err = JK_TRUE;
err_conn++;
}
if (ae-worker-prepost_timeout 0 !err) {
/* handle cping/cpong if prepost_timeout is set
* If the socket is disconnected no need to handle
* the cping/cpong
*/
if (ajp_handle_cping_cpong(ae,
ae-worker-prepost_timeout, l) == JK_FALSE) {
1562:jk_log(l, JK_LOG_INFO,
(%s) failed sending request,
socket %d prepost cping/cpong failure
(errno=%d),
ae-worker-name, ae-sd, ae-last_errno);
/* XXX: Is there any reason to try other
* connections to the node if one of them fails
* the cping/cpong heartbeat?
* Tomcat can be either too busy or simply dead, so
* there is a chance that all other connections would
* fail as well.
*/
err = JK_TRUE;
err_cping++;
}
}
So, before trying to tell me that I am missing workers.properties and
complete logs, I want to check first whether the -1 socket ids are to
be
expected here or whether there is some kind of bug involved.
The ajp_handle_cping_cpong(...) method that is called a few lines
earlier (line 1560) processes some failures by calling
ajp_abort_endpoint(...). That method closes the socket and sets it to
-1.
Thus the socket is not -1 by the start of the loop, but can become
-1 in the process.
Best regards,
Konstantin Kolinko
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
--
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
jkmanager: how to interpret Current number of backend connections
Hi, our environment is (I know, some are old ...): Apache 2.2.21 mod_jk-1.2.40 Tomcat 7.0.42 My question is how to interpret the numbers from jkmanager, expecially the Busy/Max/Con numbers. Busy and Max make sense to me. Busy is always less than Max and Max gets bumped up from time to time. Matches the description. Now, Con seems to grow steadily over time and is not reset when we do a manual Reset of the worker status. Somehow this does not match the description of Current number of backend connections. At least not for me :-) Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: jkmanager: how to interpret Current number of backend connections
On Mon, Jun 30, 2014 at 5:18 PM, Martin Knoblauch knobis...@gmail.com wrote: Hi, our environment is (I know, some are old ...): Apache 2.2.21 mod_jk-1.2.40 Tomcat 7.0.42 My question is how to interpret the numbers from jkmanager, expecially the Busy/Max/Con numbers. Busy and Max make sense to me. Busy is always less than Max and Max gets bumped up from time to time. Matches the description. Now, Con seems to grow steadily over time and is not reset when we do a manual Reset of the worker status. Somehow this does not match the description of Current number of backend connections. At least not for me :-) Cheers Martin OK, seems the continuous growing of the Current number of backend connections numbers is related to having timeouts configured. It also seems to have started between mod_jk-1.2.32 and mod_jk-1.2.37. I can see some heavy redo of the handling of the connected counter. In any case, the following configuration seems to trigger the behaviour: worker.template.type=ajp13 worker.template.lbfactor=1 worker.template.prepost_timeout=15000 worker.template.connect_timeout=15000 worker.template.ping_timeout=15000 worker.template.ping_mode=A worker.template.connection_ping_interval=180 worker.template.retries=3 worker.template.connection_pool_timeout=30 Without setting the timeout values, the displayed number of backend connections seems to be stable. I suspect a missing decrement somewhere, just hard to say where. Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
New warning at Tomcat Startup
Hi, I am using JDK 1.7.0_67 and Tomcat 7.0.55. At application startup I am seeing a bunch of below messages flying by. The messages have not been observed with Tomcat 7.042. There are exactly 135 of them, which is the sum of the jars in $CATALINA_HOME/lib and my applications WEB-INF/lib. Two questions: 1) Has something changed between 7.0.42 and 7.0.55, that could cause this 2) How can one find out which jars are problematic 3) Can one stifle the message without modifying all 135 jars ### Sep 04, 2014 4:23:21 PM java.util.jar.Attributes read WARNING: Duplicate name in Manifest: Depends-On. Ensure that the manifest does not have duplicate entries, and that blank lines separate individual sections in both your manifest and in the META-INF/MANIFEST.MF entry in the jar file. ### TIA Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: New warning at Tomcat Startup
On Thu, Sep 4, 2014 at 4:58 PM, Mark Thomas ma...@apache.org wrote: On 04/09/2014 15:38, Martin Knoblauch wrote: Hi, I am using JDK 1.7.0_67 and Tomcat 7.0.55. At application startup I am seeing a bunch of below messages flying by. The messages have not been observed with Tomcat 7.042. There are exactly 135 of them, which is the sum of the jars in $CATALINA_HOME/lib and my applications WEB-INF/lib. Two questions: I count three questions, not two. Yeah, I cannot count to 3 ... 1) Has something changed between 7.0.42 and 7.0.55, that could cause this Nothing jumps out at me in the changelog. It seems it happens between 7.0.42 and 7.0.47. I would bisect, but cannot find any tarballs between those two releases. 2) How can one find out which jars are problematic Not easily. Your best bet would be to write a short app that parsed the Manifest of each Jar file in turn. Have to think about it. Time you know. But thanks. 3) Can one stifle the message without modifying all 135 jars Change the log level for java.util.jar to ERROR. Not really desirable. The warning shows that something is fishy in my setup. Thanks Martin Mark ### Sep 04, 2014 4:23:21 PM java.util.jar.Attributes read WARNING: Duplicate name in Manifest: Depends-On. Ensure that the manifest does not have duplicate entries, and that blank lines separate individual sections in both your manifest and in the META-INF/MANIFEST.MF entry in the jar file. ### TIA Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: New warning at Tomcat Startup
On Fri, Sep 5, 2014 at 12:57 AM, Konstantin Kolinko knst.koli...@gmail.com wrote: 2014-09-04 19:55 GMT+04:00 Martin Knoblauch knobis...@gmail.com: It seems it happens between 7.0.42 and 7.0.47. I would bisect, but cannot find any tarballs between those two releases. Those versions have been votes as broken and not released. OK. Fair enough :-) In the meantime I solved the issue: adding absolute-ordering / to WEB-INF/web.xml silenced the warnings. As this is recommended anyway, I am fine with it. From all the jars that we ship, only one has Depends-On lines: jcchart363J.jar. It seems to be notorious for problems and I will check with our product development whether we can remove it completely. Adding it to the skip list in catalina.properties also removes the messages. But yes, the jar contains duplicate Depends-On lines, all in AFAICT separate Name: section (all separated by blank lines). There must be still something added after 7.0.42 that throws the warnings. But that may actually be the right behavior. Who am I to tell :-) Thanks for the support Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [OT] Tomcat 8 encoding issues: unable to change the default encoding iso-8859-1 to utf-8 in http header
On Sat, Jan 24, 2015 at 1:52 PM, André Warnier a...@ice-sa.com wrote: Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 André, ... Morality : in web applications, always specify the content-type (and character set, if applicable) of what you are returning. To André: the word you are looking for is Moral, not Morality. A moral is the essential message of a story, while morality is being able to tell the difference between right and wrong. ;) Thanks for the vocabulary note. I my defense, I would say that this family of English terms is a bit confusing, for a native French-speaker. The English terms moral* are originally undoubtedly imported from the French language (William the Conqueror was after all a Frenchman). But they kind of messed it up during the import procdure, since in French : moral can be a noun or an adjective. As a noun (le moral), it means what in English is the morale (like ..of the troops). As a masculine adjective, it means that which is moral/ethical. And that adjective also has a corresponding feminine form morale. So you would say : - ce conte (masculine) est très moral (this tale is very moral/ethical) but - cette fable (feminine) est très morale (this fable is very moral/ethical) On the other hand, la morale (feminine noun) is what in English would be translated as the morality (or the ethics). And to top it all, the English the moral of this story would be best translated in French as la moralité de cette histoire. A bit confusing, he ? There are many such things between different - but related - languages, some of them often leading to amusing situations. For example, the word in spanish for suffering from a cold is constipado, which is very close to the french word for being incapable to go to the toilet (constipé). And the English adjective embarassed (also imported from the French embarassé(e), and with the same meaning), is very close to the Spanish embarassada, which means pregnant. This all just in case someone was wondering about my interest in character set issues on the WWW.. Ant then there are the people who think German is a difficult language ... :-) Cheers Martin
Tomcat-7 and Oracle Universal Connection Pool (UCP)
Hi, short question. I am in the process of moving an application from using the Oracle JDBC pool to using UCP. I have some problems understanding the relations between the resource definition in the context and what to put into web.xml. So, the context includes: Resource name=jdbc/myDataSource auth=Container factory=oracle.ucp.jdbc.PoolDataSourceImpl type=oracle.ucp.jdbc.PoolDataSource description=Oracle datasource using UCP connectionFactoryClassName=oracle.jdbc.pool.OracleDataSource initialPoolSize=25 minPoolSize=15 maxPoolSize=210 maxStatements=100 connectionWaitInterval=200 timeoutCheckInterval=600 inactiveConnectionTimeout=100 abandonConnectionTimeout=600 validateConnectionOnBorrow=true sqlForValidateConnection=select 1 from DUAL connectionPoolName=MyUCPPool connectionProperties=(defaultRowPrefetch=200) url=jdbc:oracle:thin:@connect_info user=user password=passwd / What is not clear to my, what I have to put into the resource-ref in web.xml. For the old JDBC configuration we had: resource-ref descriptionMy Data Source/description res-ref-namejdbc/myDataSource/res-ref-name res-typeoracle.jdbc.pool.OracleDataSource/res-type res-authContainer/res-auth /resource-ref The res-type matched the type= from the resource definition. Unfortunately, I never found an explanation/documentation of what the res-type actually relates to. So now I see three possibilities: 1) match type= from the resource res-typeoracle.ucp.jdbc.PoolDataSource/res-type 2) match connectionFactoryClassName= from the resource res-typeoracle.jdbc.pool.OracleDataSource/res-type 3) just put in javax.sql.DataSource as someone suggested to me off-line res-typejavax.sql.DataSource/res-type Now I know that I could just experiment with that. But time is scarce and maybe someone knows the correct (or at least working :-) solution. Thanks in advance Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [COMMERCIAL] Re: Need Help: - jk doesn't work after upgrade to 1.2.40 from 1.2.23
Hi Eric, there are two things different from *my* working "mod_jk/1.2.41" setup: a) I have only the "JkMount /xxx/* xxx" line in my configuration b) in the workers list I have only the loadbalancer and the management workers listed, not the individual ones. Not sure how relevant this is Martin On Mon, Mar 14, 2016 at 8:24 AM, ZHAO Eric <eric.z...@alcatel-lucent.com> wrote: > Dear Andre, > > Thanks for your response! > We didn't use Virtual Host in our setting. I re-read the documentation > and didn't find anything wrong with the setting, also this setting worked > before. Do you have several minutes to check the setting in our server for > mod_jk? Appreciate for your time. > > Best Regards, > Eric. > > -Original Message- > From: André Warnier (tomcat) [mailto:a...@ice-sa.com] > Sent: Saturday, March 12, 2016 10:18 PM > To: users@tomcat.apache.org > Subject: [COMMERCIAL] Re: Need Help: - jk doesn't work after upgrade to > 1.2.40 from 1.2.23 > > On 12.03.2016 15:04, ZHAO Eric wrote: > > Hello, > > > > I am new to Tomcat Connectors, we are trying to upgrade the existing > mod_jk 1.2.23 to 1.2.40 for IPV6 configuration. > > > > But we always got the following error in jk.log with 1.2.40, we don't > know if anything need to adjust after upgrade to the new version: > > [Thu Mar 10 17:45:10.790 2016] [13878:140261127514080] [debug] > > jk_translate::mod_jk.c (3855): missing uri map for > > IBM001OAM01:/sso/lsm/lsm.jnlp [Thu Mar 10 17:45:10.790 2016] > > [13878:140261127514080] [debug] jk_map_to_storage::mod_jk.c (4023): > > missing uri map for IBM001OAM01:/sso/lsm/lsm.jnlp > > > > Can some one help me out from this issue? Appreciated in advance, the > following are the setting, we don't have uriworkermapping.properties file. > > > > Here is our setting for mod_jk: > > mod_jk.conf: > > > >LoadModule jk_module /usr/lib64/httpd/modules/mod_jk.so > > > > > > JkWorkersFile /etc/httpd/conf/workers.properties > > JkLogFile /var/log/jk.log > > # JkLogLevel debug > > JkLogLevel warning > > > > JkMount /MIBS ajp12 > > JkMount /MIBS/* ajp12 > > ... > > JkMount /sso csajboss > > JkMount /sso/* csajboss > > > > > > workers.properties: > > worker.list=ajp12,soapnbi,csajboss,csawebsso,loadbalancer,cfmaplayer1, > > cfmaplayer2,cfmaplayer3 > > > > worker.ajp12.port=8007 > > worker.ajp12.host=localhost > > worker.ajp12.type=ajp12 > > > > # Added for SOAP NBI > > worker.soapnbi.port=8009 > > worker.soapnbi.host=localhost > > worker.soapnbi.type=ajp13 > > > > # Added for CSA - JBOSS > > worker.csajboss.port=8011 > > worker.csajboss.host=c04s02h02IBM2 > > worker.csajboss.type=ajp13 > > > > #// next are lb related. > > > > Does this happen in an Apache httpd VirtualHost ? > If yes, make sure that you re-read the configuration documentation at > http://tomcat.apache.org/connectors-doc/reference/apache.html > and in particular, the sections about JkMount and JkMountCopy. > > > ----- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Getting application root path before servlet is initialized?
Hi, is there a way to find the absolute path of the application root before the servlet is initialized? Alternatively: is there a way to defer the initialization of a datasource until the servlet is initialized? Background: I have extended "org.apache.tomcat.jdbc.pool.DataSourceFactory" to automatically set credentials so that they are not stored in the "Catalina/localhost/XXX.xml" file. Instead they are taken from encrypted values in a file below the application root. Works fine if I know that path at "createDataSource" time. In order to avoid hard coding that path, I need a programmatic to find that value. Unfortunately the datasource is initialized before the servlet, so "getRealPath()" is not working yet. Environment is Tomcat 8 plus JDK 8. Plus an commercial application that I do not want to name :-) Thanks in advance Martin -- ---------- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Getting application root path before servlet is initialized?
On Tue, Feb 21, 2017 at 8:55 PM, Mark Thomas <ma...@apache.org> wrote: > On 21/02/2017 13:31, Martin Knoblauch wrote: > > Hi, > > > > is there a way to find the absolute path of the application root before > > the servlet is initialized? > > > > Alternatively: is there a way to defer the initialization of a datasource > > until the servlet is initialized? > > > > Background: I have extended "org.apache.tomcat.jdbc.pool. > DataSourceFactory" > > to automatically set credentials so that they are not stored in the > > "Catalina/localhost/XXX.xml" file. Instead they are taken from encrypted > > values in a file below the application root. Works fine if I know that > path > > at "createDataSource" time. > > And the decryption key for that file is stored where? > > https://wiki.apache.org/tomcat/FAQ/Password > > Thanks for link. It clearly reflects my opinion as well, but the customer demand is: - no plain-text credentials (Big multinational company security policies - fight them if you need the fun). And yes, this is all about making auditors happy - minimize the locations where credentials are stored. This is only lightly related to the decrypt issue. Having to store identical stuff in more than one place is opening up all other sorts of practical issues So, yes - any mechanism that can decrypt needs to store the key somewhere and this just shifts away the problem from securing one item to securing another one. In my case the application (that I will not reveal here) stores encrypted DB credentials in its configuration and provides an API to retrieve them decrypted. I guess, the key is somewhere in the source code (likely obfuscated to prevent casual hacking by debugging). the less I know ... :-) > In order to avoid hard coding that path, I need a programmatic to find > that > > value. Unfortunately the datasource is initialized before the servlet, so > > "getRealPath()" is not working yet. > > > > Environment is Tomcat 8 plus JDK 8. Plus an commercial application that I > > do not want to name :-) > > Ignoring what I suspect is a fundamental flaw in this plan, you probably > want a ServletContextListener and contextInitialized() > > Thanks again for the hint. Will have a look. In the meanwhile I found a way by looking at this.getClass().getProtectionDomain().getCodeSource().getLocation().getPath(); Adding some assumptions about the classpath (which are required to be true in this whole context) this gives me the needed information :-) Thanks Martin > Mark > >
Re: Getting application root path before servlet is initialized?
On Fri, Feb 24, 2017 at 6:06 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 2/22/17 5:19 AM, Martin Knoblauch wrote: > > On Tue, Feb 21, 2017 at 8:55 PM, Mark Thomas <ma...@apache.org> > > wrote: > > > >> On 21/02/2017 13:31, Martin Knoblauch wrote: > >>> Hi, > >>> > >>> is there a way to find the absolute path of the application > >>> root before the servlet is initialized? > >>> > >>> Alternatively: is there a way to defer the initialization of a > >>> datasource until the servlet is initialized? > >>> > >>> Background: I have extended "org.apache.tomcat.jdbc.pool. > >> DataSourceFactory" > >>> to automatically set credentials so that they are not stored > >>> in the "Catalina/localhost/XXX.xml" file. Instead they are > >>> taken from encrypted values in a file below the application > >>> root. Works fine if I know that > >> path > >>> at "createDataSource" time. > >> > >> And the decryption key for that file is stored where? > >> > >> https://wiki.apache.org/tomcat/FAQ/Password > >> > >> > > Thanks for link. It clearly reflects my opinion as well > > Good. At least you know this is all a farce. > Not sure I'd call it a farce, but yes this is all hide and seek > > > , but the customer demand is: > > > > - no plain-text credentials (Big multinational company security > > policies - fight them if you need the fun). And yes, this is all > > about making auditors happy > > Obviously, you are still failing this requirement. The only > requirement you are satisfying is "no plain-text credentials in a > standard configuration file". What you are doing is moving the > plain-text credentials into a non-standard configuration file. > > No, there are no plain-text credentials stored anywhere. They are stored crypted with an api to decrypt them. But of course - how good is the decryption key protected - what about the in-memory copy of the credentials once the datasource has been initialized - what about snooping them on the network when the DB connection is built. OK. We are using SSL there. This makes most auditors and penetration testers sufficiently happy. > > - minimize the locations where credentials are stored. This is > > only lightly related to the decrypt issue. Having to store > > identical stuff in more than one place is opening up all other > > sorts of practical issues > > This is a reasonable requirement, as it helps reduce the attack > surface. But when the attack surface is "a file on the disk", getting > owned means you are owned, regardless of the location of the file(s). > > As for the location of the secrets file, would it be possible to store > it *outside* of the web application's on-disk footprint? That will in > fact make you more secure. Let's say for example that a vulnerability > exists in the DefaultServlet, or one of your application's own > servlets. It allows path-traversal or whatever. A file living in your > application will then be potentially remotely-fetchable :( If you move > that file outside of the web application, you have a better change of > preventing that kind of thing. > There is no secrets file. As I said before - the app has obfuscated the key deep in the source... Thanks Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Getting application root path before servlet is initialized?
On Fri, Feb 24, 2017 at 6:00 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 2/21/17 8:31 AM, Martin Knoblauch wrote: > > Hi, > > > > is there a way to find the absolute path of the application root > > before the servlet is initialized? > > > > Alternatively: is there a way to defer the initialization of a > > datasource until the servlet is initialized? > > > > Background: I have extended > > "org.apache.tomcat.jdbc.pool.DataSourceFactory" to automatically > > set credentials so that they are not stored in the > > "Catalina/localhost/XXX.xml" file. Instead they are taken from > > encrypted values in a file below the application root. Works fine > > if I know that path at "createDataSource" time. > > Where are you configuring your ? In conf/server.xml or in > your application's META-INF/context.xml file? > conf/Catalina/localhost/XXX.xml > > > In order to avoid hard coding that path, I need a programmatic to > > find that value. Unfortunately the datasource is initialized before > > the servlet, so "getRealPath()" is not working yet. > > getRealPath is a bad idea. Also, your DataSources will be > fully-configured before any servlets are initialized, so it's too late. > > Correct :-( That is the problem I need(ed) to solve. Given enough assumptions about the deployment rules for this app, I was able to find a reliable way to deduce the AppRoot. But the fact that the DS is initialized before the/any servlet is still ugly. Thanks Martin
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi everyone, just in case the "final" solution is of interest: the problem was as usual in the configuration. We did not set the following directive for the LDAP connection pool: LDAPConnectionPoolTTL #seconds If the directive is missing, a value of "-1" is implied, meaning "keep connections open for ever". The LDAP server on the other side sets an "idle connection timeout" of 600 seconds. As a result a lookup would fail if it happened 600+ seconds after the first usage of the connection. 600 seconds is exactly the lifetime of the LDAP cache. Given the time of the year, usage of the test/integ/devel environment is minimal and there were no "new" lookups during the cache lifetime, leading to the repeated failures... Setting LDAPConnectionPoolTTL 60 solved the problem for good. Happy New Year !!! Martin On Fri, Dec 30, 2016 at 12:33 AM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 12/29/16 3:47 AM, Martin Knoblauch wrote: > > that is an interesting pointer. We are of course securing the > > "jkmanager" app. And guess what we are using: LDAP. The funky thing > > is that it is working most of the time. It fails just after some > > time. Refreshing the URL cures it again - for some time. What did > > you do to fix your problem? > > I'm glad to see you are on your way to solving your problem. > > In my case, it was an expired TLS certificate being used for the > OpenLDAP process or something similar, so it wasn't anything to do > with httpd itself. I've also been experimenting with a fall-back for > LDAP that maybe wouldn't be 100% up-to-date with the LDAP database, > but at least it wouldn't cause 500 errors. > > Good luck, > - -chris > -BEGIN PGP SIGNATURE- > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQIcBAEBCAAGBQJYZZ1PAAoJEBzwKT+lPKRYThcP/RT/zeHoLhgsUvjhteXT2crZ > mqSnIzvDKTfTuktDROxZhL+BnSo4dirt0HcHz8yQ6c+hAlS6d2JtGGtpPiNPeigX > 4+0H9H6Nq9pCwK586wPqUusPs4bh1cbXBquAsdv3mG1w/cge+mgnYI6h7DSVBOgD > ir84T+7dnEZ25ygiN1e8Hp7DLyxWD/oRd594LIcTRtGisD0hRGGOc5xujmHxdhtQ > 0X8lQIlViL67Mo13hrFJQh7DO461MYxXElP+Ui39bq/i2rxSxrU4Xz/PjYb8LUhK > rRxNR7E8b59u+HxtiGMzM6wuRHBPsw4i575DGnSbTWPEjzER5ekLnV2FGdJA7rm5 > u1qENAbq9YuJ5I7NPFxSIC4iVtAI8vYEs86vG/JOtyGwMpy3L1uTpX0oYpEB+6nh > vUvl3l9S6aBqrYpHI/fG/SH3Y9jZ746d6GjyeLnEGIdjVFTxjbtFFlZH+EiQLMPx > IIr7zloPAQ+pNl5LjHoBsTjoTHtx6vnIYYFMfsl+vLAuFfHqJPqNh0qUuHoj4Esm > Rnl5cywGGqLSWiTCSwCdAtt2U8CyA4g6L9slYGp2USkAzBFEI1OFSDuy5A+fol+y > owkMlAkoMFxg8IM0c0VJofzUz/5IYiVLLRyth5ZfoxH3YK0WKZ8wQ5489bMQbQrt > QcVRNw4hG9IEkOaWrRhB > =W4NN > -END PGP SIGNATURE- > > ----- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi, after some more playing, I will take this of the Tomcat users list. I was able to reproduce the same problem with the previous setup: Server Version:Apache/2.4.18 (Unix) OpenSSL/1.0.2g mod_jk/1.2.41 It seems to take a bit longer to reproduce, but it happens with the same traces in the logs. [Thu Dec 29 15:55:31.128662 2016] [authnz_ldap:debug] [pid 24939:tid 139865530685184] mod_authnz_ldap.c(516): [client 160.46.222.183:52181] AH01691: auth_ldap authenticate: using URL ldaps://##:6362/dc=,dc=com?uid ldap_search_ext put_filter: "(&(objectclass=*)(uid=##))" put_filter: AND put_filter_list "(objectclass=*)(uid=###)" put_filter: "(objectclass=*)" put_filter: simple put_simple_filter: "objectclass=*" put_filter: "(uid=###)" put_filter: simple put_simple_filter: "uid=" ldap_build_search_req ATTRS: uid ldap_send_initial_request ldap_send_server_request ldap_result ld 0x84f940 msgid 6 wait4msg ld 0x84f940 msgid 6 (timeout 6000 usec) wait4msg continue ld 0x84f940 msgid 6 all 1 ** ld 0x84f940 Connections: * host:###t port: 6362 (default) refcnt: 2 status: Connected last used: Thu Dec 29 15:55:31 2016 ** ld 0x84f940 Outstanding Requests: * msgid 6, origid 6, status InProgress outstanding referrals 0, parent count 0 ld 0x84f940 request count 1 (abandoned 0) ** ld 0x84f940 Response Queue: Empty ld 0x84f940 response count 0 ldap_chkResponseList ld 0x84f940 msgid 6 all 1 ldap_chkResponseList returns ld 0x84f940 NULL ldap_int_select read1msg: ld 0x84f940 msgid 6 all 1 read1msg: ld 0x84f940 msgid 0 message type extended-result ldap_chase_referrals read1msg: V2 referral chased, mark request completed, id = 0 read1msg: ld 0x84f940 0 new referrals read1msg: mark request completed, ld 0x84f940 msgid 0 request done: ld 0x84f940 msgid 0 res_errno: 11, res_error: , res_matched: <> ldap_err2string [Thu Dec 29 15:55:31.129230 2016] [authnz_ldap:info] [pid 24939:tid 139865530685184] [client #:52181] AH01695: auth_ldap authenticate: user authentication failed; URI /jkmanager [ldap_search_ext_s() for user failed][Administrative limit exceeded] Interesting is the LDAP debug blurb about "connection has been idle for too long". Maybe we are seeing an infrastructure problem here. I will raise an issue with the responsible IT group. Thats fun :-( Thanks for the patience and a "Happy and Peaceful New Year 2017" Martin On Thu, Dec 29, 2016 at 12:36 PM, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 29.12.2016 10:46, Martin Knoblauch wrote: > >> Hi, >> >> "mod_jk" is now clearly off the hook. Upping the httpd log level from >> "warn" to "info" (I was assuming an event leading to a 500 response would >> be at least "warn" :-( reveals: >> >> [Thu Dec 29 10:37:37.300421 2016] [authnz_ldap:info] [pid 20325:tid >> 139641195009792] [client xxx.xxx.xxx.xxx:49959] AH01695: auth_ldap >> authenticate: user yyy authentication failed; URI /jkmanager >> [ldap_search_ext_s() for user failed][Administrative limit exceeded] >> >> @Christopher: thanks for the LDAP hint !!! >> >> > Perhaps also if you did not already know this : httpd 2.4 allows for > setting the LogLevel on a per-module base, see here : > https://httpd.apache.org/docs/2.4/logs.html -> Per-module logging > > > > Cheers >> Martin >> >> >> >> On Thu, Dec 29, 2016 at 10:02 AM, André Warnier (tomcat) <a...@ice-sa.com> >> wrote: >> >> On 29.12.2016 09:47, Martin Knoblauch wrote: >>> >>> Hi Christopher, >>>> >>>>that is an interesting pointer. We are of course securing the >>>> "jkmanager" >>>> app. And guess what we are using: LDAP. The funky thing is that it is >>>> working most of the time. It fails just after some time. Refreshing the >>>> URL >>>> cures it again - for some time. What did you do to fix your problem? >>>> >>>>As I mentioned elsewhere, setting "JkLogLevel debug" just filled the >>>> log >>>> without anything suspicious showing up. I can see "jkmanager" fire/work >>>> every 10 seconds (autorefresh), returning a 200 status. And then it >>>> nothing >>>> until I refresh the URL.So it seems the problem is elsewhere, before >>>> "mod_jk" come into play. >>>> >>>> >>> So setting JkLogLevel higher was far from useless : at least it tells you >>> where the problem isn't. >>> >>> "How often have I said to you that when you have eliminated the >>> impossible
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi Christopher, that is an interesting pointer. We are of course securing the "jkmanager" app. And guess what we are using: LDAP. The funky thing is that it is working most of the time. It fails just after some time. Refreshing the URL cures it again - for some time. What did you do to fix your problem? As I mentioned elsewhere, setting "JkLogLevel debug" just filled the log without anything suspicious showing up. I can see "jkmanager" fire/work every 10 seconds (autorefresh), returning a 200 status. And then it nothing until I refresh the URL.So it seems the problem is elsewhere, before "mod_jk" come into play. I will now try to investigate towards "mod_ldap" and maybe towards the OpenSSL stuff (we use LDAP over SSL). Fortunately rolling back versions is simple. As for being current, as far as I know we are up2date: ==> Server Version: Apache/2.4.23 (Unix) OpenSSL/1.0.2j mod_jk/1.2.42 Thanks Martin On Wed, Dec 28, 2016 at 9:43 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Martin, > > On 12/28/16 10:38 AM, Martin Knoblauch wrote: > > Hi, > > > > today we updated our Devel/Integration environments from > > > > HTTPD 2.4.18/mod_jk 1.2.41/OpenSSL-1.0.2h > > > > to > > > > HTTPD 2.4.23/mod_jk 1.2.42/OpenSSL-1.0.2j > > > > > > Since then we observe on both systems spurious "500" messages when > > accessing the "jkmanager" page. Unfortunately there isn't much info > > besides that. Only "access_log" shows > > > > access_log:xxx.xxx.xxx.xxx - [28/Dec/2016:16:29:18 +0100] > > "GET /jkmanager HTTP/1.1" 500 536 > > > > Any ideas how to get more insight > > I had a problem a while back where I would get 500 responses and > *nothing* else back. It took a lot of tinkering-around to figure out > the problem: my LDAP server wasn't acceptable for some reason and > mod_auth_ldap was choking. > > I spent all my time trying to figure out what was wrong with mod_jk > and it was the authentication layer way before mod_jk was being consulte > d. > > If you require authorization for jkmanager (and you should!) make sure > that's working as expected before banging your head against mod_jk. > > Also, make sure you are using the latest mod_jk that you can: the > distribution is separate from httpd. > > - -chris > -BEGIN PGP SIGNATURE- > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQIcBAEBCAAGBQJYZCPtAAoJEBzwKT+lPKRY82gP/1eG7zYY0dfxBKs8WTl80Wdp > o3qNaUeDROOdwER8VMmyVb7bmiPkmlj9FGGdKJqhjOSGeaHOLC6cEGce5JZSAzgl > q+/dOJ4xPaFqbmWUPfvQD7+pJZdFgcVqDowuSx2XWFUy/4L8CAjGii1jSHq3aEWu > umXiFT37igb0ApfpqYm1BNLtIuNvhoOdtpNxMWKULVF+kOjDPNK4+VE2Zj/2KCdk > Msm6jmSPvEKKbr+FaawdNyJl2D5qRMDrLwtzy+eGOFzatz6wQYQ6bc+i8JUqLjFo > 9+id+SLMlCSZxrZo3iTJBna/kUy1TZmqhLu1IpkqqRmapqdlMQpouCDfkpbO6g6B > Ot0/hffM9r8Ggp+OMd1GNBIzLwZAn3jRumZ/HxUmds5O2U/tJw0C4ajggXBwtZ5D > fz1ZEPkdkCcyP+3hB8G76BglfhcOfqti4jPmoVj+jqJ3QAQA7FdFcKVrS5erJB3z > YA3BSasWaOkO6Eg0UhZmwYvjy7YpptaF4NjRlftTiIgSd1gnoZOE1CMpItajjPYx > LajaudBoXy/wdvXHjydZXOZgzFS4a3UCReZvCwD/upegJsU2UbAoFswX8vq8lW3I > hu3WwazKja975ANKNQtLzDmKS0W4Hto4+oO94CmvGpY9s6oOkycu93Dnesgx73kS > TGIwfW3anqIyev1SG5w5 > =v9/q > -END PGP SIGNATURE- > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi Andre, yup - I know that. My httpd is now running with LogLevel notice ldap:debug authz_core:debug authnz_ldap:debug And LDAPLibraryDebug 7 Will see what comes out. Thanks Martin On Thu, Dec 29, 2016 at 12:36 PM, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 29.12.2016 10:46, Martin Knoblauch wrote: > >> Hi, >> >> "mod_jk" is now clearly off the hook. Upping the httpd log level from >> "warn" to "info" (I was assuming an event leading to a 500 response would >> be at least "warn" :-( reveals: >> >> [Thu Dec 29 10:37:37.300421 2016] [authnz_ldap:info] [pid 20325:tid >> 139641195009792] [client xxx.xxx.xxx.xxx:49959] AH01695: auth_ldap >> authenticate: user yyy authentication failed; URI /jkmanager >> [ldap_search_ext_s() for user failed][Administrative limit exceeded] >> >> @Christopher: thanks for the LDAP hint !!! >> >> > Perhaps also if you did not already know this : httpd 2.4 allows for > setting the LogLevel on a per-module base, see here : > https://httpd.apache.org/docs/2.4/logs.html -> Per-module logging > > > > Cheers >> Martin >> >> >> >> On Thu, Dec 29, 2016 at 10:02 AM, André Warnier (tomcat) <a...@ice-sa.com> >> wrote: >> >> On 29.12.2016 09:47, Martin Knoblauch wrote: >>> >>> Hi Christopher, >>>> >>>>that is an interesting pointer. We are of course securing the >>>> "jkmanager" >>>> app. And guess what we are using: LDAP. The funky thing is that it is >>>> working most of the time. It fails just after some time. Refreshing the >>>> URL >>>> cures it again - for some time. What did you do to fix your problem? >>>> >>>>As I mentioned elsewhere, setting "JkLogLevel debug" just filled the >>>> log >>>> without anything suspicious showing up. I can see "jkmanager" fire/work >>>> every 10 seconds (autorefresh), returning a 200 status. And then it >>>> nothing >>>> until I refresh the URL.So it seems the problem is elsewhere, before >>>> "mod_jk" come into play. >>>> >>>> >>> So setting JkLogLevel higher was far from useless : at least it tells you >>> where the problem isn't. >>> >>> "How often have I said to you that when you have eliminated the >>> impossible, whatever remains, however improbable, must be the truth?" >>> >>> Sherlock Holmes - The Sign of the Four >>> >>> >>> >>> >>>I will now try to investigate towards "mod_ldap" and maybe towards the >>>> OpenSSL stuff (we use LDAP over SSL). Fortunately rolling back versions >>>> is >>>> simple. >>>> >>>> As for being current, as far as I know we are up2date: >>>> >>>> ==> Server Version: Apache/2.4.23 (Unix) OpenSSL/1.0.2j mod_jk/1.2.42 >>>> >>>> Thanks >>>> Martin >>>> >>>> >>>> On Wed, Dec 28, 2016 at 9:43 PM, Christopher Schultz < >>>> ch...@christopherschultz.net> wrote: >>>> >>>> -BEGIN PGP SIGNED MESSAGE- >>>> >>>>> Hash: SHA256 >>>>> >>>>> Martin, >>>>> >>>>> On 12/28/16 10:38 AM, Martin Knoblauch wrote: >>>>> >>>>> Hi, >>>>>> >>>>>> today we updated our Devel/Integration environments from >>>>>> >>>>>> HTTPD 2.4.18/mod_jk 1.2.41/OpenSSL-1.0.2h >>>>>> >>>>>> to >>>>>> >>>>>> HTTPD 2.4.23/mod_jk 1.2.42/OpenSSL-1.0.2j >>>>>> >>>>>> >>>>>> Since then we observe on both systems spurious "500" messages when >>>>>> accessing the "jkmanager" page. Unfortunately there isn't much info >>>>>> besides that. Only "access_log" shows >>>>>> >>>>>> access_log:xxx.xxx.xxx.xxx - [28/Dec/2016:16:29:18 +0100] >>>>>> "GET /jkmanager HTTP/1.1" 500 536 >>>>>> >>>>>> Any ideas how to get more insight >>>>>> >>>>>> >>>>> I had a problem a while back where I would get 500 responses and >>>>> *nothing* else back.
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi Andre, upping the level to "debug" did not reveal much more about the effect. But you are right of course. Thanks Martin On Wed, Dec 28, 2016 at 5:02 PM, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 28.12.2016 16:38, Martin Knoblauch wrote: > >> Hi, >> >> today we updated our Devel/Integration environments from >> >> HTTPD 2.4.18/mod_jk 1.2.41/OpenSSL-1.0.2h >> >> to >> >> HTTPD 2.4.23/mod_jk 1.2.42/OpenSSL-1.0.2j >> >> >> Since then we observe on both systems spurious "500" messages when >> accessing the "jkmanager" page. Unfortunately there isn't much info >> besides >> that. Only "access_log" shows >> >> access_log:xxx.xxx.xxx.xxx - [28/Dec/2016:16:29:18 +0100] "GET >> /jkmanager HTTP/1.1" 500 536 >> >> Any ideas how to get more insight >> >> > Set JkLogLevel higher (in httpd) ? > > > > ----- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi, "mod_jk" is now clearly off the hook. Upping the httpd log level from "warn" to "info" (I was assuming an event leading to a 500 response would be at least "warn" :-( reveals: [Thu Dec 29 10:37:37.300421 2016] [authnz_ldap:info] [pid 20325:tid 139641195009792] [client xxx.xxx.xxx.xxx:49959] AH01695: auth_ldap authenticate: user yyy authentication failed; URI /jkmanager [ldap_search_ext_s() for user failed][Administrative limit exceeded] @Christopher: thanks for the LDAP hint !!! Cheers Martin On Thu, Dec 29, 2016 at 10:02 AM, André Warnier (tomcat) <a...@ice-sa.com> wrote: > On 29.12.2016 09:47, Martin Knoblauch wrote: > >> Hi Christopher, >> >> that is an interesting pointer. We are of course securing the >> "jkmanager" >> app. And guess what we are using: LDAP. The funky thing is that it is >> working most of the time. It fails just after some time. Refreshing the >> URL >> cures it again - for some time. What did you do to fix your problem? >> >> As I mentioned elsewhere, setting "JkLogLevel debug" just filled the log >> without anything suspicious showing up. I can see "jkmanager" fire/work >> every 10 seconds (autorefresh), returning a 200 status. And then it >> nothing >> until I refresh the URL.So it seems the problem is elsewhere, before >> "mod_jk" come into play. >> > > So setting JkLogLevel higher was far from useless : at least it tells you > where the problem isn't. > > "How often have I said to you that when you have eliminated the > impossible, whatever remains, however improbable, must be the truth?" > > Sherlock Holmes - The Sign of the Four > > > > >> I will now try to investigate towards "mod_ldap" and maybe towards the >> OpenSSL stuff (we use LDAP over SSL). Fortunately rolling back versions is >> simple. >> >> As for being current, as far as I know we are up2date: >> >> ==> Server Version: Apache/2.4.23 (Unix) OpenSSL/1.0.2j mod_jk/1.2.42 >> >> Thanks >> Martin >> >> >> On Wed, Dec 28, 2016 at 9:43 PM, Christopher Schultz < >> ch...@christopherschultz.net> wrote: >> >> -BEGIN PGP SIGNED MESSAGE- >>> Hash: SHA256 >>> >>> Martin, >>> >>> On 12/28/16 10:38 AM, Martin Knoblauch wrote: >>> >>>> Hi, >>>> >>>> today we updated our Devel/Integration environments from >>>> >>>> HTTPD 2.4.18/mod_jk 1.2.41/OpenSSL-1.0.2h >>>> >>>> to >>>> >>>> HTTPD 2.4.23/mod_jk 1.2.42/OpenSSL-1.0.2j >>>> >>>> >>>> Since then we observe on both systems spurious "500" messages when >>>> accessing the "jkmanager" page. Unfortunately there isn't much info >>>> besides that. Only "access_log" shows >>>> >>>> access_log:xxx.xxx.xxx.xxx - [28/Dec/2016:16:29:18 +0100] >>>> "GET /jkmanager HTTP/1.1" 500 536 >>>> >>>> Any ideas how to get more insight >>>> >>> >>> I had a problem a while back where I would get 500 responses and >>> *nothing* else back. It took a lot of tinkering-around to figure out >>> the problem: my LDAP server wasn't acceptable for some reason and >>> mod_auth_ldap was choking. >>> >>> I spent all my time trying to figure out what was wrong with mod_jk >>> and it was the authentication layer way before mod_jk was being consulte >>> d. >>> >>> If you require authorization for jkmanager (and you should!) make sure >>> that's working as expected before banging your head against mod_jk. >>> >>> Also, make sure you are using the latest mod_jk that you can: the >>> distribution is separate from httpd. >>> >>> - -chris >>> -BEGIN PGP SIGNATURE- >>> Comment: GPGTools - http://gpgtools.org >>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >>> >>> iQIcBAEBCAAGBQJYZCPtAAoJEBzwKT+lPKRY82gP/1eG7zYY0dfxBKs8WTl80Wdp >>> o3qNaUeDROOdwER8VMmyVb7bmiPkmlj9FGGdKJqhjOSGeaHOLC6cEGce5JZSAzgl >>> q+/dOJ4xPaFqbmWUPfvQD7+pJZdFgcVqDowuSx2XWFUy/4L8CAjGii1jSHq3aEWu >>> umXiFT37igb0ApfpqYm1BNLtIuNvhoOdtpNxMWKULVF+kOjDPNK4+VE2Zj/2KCdk >>> Msm6jmSPvEKKbr+FaawdNyJl2D5qRMDrLwtzy+eGOFzatz6wQYQ6bc+i8JUqLjFo >>> 9+id+SLMlCSZxrZo3iTJBna/kUy1TZmqhLu1IpkqqRmapqdlMQpouCDfkpbO6g6B >>> Ot0/hffM9r8Ggp+OMd1GNBIzLwZAn3jRumZ/HxUmds5O2U/tJw0C4ajggXBwtZ5D >>> fz1ZEPkdkCcyP+3hB8G76BglfhcOfqti4jPmoVj+jqJ3QAQA7FdFcKVrS5erJB3z >>> YA3BSasWaOkO6Eg0UhZmwYvjy7YpptaF4NjRlftTiIgSd1gnoZOE1CMpItajjPYx >>> LajaudBoXy/wdvXHjydZXOZgzFS4a3UCReZvCwD/upegJsU2UbAoFswX8vq8lW3I >>> hu3WwazKja975ANKNQtLzDmKS0W4Hto4+oO94CmvGpY9s6oOkycu93Dnesgx73kS >>> TGIwfW3anqIyev1SG5w5 >>> =v9/q >>> -END PGP SIGNATURE- >>> >>> - >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >>> For additional commands, e-mail: users-h...@tomcat.apache.org >>> >>> >>> >> >> > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Spurious "Internal Server Errors" accessing "jkmanager" after upgrading Apache, "mod_jk" and OpenSSL
Hi, today we updated our Devel/Integration environments from HTTPD 2.4.18/mod_jk 1.2.41/OpenSSL-1.0.2h to HTTPD 2.4.23/mod_jk 1.2.42/OpenSSL-1.0.2j Since then we observe on both systems spurious "500" messages when accessing the "jkmanager" page. Unfortunately there isn't much info besides that. Only "access_log" shows access_log:xxx.xxx.xxx.xxx - [28/Dec/2016:16:29:18 +0100] "GET /jkmanager HTTP/1.1" 500 536 Any ideas how to get more insight Thanks in advance Martin -- ---------- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Automatically compressing localhost_access_log after rotation
Hi, is there a way to compress the localhost_access_log.#.txt file automatically after rotation? Alternatively/preferably is there a way to put the access logging under "log4j"? I am on Tomcat 7.0.62 (no, do not tell me to update. I know. I want to. But I am not allowed to. So, do not tell me. Please :-) Thanks Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Automatically compressing localhost_access_log after rotation
On Thu, Aug 3, 2017 at 7:16 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 8/3/17 5:47 AM, Martin Knoblauch wrote: > > is there a way to compress the localhost_access_log.#.txt file > > automatically after rotation? > > Not really. The file is rotated *during* log events, and stalling to > compress a log file is probably not a great solution. > > Hmm. Smart way would be to rotate the log, continue logging to the new file and compress the old one "in the background". That way stalling would not be an issue. > Alternatively/preferably is there a way to put the access logging > > under "log4j"? > > Also not really, but if you are willing to write code, you can do it. > The AccessLogValve handles its own logging to a file, but if you were > to subclass AccessLogValve and override the "open" method and assign a > value to the AccessLogValve.writer member that writes to a log4j > logger, then I think you could probably do this. > Writing my own class is clearly an option, but I wanted to first check the already available options. > I believe that log4j will stall your access log during the > compression, though, so you might want to think about whether or not > you want to implement it this way. > > Need to check how they are doing it. We use rolling with compression, but not on a high volume log like the access log. > What is your operating system? If it were me, I think I'd write a cron > job to check the directory for uncompressed, rotated log files and > compress them in a separate process. > > Linux. Sure, we can deploy Cron to do the compression. It just adds another external process to an already complex setup. But I guess nothing in life is free :-) > > I am on Tomcat 7.0.62 (no, do not tell me to update. I know. I want > > to. But I am not allowed to. So, do not tell me. Please :-) > > You should upgrade :) > I told you not to tell me :-) Thanks Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Small question on mod_jk load balancing methon "Next"
Hi, just need some clarification on the mod_jk load blanacing method "Next". The documentation states: "If method is set to N[ext] the balancer will again use the number of sessions to find the best worker. All remarks concerning the Session method apply as well. The difference to the Session method is how the session count is handled in the sliding time window. The Next method does not divide by 2, instead it subtracts the current minimum number. This should effectively result in a round-robin session balancing, thus the name Next. Under high load, the two session balancing methods will result in a similar distribution, but Next will be better if you need to distribute small numbers of sessions. " What exactly is the "current minimum number"? How is the minimum taken? >From all workers in the balancer set, or only the ACTive ones? I know, I should look it up in the code :-) Thanks in advance Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Automatically compressing localhost_access_log after rotation
On Thu, Aug 3, 2017 at 9:43 PM, Leon Rosenberg <rosenberg.l...@gmail.com> wrote: > On Thu, Aug 3, 2017 at 8:16 PM, Christopher Schultz < > ch...@christopherschultz.net> wrote: > > > -BEGIN PGP SIGNED MESSAGE- > > Hash: SHA256 > > > > Martin, > > > > On 8/3/17 5:47 AM, Martin Knoblauch wrote: > > > is there a way to compress the localhost_access_log.#.txt file > > > automatically after rotation? > > > > Not really. The file is rotated *during* log events, and stalling to > > compress a log file is probably not a great solution. > > > > > Alternatively/preferably is there a way to put the access logging > > > under "log4j"? > > > > Also not really, but if you are willing to write code, you can do it. > > The AccessLogValve handles its own logging to a file, but if you were > > to subclass AccessLogValve and override the "open" method and assign a > > value to the AccessLogValve.writer member that writes to a log4j > > logger, then I think you could probably do this. > > > > I believe that log4j will stall your access log during the > > compression, though, so you might want to think about whether or not > > you want to implement it this way. > > > > I think at least logback is performing file operations asynchronously to > log events so maybe using slf4j over logback would be a more reliable way. > > regards > Leon > Hi Leon, thanks for the hint. Will look into it. Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Small question on mod_jk load balancing methon "Next"
On Fri, Aug 4, 2017 at 11:49 PM, Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 8/4/17 4:53 AM, Martin Knoblauch wrote: > > Hi, > > > > just need some clarification on the mod_jk load blanacing method > > "Next". The documentation states: > > > > "If method is set to N[ext] the balancer will again use the number > > of sessions to find the best worker. All remarks concerning the > > Session method apply as well. The difference to the Session method > > is how the session count is handled in the sliding time window. The > > Next method does not divide by 2, instead it subtracts the current > > minimum number. This should effectively result in a round-robin > > session balancing, thus the name Next. Under high load, the two > > session balancing methods will result in a similar distribution, > > but Next will be better if you need to distribute small numbers of > > sessions. " > > > > What exactly is the "current minimum number"? How is the minimum > > taken? > >> From all workers in the balancer set, or only the ACTive ones? > > I don't know for sure, but just in case this is a language issue, I > believe it means it will subtract the "lowest number of sessions > amongst all active workers". So if you had workers 1, 2, and 3 with > 45, 50, and 55 sessions respectively, it will subtract the value 45 > from all worker session counts to determine the priority for which > node will get the next session. > > Now my question was: which workers are considered for the lowest number of sessions. All of them, or only the ones in ACT state? Thanks Martin > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Small question on mod_jk load balancing methon "Next"
On Fri, Aug 4, 2017 at 11:47 PM, Rainer Jung <rainer.j...@kippdata.de> wrote: > Hi Martin, > > Am 04.08.2017 um 10:53 schrieb Martin Knoblauch: > >> Hi, >> >> just need some clarification on the mod_jk load blanacing method "Next". >> The documentation states: >> >> "If method is set to N[ext] the balancer will again use the number of >> sessions to find the best worker. All remarks concerning the Session >> method >> apply as well. The difference to the Session method is how the session >> count is handled in the sliding time window. The Next method does not >> divide by 2, instead it subtracts the current minimum number. This should >> effectively result in a round-robin session balancing, thus the name Next. >> Under high load, the two session balancing methods will result in a >> similar >> distribution, but Next will be better if you need to distribute small >> numbers of sessions. " >> >> What exactly is the "current minimum number"? How is the minimum taken? >> From all workers in the balancer set, or only the ACTive ones? I know, I >> should look it up in the code :-) >> > > I looked up the code I wrote 6 years ago. > > First: when using the session base lb methods, mod_jk needs to estimate > session counts. No lb method of mod_jk contacts the backends to get real > data, instead mod_jk uses the request info it sees to estimate the backend > situation. > > For session based methods, mod_jk counts requests, that do not include a > session id assuming that those are exactly the ones that create new > sessions. Of course: > > a) a session id can be outdated, meaning mod_jk would not count the > request as session creating but in fact it would create a new one. One can > at least configure mod_jk to be aware of login pages which will always > create a new session (see http://tomcat.apache.org/conne > ctors-doc/reference/uriworkermap.html and http://tomcat.apache.org/conne > ctors-doc/reference/apache.html and there look for "sticky_ignore"). > > b) a request without a session ID might not actually create a session, > depending on app details. There are additional config options to teach > mod_jk which URIs do not create sessions (see > http://tomcat.apache.org/connectors-doc/reference/uriworkermap.html and > http://tomcat.apache.org/connectors-doc/reference/apache.html and there > look for "stateless"). > > c) sessions time out in backends and users can log out. mod_jk does not > track that. One can remove the session cookie during the logout, so that > the "new" requests from that user will be counted by the mod_jk session > counter. > > Because of these problems I typically recommend to stick to the default lb > method (request counting, not session counting). But sometimes apps have > resource usage dominated by sessions and then a "session" based lb method > can help, especially if you find a configuration which keeps the effect of > a)-c) above small. > > Since all counting methods, not only session based ones, would count stuff > since the last restart of mod_jk, but the current backend load situation > depends much more on stuff that happened recently, we try to get rid of > past counts by reducing the counters regularly. By default this happens > once per minute and is done in a way that the counters are divided by 2 > once per minute. That way old counter increases contribute less and less to > the current counter value. For the session based method this would mean we > assume half of the counted sessions die after one minute, 50% of the rest > during the next minute etc. Note that the counters are integers, so e.g. a > counter value of 1 will after division by 2 result in a new value 0. Most > often that is no problem, because on a loaded system numbers are big and > rounding down doesn't change a lot. > > The next request without session id will be send to the worker with the > smallest such "session" counter. > > The "Next" message varies that procedure by not dividing by 2 every > minute, but instead subtracting the minimum value of the backend counters. > Assume after the first minute, your 4 backends have "session" counters 2, > 3, 3 and 2. Then the minimum is 2, so after the minute we correct the > values to 0, 1, 1 and 0. Then we add for the next minute new sessions to > that counter and again subtract the new minimum etc. > > When would that be helpful? It was for an application with really huge > sessions but small session numbers. There was a risk that if for a minute > only 0 or one sessions were created on the backends, after dividing by 2 > all
Re: Question on Tomcat clustering
Hi Louis, thanks. Yeah, currently the level is "INFO". I am a bit afraid of FINE/FINEST, fearing an overflow of messages that are to detailed. I had hoped for something on a higher severity - the event I am looking for is in the end an exception from the normal work. something noteworthy, not to be hidden on a Debug level. But yes, worth a try. Or maybe I just look at the source Happy weekend Martin On Thu, May 3, 2018 at 7:01 PM, Luis Rodríguez Fernández <uo67...@gmail.com> wrote: > Hello Martin, > > If you use the default tomcat logging [1] probably setting > > org.apache.catalina.ha.level = FINE or FINEST > > in your $CATALINA_BASE/conf/logging.properties will give you the > information you need. > > Hope it helps, > > Luis > > [1] https://tomcat.apache.org/tomcat-7.0-doc/logging.html > > > > > > > > > 2018-05-03 16:17 GMT+02:00 Martin Knoblauch <kn...@knobisoft.de>: > > > Hi, > > > > one question on Tomcat HA clustering. Is there a way (e.g. a log file > > pattern to look for) to find out that a sessions activity has actually > been > > transfered to another cluster node? We are operating a productive > > environment with 6 clustered TC processes and we would like to know how > > oftenthe clustering actually kicks in. > > > > We are on Tomcat 7. Please don't tell me to upgrade, but the productive > > version of the software is not certified with anything newer. > > > > Here is the cluster part of the configuration: > > > > > channelSendOptions="8"> > > > >>expireSessionsOnShutdown="false" > >notifyListenersOnReplication="true"/> > > > >> className="org.apache.catalina.tribes.group.GroupChannel"> > > > className="org.apache.catalina.tribes.membership.McastService" > > address="228.0.0.4" > > bind="real hostname censored" > > port="45564" > > frequency="1000" > > dropTime="3"/> > > > className="org.apache.catalina.tribes.transport.nio.NioReceiver" > > address="auto" > > port="4001" > > autoBind="100" > > selectorTimeout="5000" > > maxThreads="6"/> > > > > > className="org.apache.catalina.tribes.transport.ReplicationTransmitter" > > > >> className="org.apache.catalina.tribes.transport.nio. > PooledParallelSender" > > timeout="6"/> > > > > > className="org.apache.catalina.tribes.group.interceptors. > > TcpFailureDetector"/> > > > className="org.apache.catalina.tribes.group.interceptors. > > MessageDispatch15Interceptor"/> > > > className="org.apache.catalina.tribes.group.interceptors. > > ThroughputInterceptor" > > interval="10"/> > > > > > > > > > > > filter=".*\.gif;.*\.js;.*\.jpg;.*\.png;.*\.htm;.*\.html;. > > *\.css;.*\.txt;"/> > > > > > className="org.apache.catalina.ha.deploy.FarmWarDeployer" > > tempDir="/tmp/war-temp/" > > deployDir="/tmp/war-deploy/" > > watchDir="/tmp/war-listen/" > > watchEnabled="false"/> > > > > > className="org.apache.catalina.ha.session.JvmRouteBinderValve" > >enabled="true" sessionIdAttribute="takeoverSessionid"/> > > > className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListene > r" > > /> > > > className="org.apache.catalina.ha.session.ClusterSessionListener"/> > > > > > > Cheers > > Martin > > -- > > -- > > Martin Knoblauch > > email: k n o b i AT knobisoft DOT de > > www: http://www.knobisoft.de > > > > > > -- > > "Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better." > > - Samuel Beckett > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Question on Tomcat clustering
Hi, one question on Tomcat HA clustering. Is there a way (e.g. a log file pattern to look for) to find out that a sessions activity has actually been transfered to another cluster node? We are operating a productive environment with 6 clustered TC processes and we would like to know how oftenthe clustering actually kicks in. We are on Tomcat 7. Please don't tell me to upgrade, but the productive version of the software is not certified with anything newer. Here is the cluster part of the configuration: Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Apache 2.4.33 with mod_jk 1.2.43 keeps shutting down when no Tomcat worker is available
Hi, we have Apache-2.4.33 as a frontend to a bunch of Tomcats using mod_jk-1.2.43 as load balancer. Since we upgraded to this combination, we observe Apache shutdowns as soon as the last Tomcat worker is down. This has not happened before up until apache-2.4.33+mod_jk-1.2.42. error-log is pretty non-informative: [Fri Aug 17 11:39:24.172531 2018] [ldap:info] [pid 2312:tid 139990203332352] AH01318: APR LDAP: Built with OpenLDAP LDAP SDK [Fri Aug 17 11:39:24.172712 2018] [ldap:info] [pid 2312:tid 139990203332352] AH01319: LDAP: SSL support available [Fri Aug 17 11:39:24.191526 2018] [mpm_worker:notice] [pid 2312:tid 139990203332352] AH00292: Apache/2.4.33 (Unix) OpenSSL/1.0.2p mod_jk/1.2.43 configured -- resuming normal operations [Fri Aug 17 11:39:24.191566 2018] [core:notice] [pid 2312:tid 139990203332352] AH00094: Command line: '/opt/cb2/apache2/bin/httpd -D STATUS -f /opt/cb2/apache2/conf/httpd.conf' [Fri Aug 17 11:39:27.106795 2018] [mpm_worker:notice] [pid 2312:tid 139990203332352] AH00295: caught SIGTERM, shutting down Is there anything known along this lines? Thanks Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Apache 2.4.33 with mod_jk 1.2.43 keeps shutting down when no Tomcat worker is available
Hi, please ignore. There was another component (corosync) that I did not consider. When all tomcats are down, that ends up in a 503 which confused the HA software ... Happy Weekend Martin On Fri, Aug 17, 2018 at 12:15 PM Martin Knoblauch wrote: > Hi, > > we have Apache-2.4.33 as a frontend to a bunch of Tomcats using > mod_jk-1.2.43 as load balancer. Since we upgraded to this combination, we > observe Apache shutdowns as soon as the last Tomcat worker is down. This > has not happened before up until apache-2.4.33+mod_jk-1.2.42. > > error-log is pretty non-informative: > > [Fri Aug 17 11:39:24.172531 2018] [ldap:info] [pid 2312:tid > 139990203332352] AH01318: APR LDAP: Built with OpenLDAP LDAP SDK > [Fri Aug 17 11:39:24.172712 2018] [ldap:info] [pid 2312:tid > 139990203332352] AH01319: LDAP: SSL support available > [Fri Aug 17 11:39:24.191526 2018] [mpm_worker:notice] [pid 2312:tid > 139990203332352] AH00292: Apache/2.4.33 (Unix) OpenSSL/1.0.2p mod_jk/1.2.43 > configured -- resuming normal operations > [Fri Aug 17 11:39:24.191566 2018] [core:notice] [pid 2312:tid > 139990203332352] AH00094: Command line: '/opt/cb2/apache2/bin/httpd -D > STATUS -f /opt/cb2/apache2/conf/httpd.conf' > [Fri Aug 17 11:39:27.106795 2018] [mpm_worker:notice] [pid 2312:tid > 139990203332352] AH00295: caught SIGTERM, shutting down > > Is there anything known along this lines? > > Thanks > Martin > -- > -- > Martin Knoblauch > email: k n o b i AT knobisoft DOT de > www: http://www.knobisoft.de > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Tomcat JDBC Pool memory leak when using StatementFinalizer interceptor
Hi, while analyzing some heap dump for other reasons, I found that our application is apparently aggregating a considerable amount of memory in "org.apache.tomcat.jdbc.pool.TrapException", which is never cleaned by GC. Digging deeper, it seems that the entries of the "statements" linked list in the StatementFinalizer are never removed from the list, so after three weeks of lifetime one ends up with a list of 7 million entries, each 80 bytes. Now it might be, that we are just using the StatementFinalizer in a wrong manner. And what we see is expected behavior. Below is our pool configuration. Maybe something is just missing :-) We are at Tomcat 8.0.36 (yeah, I know, but that is the version we have to use) and Java 8 (1.8.0_171). Underlying DB is Oracle 12.1.0.2 and we are using the latest "ojdbc7.jar" from Oracle. Thanks Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
mod_jk: sessions connecting although worker is oset to DIS
Hi, we have this annoying problem that although Tomcat workers are set to DIS state in mod_jk, there are new connections being opened. This prevents us from "idle down" Tomcats in a timely fashion. This hurts when a restart is needed. While our empathy for human clients is pretty limited, we care a lot about our automated workloads :-) Our setup: Apache 2.4.x frontend -> mod_jk(1.2.42) balancer, sticky sessions -> a bunch of Tomcat 7 (yes I know. Oldie but Goldie :-) workers. As far as I understand, the problem arises with HTTP(S) clients that cache the session cookies. If such a client tries to open a new connection to the backend, the old cookie passes to a DIS worker even if it is no longer valid on the Tomcat side. Tomcat/Application then sees that the cookie is not valid and doles out a new cookie which then allows the new connection to be established. My idea is now to delete existing session cookies for certain request pattern on the Apache frontend. If it works, it should force mod_jk to a new ACT Tomcat. But I am nor really fond of messing with the Apache configuration, as the reason lies on the mod_jk/Tomcat side. So if anybody has another suggestion I would love to hear it. Before I forget, changing the application itself is not an option. TIA Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: mod_jk: sessions connecting although worker is oset to DIS
Hi Rainer,
so basically I took the Apache path and ended up with the following brute
force method:
RewriteCond "%{REQUEST_METHOD}" "GET"
RewriteRule ^/xxx/facelets/logon.xhtml$ - [E=login_jsid:%{HTTP_COOKIE}]
CustomLog "/opt/xxx/apache2/logs/login_log" xxxlogheader env=login_jsid
RequestHeader unset Cookie env=login_jsid
GET requests on the login page from the same client/browser now end up on
different nodes.
Looking at JK_STICKY_IGNORE, this seems also to work as well:
RewriteCond "%{REQUEST_METHOD}" "GET"
RewriteRule ^/xxx/facelets/logon.xhtml$ - [E=JK_STICKY_IGNORE]
CustomLog "/opt/xxx/apache2/logs/login_log" xxxlogheader
env=JK_STICKY_IGNORE
I like this actually better, as it does not need to mess with the Cookie.
This is better when components like SiteMinder (for SSO) are involved. I
will give that into our testing/integration environments to see whether we
have any bad side effects.
So thanks again for the valuable input. I will also have a look at the
filter/valve suggestions. But they look more intrusive and getting approval
[did I say big professional organisation somewhere :-)] for that might be
more difficult.
Cheers
Martin
On Thu, Mar 22, 2018 at 3:07 PM, Rainer Jung <rainer.j...@kippdata.de>
wrote:
> Am 22.03.2018 um 11:58 schrieb Martin Knoblauch:
>
>> Hi,
>>
>> we have this annoying problem that although Tomcat workers are set to
>> DIS
>> state in mod_jk, there are new connections being opened. This prevents us
>> from "idle down" Tomcats in a timely fashion. This hurts when a restart is
>> needed. While our empathy for human clients is pretty limited, we care a
>> lot about our automated workloads :-)
>>
>> Our setup: Apache 2.4.x frontend -> mod_jk(1.2.42) balancer, sticky
>> sessions -> a bunch of Tomcat 7 (yes I know. Oldie but Goldie :-) workers.
>>
>> As far as I understand, the problem arises with HTTP(S) clients that
>> cache
>> the session cookies. If such a client tries to open a new connection to
>> the
>> backend, the old cookie passes to a DIS worker even if it is no longer
>> valid on the Tomcat side. Tomcat/Application then sees that the cookie is
>> not valid and doles out a new cookie which then allows the new connection
>> to be established.
>>
>> My idea is now to delete existing session cookies for certain request
>> pattern on the Apache frontend. If it works, it should force mod_jk to a
>> new ACT Tomcat. But I am nor really fond of messing with the Apache
>> configuration, as the reason lies on the mod_jk/Tomcat side.
>>
>> So if anybody has another suggestion I would love to hear it. Before I
>> forget, changing the application itself is not an option.
>>
>
> Either your idea, or - if those unwanted new request flows start with a
> specific URL, e.g. a login - mark those URLs specifically as not using the
> cookie. If you are using a uriworkermap.properties, look for
> "sticky_ignore" below http://tomcat.apache.org/conne
> ctors-doc/reference/uriworkermap.html#Rule%20extensions. If you are
> instead using JkMount style maps, look for JK_STICKY_IGNORE in
> http://tomcat.apache.org/connectors-doc/reference/apache.
> html#Advanced%20Environment%20Variables.
>
> Whenever you set the Apache environment variable JK_STICKY_IGNORE during a
> request execution, mod_jk will ignore the cookie and distribute like if it
> didn't see one. Such environment variables (different from Unix environment
> variables) can be set e.g. with mod_setenvif based on request properties
> and also with mod_rewrite as a [E=] side effect.
>
> But that would only help if you can describe request properties that alow
> you to decide - based on the request alone - whether the cookie needs to be
> ignored or not. You can use any request info including all headers for that
> decision. What is not possible, is to first check, whether the session is
> valid.
>
> You could also write a filter that checks the cookie for validity in
> Tomcat and if it is not valid, reply with a self-referential redirect
> including a Set-Cookie-Header that deletes the cookie. This filter should
> be written, such that it does not itself create a new session.
>
> HTH,
>
> Rainer
>
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
--
--
Martin Knoblauch
email: k n o b i AT knobisoft DOT de
www: http://www.knobisoft.de
Acessing static content - Tomcat 7 vs. Tomcat9
Dear fellow Tomcat users, recently we migrated our application from Tomcat7 to Tomcat9. Most things work great so far, but we observed on issue. Basically serving static pages has stopped for us. Our setup is Tomcat (7.0.62 or 9.0.12) behind Apache HTTPD (2.4.41 using mod_jk 1.2.46). Yes, 9.0.12 is not recent, but we are forced to that version. The mod_jk configuration basically looks like: LoadModule jk_module modules/mod_jk.so JkWorkersFile "conf/cb2/workers.properties" JkShmFile "logs/jk-runtime-status" JkLogFile "logs/mod_jk.log" JkLogLevel info JkWatchdogInterval 60 And then later inside a virtual host: # # CB2 - Portal # # Mount the "/cb2" application to worker "cb2" # JkMount /cb2/* cb2 # # Unmount "/cb2/docs" from worker "cb2" to allow static content # beeing served by apache. Same for "/cb2/cgi-bin" # JkUnMount /cb2/docs/* cb2 So we JkUnMount the "/cb2/docs" directory from the application base in order to server the content directly from Apache. "docs" itself is a symbolic link pointing outside the application base. With TC7, we observe the following in the apache access_log: [07/Oct/2019:12:30:47 +0200] [2 ms] 160.46.219.110 - "POST /cb2/docs HTTP/1.1" s:302 l:- S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 [07/Oct/2019:12:30:47 +0200] [20 ms] 160.46.219.110 - "GET /cb2/docs/ HTTP/1.1" s:200 l:6367 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 So the POST from the application is redirected to the static content, which is served OK. With TC9 we see: [05/Oct/2019:02:58:13 +0200] [0 ms] #160.46.219.110# - "GET /docs HTTP/1.1" s:404 l:196 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 As said, the major difference between the setups is TC7 vs. TC9. Any ideas for me to follow? I did not find anything in the migration 7->8 or 8->9 guides. Kind regards Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Acessing static content - Tomcat 7 vs. Tomcat9
Hi Konstantin, On Mon, Oct 7, 2019 at 2:36 PM Konstantin Kolinko wrote: > пн, 7 окт. 2019 г. в 14:23, Martin Knoblauch : > > > > Dear fellow Tomcat users, > > > > recently we migrated our application from Tomcat7 to Tomcat9. Most > things > > work great so far, but we observed on issue. Basically serving static > pages > > has stopped for us. > > > > Our setup is Tomcat (7.0.62 or 9.0.12) behind Apache HTTPD (2.4.41 using > > mod_jk 1.2.46). Yes, 9.0.12 is not recent, but we are forced to that > > version. > > > > The mod_jk configuration basically looks like: > > > > > > LoadModule jk_module modules/mod_jk.so > > > > JkWorkersFile "conf/cb2/workers.properties" > > JkShmFile "logs/jk-runtime-status" > > JkLogFile "logs/mod_jk.log" > > JkLogLevel info > > JkWatchdogInterval 60 > > > > > > And then later inside a virtual host: > > > > # > > # CB2 - Portal > > # > > # Mount the "/cb2" application to worker "cb2" > > # > > JkMount /cb2/* cb2 > > # > > # Unmount "/cb2/docs" from worker "cb2" to allow static content > > # beeing served by apache. Same for "/cb2/cgi-bin" > > # > > JkUnMount /cb2/docs/* cb2 > > > > So we JkUnMount the "/cb2/docs" directory from the application base in > > order to server the content directly from Apache. "docs" itself is a > > symbolic link pointing outside the application base. > > > > With TC7, we observe the following in the apache access_log: > > > > [07/Oct/2019:12:30:47 +0200] [2 ms] 160.46.219.110 - "POST /cb2/docs > > HTTP/1.1" s:302 l:- S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > > [07/Oct/2019:12:30:47 +0200] [20 ms] 160.46.219.110 - "GET /cb2/docs/ > > HTTP/1.1" s:200 l:6367 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > > > > So the POST from the application is redirected to the static content, > which > > is served OK. > > A web server will happily server a static content in response to a > POST request. The redirect happens because you are requesting a > directory and your request URI does not end with a '/'. > > OK. Thanks for the explanation. > With TC9 we see: > > > > [05/Oct/2019:02:58:13 +0200] [0 ms] #160.46.219.110# - "GET /docs > HTTP/1.1" > > s:404 l:196 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > > > > As said, the major difference between the setups is TC7 vs. TC9. Any > ideas > > for me to follow? I did not find anything in the migration 7->8 or 8->9 > > guides. > > 1. In your access log here I see "GET /docs" instead of "/cb2/docs". > Is it intentional? > > No, just to many windows to cut and past from. And not enough caffeine... 2. For Tomcat to issue a redirect, the "docs" directory must be > present in your web application. It can be empty, but it must be > present. (If there is none, Tomcat does not know that the requested > resource is a directory). > > OK. The "docs" directory is actually a symbolic link to a directory elsewhere. 3. Starting with some version (January 2016) the default place where > the redirect is issued by Tomcat was moved from Mapper (in earlier > stages of request processing) to the DefaultServlet. This behaviour is > controlled by configuration attributes on a Context (in the file > META-INF/context.xml of your web application). See CVE-2015-5345 > > http://tomcat.apache.org/security-9.html > > Sounds interesting. Need to look at it. Thanks Martin > Best regards, > Konstantin Kolinko > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
Re: Acessing static content - Tomcat 7 vs. Tomcat9
On Mon, Oct 7, 2019 at 5:10 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > On 10/7/19 07:22, Martin Knoblauch wrote: > > The mod_jk configuration basically looks like: > > > > LoadModule jk_module modules/mod_jk.so > > > > JkWorkersFile "conf/cb2/workers.properties" JkShmFile > > "logs/jk-runtime-status" JkLogFile "logs/mod_jk.log" JkLogLevel > > info JkWatchdogInterval 60 > > So, you only configure mod_jk if it's NOT available? You have a "!" in > your which looks like it's going to never ever work. > > For a second you worried me. As far as I understand the documentation the section above means that I configure mod_jk if it is not yet loaded. And years of operations indicate my understanding is right :-) Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Acessing static content - Tomcat 7 vs. Tomcat9
On Mon, Oct 7, 2019 at 2:37 PM André Warnier (tomcat) wrote: > On 07.10.2019 13:45, Mark Thomas wrote: > > On 07/10/2019 12:22, Martin Knoblauch wrote: > >> Dear fellow Tomcat users, > >> > >> recently we migrated our application from Tomcat7 to Tomcat9. Most > things > >> work great so far, but we observed on issue. Basically serving static > pages > >> has stopped for us. > >> > >> Our setup is Tomcat (7.0.62 or 9.0.12) behind Apache HTTPD (2.4.41 > using > >> mod_jk 1.2.46). Yes, 9.0.12 is not recent, but we are forced to that > >> version. > >> > >> The mod_jk configuration basically looks like: > >> > >> > >>LoadModule jk_module modules/mod_jk.so > >> > >>JkWorkersFile "conf/cb2/workers.properties" > >>JkShmFile "logs/jk-runtime-status" > >>JkLogFile "logs/mod_jk.log" > >>JkLogLevel info > >>JkWatchdogInterval 60 > >> > >> > >> And then later inside a virtual host: > >> > >> # > >> # CB2 - Portal > >> # > >> # Mount the "/cb2" application to worker "cb2" > >> # > >> JkMount /cb2/* cb2 > >> # > >> # Unmount "/cb2/docs" from worker "cb2" to allow static content > >> # beeing served by apache. Same for "/cb2/cgi-bin" > >> # > >> JkUnMount /cb2/docs/* cb2 > >> > >> So we JkUnMount the "/cb2/docs" directory from the application base in > >> order to server the content directly from Apache. "docs" itself is a > >> symbolic link pointing outside the application base. > >> > >> With TC7, we observe the following in the apache access_log: > >> > >> [07/Oct/2019:12:30:47 +0200] [2 ms] 160.46.219.110 - "POST /cb2/docs > >> HTTP/1.1" s:302 l:- S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > >> [07/Oct/2019:12:30:47 +0200] [20 ms] 160.46.219.110 - "GET /cb2/docs/ > >> HTTP/1.1" s:200 l:6367 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > >> > >> So the POST from the application is redirected to the static content, > which > >> is served OK. > >> > >> With TC9 we see: > >> > >> [05/Oct/2019:02:58:13 +0200] [0 ms] #160.46.219.110# - "GET /docs > HTTP/1.1" > >> s:404 l:196 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 > > > > Where and how is the redirect generated? > > > >> As said, the major difference between the setups is TC7 vs. TC9. Any > ideas > >> for me to follow? I did not find anything in the migration 7->8 or 8->9 > >> guides. > > > > I can't think of anything and a review of the migration guides doesn't > > bring anything to mind. I'd be looking more at configuration differences > > between the old and new systems at this point. > > > > Mark > > > > The URLs appear different : > > - the first case shows a URL of "/cb2/docs" (with some kind of redirect > from POST to a > GET). According to the mod_jk configuration, this is NOT being proxied to > Tomcat. Assuming > thus that the httpd DOCUMENT_ROOT is set to (e.g.) /var/www/docs, it would > mean that the > file being served is some directory list (or index document) located below > /var/www/docs/cb2/docs/ (on the httpd host). > > Dear Andre, unfortunately this was cut/paste foul-up on my side :-( Martin > - the second example shows a URL of "/docs", which according to the mod_jk > configuration > above is also NOT being proxied to Tomcat; thus Apache httpd attempts to > serve it locally. > But that URL does not match any Apache-httpd-local file (because > /var/www/docs/docs does > not exist), and thus results in a 404 status "not found". > > To verify this is quite simple : > In the URL bar of a browser, enter the URL "http://yourservername/cb2/docs; > directly, > replacing "yourservername" alernatively with the one for tomcat 7 and the > one for tomcat9. > The result should be the same, because this URL is not being proxied to > Tomcat, and is > served locally by httpd. > > (On the other hand, the URL "/docs" should give "404 not found" in both > cases too) > > What I suspect is : > > The above URL "/cb2/docs" is not being entered in the browser URL bar. > Instead, the link > to that URL occurs in a html page previously returned by Tomcat, and the > user just clicks > on the link. > Then maybe it is that previous page returned by Tomcat, that is different > between Tomcat 7 > and Tomcat 9. > (That could be consistent with "the major difference between the setups is > TC7 vs. TC9."). > > Further speculation under incomplete information (I love this) : could it > be that under > Tomcat 7, your application is deployed under ../webapps/cb2, while under > tomcat 9, you > have relocated it to ../webapps/ROOT ? > > Nope. Application is at ".../webapps/cb2" Thanks Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Acessing static content - Tomcat 7 vs. Tomcat9
On Mon, Oct 7, 2019 at 3:31 PM Konstantin Kolinko wrote: > пн, 7 окт. 2019 г. в 15:44, Martin Knoblauch : > > > > Hi Konstantin, > > > > On Mon, Oct 7, 2019 at 2:36 PM Konstantin Kolinko < > knst.koli...@gmail.com> > > wrote: > > > > > > 2. For Tomcat to issue a redirect, the "docs" directory must be > > > present in your web application. It can be empty, but it must be > > > present. (If there is none, Tomcat does not know that the requested > > > resource is a directory). > > > > > > > > OK. The "docs" directory is actually a symbolic link to a directory > > elsewhere. > > Symbolic links by default are not allowed inside a web application. > The option to allow them differs between Tomcat 7 and 8.0, due to a > different underlying implementation of Web Application resources. > > http://tomcat.apache.org/migration-8.html#Web_application_resources > > (As a reminders: symbolic links must never be enabled on a > case-insensitive filesystem such as used by Windows, as it disables > the necessary security checks.) > > http://tomcat.apache.org/tomcat-9.0-doc/security-howto.html > > Best regards, > Konstantin Kolinko > > OK, this is interesting. We still had the TC7 style "allowLiniking" in the Context tag. Now moved to Resources. I also added the two mapper directives to Context: . . . What I now get is weird in another way. But maybe it gives a hint: [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "POST /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:10 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.46.219.110# - "GET /cb2/docs HTTP/1.1" s:302 l:219 S:TLSv1.2 C:ECDHE-RSA-AES256-GCM-SHA384 #"Mozilla/5.0 (X11; Linux x86_64; rv:69.0) Gecko/20100101 Firefox/69.0" [07/Oct/2019:16:10:11 +0200] [0 ms] #160.
Re: Acessing static content - Tomcat 7 vs. Tomcat9
Hi Andre, thanks for the extensive response. this is really appreciated. See below for comments On Mon, Oct 7, 2019 at 10:27 PM André Warnier (tomcat) wrote: > Hi. > see at end. > > > > Then the browser tells me that "The page isn’t redirecting properly". > > > > What one can see above is that the POST is now redirected, but the > trailing > > "/" is not added as I would have expected using the two mapper directives > > ... > Not only that, but also all the subsequent browser GET requests for > "/cb2/docs", get a 302 > response with a HTTP header : > Location: /cb2/docs > So the browser immediately re-issues a request for "/cb2/docs", which > receives the same > response, etc. > Until the (smart) browser realises that it is always requesting and > receiving the same > thing and calls quits. > > > > > What you are showing above as log, is only the httpd access log. > Do you have an access log enabled in tomcat, and do you see these same > accesses there ? > (And if yes, then forget all the rest below). > > Yes, we have access logging enabled for Tomcat and for the TC9 setup I *DO NOT* see requests for "/cb2/docs" in there. Looking at the TC7 setup I can find the access and redirect in the Tomcat access log: 160.46.219.110 - - [08/Oct/2019:12:49:31 +0200] "POST /cb2/docs HTTP/1.1" 302 - Or else, can you modify the "LogLevel" directive in httpd, so that you see > (in the httpd > error log) how (or if) these accesses are really being proxied to tomcat ? > (à la "Loglevel info proxy:debug proxy_ajp:debug" e.g.) > > Have added "JkLogLevel debug" to the mod_jk setup. Nothing interesting to see there. So I am still confused here. Based on this, from a previous post : > quote > # > # CB2 - Portal > # > # Mount the "/cb2" application to worker "cb2" > # > JkMount /cb2/* cb2 > # > # Unmount "/cb2/docs" from worker "cb2" to allow static content > # beeing served by apache. Same for "/cb2/cgi-bin" > # > JkUnMount /cb2/docs/* cb2 > > So we JkUnMount the "/cb2/docs" directory from the application base in > order to server the content directly from Apache. "docs" itself is a > symbolic link pointing outside the application base. > > unquote > > , I would indeed tend to say that a request with the URL "/cb2/docs" > SHOULD be proxied to > tomcat (and that the redirect should thus be coming from tomcat), but > better make > absolutely sure maybe ? (hence the additional logging above) > > The thing I'm unsure of myself, is this : > Apache httpd gets the original request first. Initially, it doesn't know > if this request > should be proxied to tomcat or not. So, in its "map to storage" phase - > which happens > fairly early in the request processing -, it might try to map this URL to > its local > filesystem. Because you do indeed have files locally under /cb2/docs/, it > would then find > that "/cb2/docs" is indeed a directory (or a link to ditto). > Now two things can happen, depending on the stage at which the mod_jk > proxying directives > intervene in httpd : > - either httpd considers that the request, being for a directory, should > have a trailing > "/", and *httpd* issues the 302 redirect to the browser, without even > getting to the > proxying-to-tomcat stage > - or the proxying directive has insinuated itself somewhere in-between, > and httpd does in > fact proxy the original request to tomcat (rather than trying to map it to > its own > filesystem first), and it is tomcat (which also would need to find a > directory at > ../webapps/cb2/docs) which issues that redirect. > > I do not know which of the above is true, because I am unsure of the httpd > request cycle > stage at which mod_jk inserts its URL mapping logic. > But the logs which you have so far provided do not really *prove* that > these requests make > it to tomcat. So, if it was me, before spending time maybe looking in the > wrong place, I > would want to make sure of that first. > > Another way would be to enable some local debugging console in the > browser, and have a > look at what these 302 replies look like. Apart from the "Location:" > header, there must be > a header there indicating from which webserver this response is coming > (httpd or tomcat). > > Again, because of the configuration that you showed, and because the > behaviour seems to > change after making a change in the tomcat configuration, I would tend to > believe that > these requests are being proxied to tomcat. But let's be 100% sure, rather > than 99%.. > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Acessing static content - Tomcat 7 vs. Tomcat9
On Tue, Oct 8, 2019 at 9:02 PM Christopher Schultz < ch...@christopherschultz.net> wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Martin, > > > > Sorry, I actually didn't notice the "LoadModule" in there. > > It's strange to see a LoadModule and the other configuration all > together for me. Typically, I load modules in one section and > configure then in another. It's all rather subjective, of course. > > > > Yeah, that one is a bit different. Two reasons. First, the configurations has been used for years and the guy doing it just preferred it that way. Second, as "mod_jk" is an "external" module to "httpd", it actually makes sense to check it that way. We know we want it. And if it cannot be loaded "httpd" should not come up at all. And now back to my redirect issue :-) Cheers Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: How to upload Files larger than 2GB
Hi Mark, thanks for the reply. The mentioning of "Chunked" helped me solve the problem. Our perl based REST client clearly tried to send the file as one buffer. Fortunately changing that was on line of code. > > > > Then looking at the Tomcat configuration. The HTTP connector (working) > > looks like this: > > > > >connectionTimeout="2" > >maxPostSize="209715200" > >redirectPort="8443" /> > > > > Which makes me wonder why it works. It should bail out at 200 MB. > > That limit only applies to the automated processing of request bodies as > per section 3.1.1 of the Servlet 4.0 specification. > > If the application (or a library it uses) reads the request body > directly, there is no limit. The application is meant to provide > whatever limits it considers appropriate. > > Thanks for clarifying that. > I wonder if httpd/mod_jk is trying cache the entire request body before > forwarding. How is the request sent? With chunked-encoding or with a > content-length? Does switching to the other one help? > > Apparently with content-length. Changing to "chunked" solved it. Thanks again Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
How to upload Files larger than 2GB
Hi, our customer has the following setup: Apache/HTTPD(2.4.43)->mod_jk(1.2.48)->Tomcat(9.0.12). The application hosted by Tomcat has a REST interface that allows file upload using POST requests. The problem now is that we get a 500 response when we try to upload files larger than 2 GB. But this happens only when using the full path from Apache to Tomcat. When we try the upload directly via the Tomcat HTTP port all is good This leaves me with the question where the problem is. Is it the Tomcat AJP part, is it mod_jk or is it HTTPD? From the logs I have the impression the file never appears on the Tomcat side. So I found the LimitRequestBody directive for HTTPD. But the description leaves me wondering. It says "This directive specifies the number of bytes from 0 (meaning unlimited) to 2147483647 (2GB) that are allowed in a request body." Means "unlimited" really no limit, or is it 2GB. Anyway, I set it to 0 with no success. Then looking at the Tomcat configuration. The HTTP connector (working) looks like this: Which makes me wonder why it works. It should bail out at 200 MB. The AJP connector looks like: I upped the maxPostSize to 20GB, but it did not help. Any advice is highly welcome. Just not - Don't use HTTP for uploading large files. It is the mechanism the application offers - Don't allow upload of large files. Unfortunately it is a valid use-case. Thanks in advance Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: Memory vs. Heap Space
Hi Jerry, one thing that could happen is that for some reason your server machine ran out of memory and in an attempt to keep the OS running it just killed the biggest memory user which just might be your Tomcat process. We had that happen to some development systems where the Tomcat "vanished" after 3 month uptime, because it was killed from the outside due to memory pressure. But your request does not provide enough info to tell. What I would do is turn on Java GC logging and see how the activity goes. Or use something like "visualvm". Add "-XX:+HeapDumpOnOutOfMemoryError" to the Java options. Look at the dump to see whether you have a leak in some rarely exercised code path. Cheers Martin On Tue, Apr 13, 2021 at 1:12 AM Jerry Malcolm wrote: > I'm getting TC crashes (ver 8.5) with "out of memory" errors. I > understand heap space and the parameters to set it. But I need a bit of > help with "out of memory", not "out of heap space". This is running in > an EC2 on AWS. It's a development server, so activity is minimal > compared to production. We also are running it on a smaller EC2 which > is likely contributing to the problem. But interestingly the last crash > happened in the middle of the night. The last request came in 15 > minutes prior to the crash, and it was some php intrusion attempt URL > that returned 404. Crashes don't happen often. A couple of servers have > failed once or twice a week. And this is the first time the dev server > crashed. But I still really need to get a handle on it. > > We have a pretty large max heap space configured. Does this problem > have something to do with specifying a heap space larger than TC can > allocate with physical memory? I figured there was still the option of > virtual memory and swapping if it ran out of physical memory. > > I need to learn more and tuning the memory usage. Is there some formula > that tells me the max heap space given a certain size of physical > memory? Or is there something else I'm totally missing about memory > configuration? > > Thanks. > > Jerry > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Migrating from TC9 to TC10. Can the jakartaConverter log problem cases?
Hi, I am in the process of migrating a large application from Tomcat9 to Tomcat10. While we have fixed the javax->jakarta issues in our code directly, we still need to specify jakartaConverter=TOMCAT to make the application work correctly. My question is now: does the converter log its work somewhere? How to enable it? Sorry if this is a stupid question. Thanks in advance Martin -- -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [mod_jk] Is it possible to set the value of a specific attribute via HTTP?
On 6/29/23 17:06, Rainer Jung wrote: > > > Since I try to push people into mod_proxy, I am hesitant to implement > > more and more features which keep people from switching ;) > > Hi Rainer, so, what do you suggest for the mod_jk retirement? mod_proxy+mod_proxy_ajp, or just proxy to the HTTPS port? Is there an equivalent to jkmanager with mod_proxy? Thanks Martin -- ------ Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
