Re: Tomcat/Java starts using too much memory and not by the heap or non-heap memory

[Stefan Mayr]

Hi,

Am 05.01.2024 um 23:21 schrieb Brian Braun:

Tracking native memory usage can be tricky depending upon your
environment. I would only look into that if there were somethng very odd
going on, like your process memory space seems to be more than 50% taken
by non-java-heap memory.



Well, actually that is my case. The heap memory (the 3 G1s) and non-heap
memory (3 CodeHeaps + compressed class space + metaspace) together use just
a few hundred MBs. I can see that using Tomcat Manager as well as the other
monitoring tools. And the rest of the memory (about 1GB) is being used by
the JVM but I don't know why or how, and that started 2 months ago. In your
case you have just 20-25% extra memory used in a way that you don't
understand, in my case it is about 200%.


Have you tried limiting native memory (-XX:MaxDirectMemorySize)? If not 
set this can be as large as your maximum heap size according to 
https://github.com/openjdk/jdk/blob/ace010b38a83e0c9b43aeeb6bc5c92d0886dc53f/src/java.base/share/classes/jdk/internal/misc/VM.java#L130-L136


From what I know:

total memory ~ heap + metaspace + code cache + (#threads * thread stack 
size) + direct memory


So if you set -Xmx to 1GB this should also allow 1GB of native memory 
which may result in more then 2GB of memory used by the JVM


Regards,

   Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat/Java starts using too much memory and not by the heap or non-heap memory

[Stefan Mayr]

Hi Brian,

Am 30.12.2023 um 21:42 schrieb Brian Braun:

I don't have any Java OOME exceptions, so it is not that my objects don't
fit. Even if I supply 300MB to the -Xmx parameter. In fact, as I wrote, I
don't think the Heap and non-heap usage is the problem. I have been
inspecting those and their usage seems to be normal/modest and steady. I
can see that using the Tomcat Manager as well as several other tools (New
Relic, VisualVM, etc).

Regarding the 1GB I am giving now to the -Xms parameter: I was giving just
a few hundreds and I already had the problem. Actually I think it is the
same if I give a few hundreds of MBs or 1GB, the JVM still starts using
more memory after 3-4 days of running until it takes 1.5GB. But during the
first 1-4 days it uses just a few hundred MBs.


So if this is not heap memory (-Xmx) it must be some other memory the 
JVM uses.


I guess we can rule out reserved code cache (-XX:ReservedCodeCacheSize) 
and stack size (-Xss) because they should have fixed sizes and you've 
written earlier that you checked the number of threads.


This leaves us with meta space (-XX:MaxMetaspaceSize) and native/direct 
memory (-XX:MaxDirectMemorySize). You can try to limit that or use the 
Java flight recorder and tools like Mission Control or VisualVM to make 
that kind of memory usage visible.


Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [SECURITY] CVE-2023-34981 Apache Tomcat - Information disclosure

[Stefan Mayr]

Hi,

Am 21.06.2023 um 12:20 schrieb Mark Thomas:

CVE-2023-34981 Apache Tomcat - Information disclosure

Severity: Important

Vendor: The Apache Software Foundation

Versions Affected:
Apache Tomcat 11.0.0-M5
Apache Tomcat 10.1.8
Apache Tomcat 9.0.74
Apache Tomcat 8.5.88

Description:
The fix for bug 66512 introduced a regression that was fixed as bug 
66591. The regression meant that, if a response did not have any HTTP 
headers set, no AJP SEND_HEADERS message would be sent which in turn 
meant that at least one AJP based proxy (mod_proxy_ajp) would use the 
response headers from the previous request for the current request 
leading to an information leak.

> ...

Are setups with mod_jk also affected?

Thanks,

Stefan



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: HTTP Error 414. The request URL is too long.

[Stefan Mayr]

Am 10.03.2023 um 08:27 schrieb Stefan Mayr:

Am 10.03.2023 um 07:58 schrieb Thomas Hoffmann (Speed4Trade GmbH):

Hello,


-Ursprüngliche Nachricht-
Von: Seth Mayers 
Gesendet: Freitag, 10. März 2023 01:14
An: Tomcat Users List 
Betreff: Re: HTTP Error 414. The request URL is too long.

Thanks.  Sadly I know how the data is being pushed.  It is poorly 
architected.
Developers are working to re-write it but I am looking for a short 
term solution
until they can complete that project.  I found where windows is 
logging the
error.  I am just not sure which process writes to this log or how to 
tell it to
ignore the error.  I thought I had it with a registry entry but it 
didn't fix the

problem.

C:\Windows\System32\LogFiles\HTTPERR


This looks like a windows system running IIS.
IIS also has some max-settings for requests.
Maybe you can check the IIS advanced settings for this site.


This could be an IIS with mod_jk or an ARR reverse proxy. When this is 
specific to GET requests increasing MaxFieldLength for http.sys should help


And increase MaxRequestBytes. I forgot that because we already increased 
that for our Kerberos tokens in request headers.


Reference: 
https://learn.microsoft.com/en-US/troubleshoot/developer/webapps/iis/iisadmin-service-inetinfo/httpsys-registry-windows


It might be necessary to tune other settings but those should have 
different status codes.




2023-03-09 23:55:40 192.168.103.30 56355 192.168.0.27 80 - GET - - 414 -
URL_Length -


You should keep an eye on this log entry: this is a GET request as your 
SAMPLE POST already indicated. Maybe you can check back with your 
developers that they change their code to really use the POST method



On Thu, Mar 9, 2023 at 7:06 PM Terence M. Bandoian 
wrote:


On 3/9/2023 5:47 PM, Jason Tan wrote:

Hi Seth,
One of my customer had the same error. We also tried increasing

maxHttpHeaderSize as suggested on Google.  It worked for him for a
while, but he encountered the same error again later on.

Our developers got involved and later discovered that our app was
trying

to display all his favourites and the customer somehow managed to
accumulate a huge list of favourites. So, a hotfix was created to
limit the size of the favourites that get displayed on to the browser.

I hope that helps you sort it out with the application to find what
huge

data is being push onto the url and limit it.

-Original Message-
From: Mark Thomas 
Sent: Friday, March 10, 2023 9:34 AM
To: users@tomcat.apache.org
Subject: Re: HTTP Error 414. The request URL is too long.

On 09/03/2023 20:59, Seth Mayers wrote:

I am running Apache Tomcat Version 9.0.48.

If I post a transaction that is very large, I get the "414; The
request URL is too long".

I have tried adding a bunch of parameters to my server.xml file but
none of them seem to work.  I have tried:
maxHttpHeaderSize="262144"
maxSavePostSize="-1"
maxPostSize="-1"
maxHttpRequestHeaderSize="262144"

I have someone that was able to resolve the same issue running
Tomcat
8.5 but his solution (maxhttpheadersize) did not work for me.  Did
something change between 8.5 and 9.0.48 that might affect things?
Is something involved beyond just server.xml?

Tomcat 9.0.x never returns a 414 status code. Wherever that status
code

is originating, it isn't Tomcat. It must be in the application or in a
component before Tomcat.


Mark




SAMPLE POST (the one that fails is FAR larger than this)



http://servername/gatewayAdminTest/GatewayClient?OutputType=1

Typ
e=1=

 
    
   LSF
   01092390
   02
   
   test_promo_freeProduct
   DROP
   
   
   01092391
   WEB001232
   O
    
    
   
  39012621
  1.0
  BAG
   
    
 
  

=xxx
--



Are you sure that's POST request? I'd suggest looking at the Tomcat
access log if you can get a request that far.

-Terence Bandoian


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




--


[image: photograph]

*Seth Mayers* Chief Technology Officer
*p:*
*e:* seth.may...@petfoodexperts.com
www.petfoodexperts.com
[image: facebook icon] <https://www.facebook.com/PFExperts>  [image:
instagram icon] <https://www.instagram.com/petfoodexperts>  [image:
linkedin icon] <https://www.linkedin.com/company/pet-food-experts>  
[image:
twitter icon] <https://twitter.com/petfoodexperts>  [image: youtube 
icon]

<https://www.youtube.com/c/PetFoodExperts>


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apac

Re: HTTP Error 414. The request URL is too long.

[Stefan Mayr]

Am 10.03.2023 um 07:58 schrieb Thomas Hoffmann (Speed4Trade GmbH):

Hello,


-Ursprüngliche Nachricht-
Von: Seth Mayers 
Gesendet: Freitag, 10. März 2023 01:14
An: Tomcat Users List 
Betreff: Re: HTTP Error 414. The request URL is too long.

Thanks.  Sadly I know how the data is being pushed.  It is poorly architected.
Developers are working to re-write it but I am looking for a short term solution
until they can complete that project.  I found where windows is logging the
error.  I am just not sure which process writes to this log or how to tell it to
ignore the error.  I thought I had it with a registry entry but it didn't fix 
the
problem.

C:\Windows\System32\LogFiles\HTTPERR


This looks like a windows system running IIS.
IIS also has some max-settings for requests.
Maybe you can check the IIS advanced settings for this site.


This could be an IIS with mod_jk or an ARR reverse proxy. When this is 
specific to GET requests increasing MaxFieldLength for http.sys should help


Reference: 
https://learn.microsoft.com/en-US/troubleshoot/developer/webapps/iis/iisadmin-service-inetinfo/httpsys-registry-windows


It might be necessary to tune other settings but those should have 
different status codes.




2023-03-09 23:55:40 192.168.103.30 56355 192.168.0.27 80 - GET - - 414 -
URL_Length -


You should keep an eye on this log entry: this is a GET request as your 
SAMPLE POST already indicated. Maybe you can check back with your 
developers that they change their code to really use the POST method



On Thu, Mar 9, 2023 at 7:06 PM Terence M. Bandoian 
wrote:


On 3/9/2023 5:47 PM, Jason Tan wrote:

Hi Seth,
One of my customer had the same error. We also tried increasing

maxHttpHeaderSize as suggested on Google.  It worked for him for a
while, but he encountered the same error again later on.

Our developers got involved and later discovered that our app was
trying

to display all his favourites and the customer somehow managed to
accumulate a huge list of favourites. So, a hotfix was created to
limit the size of the favourites that get displayed on to the browser.

I hope that helps you sort it out with the application to find what
huge

data is being push onto the url and limit it.

-Original Message-
From: Mark Thomas 
Sent: Friday, March 10, 2023 9:34 AM
To: users@tomcat.apache.org
Subject: Re: HTTP Error 414. The request URL is too long.

On 09/03/2023 20:59, Seth Mayers wrote:

I am running Apache Tomcat Version 9.0.48.

If I post a transaction that is very large, I get the "414; The
request URL is too long".

I have tried adding a bunch of parameters to my server.xml file but
none of them seem to work.  I have tried:
maxHttpHeaderSize="262144"
maxSavePostSize="-1"
maxPostSize="-1"
maxHttpRequestHeaderSize="262144"

I have someone that was able to resolve the same issue running
Tomcat
8.5 but his solution (maxhttpheadersize) did not work for me.  Did
something change between 8.5 and 9.0.48 that might affect things?
Is something involved beyond just server.xml?

Tomcat 9.0.x never returns a 414 status code. Wherever that status
code

is originating, it isn't Tomcat. It must be in the application or in a
component before Tomcat.


Mark




SAMPLE POST (the one that fails is FAR larger than this)



http://servername/gatewayAdminTest/GatewayClient?OutputType=1

Typ
e=1=

 

   LSF
   01092390
   02
   
   test_promo_freeProduct
   DROP
   
   
   01092391
   WEB001232
   O


   
  39012621
  1.0
  BAG
   

 
  

=xxx
--



Are you sure that's POST request? I'd suggest looking at the Tomcat
access log if you can get a request that far.

-Terence Bandoian


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




--


[image: photograph]

*Seth Mayers* Chief Technology Officer
*p:*
*e:* seth.may...@petfoodexperts.com
www.petfoodexperts.com
[image: facebook icon]   [image:
instagram icon]   [image:
linkedin icon]   [image:
twitter icon]   [image: youtube icon]



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat in distroless image

[Stefan Mayr]

Hi Chris,

Am 07.07.2022 um 00:59 schrieb Christopher Schultz:

Stefan,

On 7/6/22 18:50, Stefan Mayr wrote:

Am 05.07.2022 um 23:36 schrieb Pawel Veselov:

Christopher, Stephan,

On Tue, Jul 5, 2022 at 11:18 PM Christopher Schultz
 wrote:


Stefan,

On 7/2/22 09:45, Stefan Mayr wrote:

Hi,

Am 01.07.2022 um 17:10 schrieb Christopher Schultz:

Thomas,

On 6/30/22 13:52, Thomas Meyer wrote:

Sadly currently Tomcat startup relies on shell script to bootstrap
JVM process.

In the light of distroless images (e.g.
https://blog.chainguard.dev/introducing-apko-bringing-distroless-nirvana-to-alpine-linux/) 




What are you thoughts on packaging tomcat in distroless base OCI
images that doesn't even contain any shell anymore?

Would it be possible to provide an alternative start mechanism which
directly starts JVM process which setup/prepare env like the current
catalina.sh shell script does?

What are your thoughts on above topic?


Speaking for myself, here, of course.

The Tomcat team doesn't package anything other than the "standard
distributions" such as the source and the pre-built Java binaries, 
plus

the Windows binaries for various things. We don't package anything
specific like Docker base containers, etc. and I don't think we 
want to

get into that business.

If someone in the community wants to do something like that: go 
for it.

But we have enough to do already and don't want to play favorites.

The only reason catalina.sh exists is to figure out what's going 
on with
the hosting environment. If you want to build a launch-process 
that is

integrated into a specific environment, then you don't need all that
tooling to figure out what is what: you can just write one big 
command

line and launch the JVM with all the necessary arguments.


I agree with that. In my opinion this part of environment detection is
not necessary in a container because a container is immutable. So it
should be enough to run Tomcat the way you want to have it and copy &
paste the java command line into our container image generation 
manifest

(e.g. Dockerfile).

To use Tomcat in a more container friendly way you should consider 
also

some other things too:
- because containers are static there is no need to explode WAR 
files or
scan the directory for changes. The extracted application can be 
copied

into the image and server.xml can be tuned to disable those features.
- you should avoid logging into files and tune the logging 
configuration

to log everything to stdout and stderr


Logging to stdout and stderr is a sure way to get oneself into a
serious bottleneck.


Maybe, but this is the convention most have agreed on for stateless 
(container) applications. It is factor #11 of the 12 factor app 
principles (https://12factor.net/logs).
As always, your choices depend on the infrastructure you have. 
Runnning on a single docker host with an image customized for your 
needs? Then you can always mount the local filesystem into your 
container to write your logs on. Is this an image you deliver to your 
customers or that will run in kubernetes? So don't log into files and 
stick to the defaults: log to stdout and stderr



This is something that I really don't understand about containerized
applications running like this. Maybe I'm too old-skool and think that
application logs go into application-log files and access-logs go into
access-log files.

Assuming an ideal case, where should all of these things go if your
choices are "stdout" or "stderr"?

1. HTTP Access logs
2. HTTP access logs from multiple virtual hosts (or is the idea that 
one

container ~= 1 virtual host)
3. Server logs (e.g. DEBUG/INFO/WARN coming from the app server)
4. Application logs (e.g. DEBUG/INFO/WARN coming from the application)
5. Application logs for applications other than the primary (e.g. 
Tomcat

Manager)


They don't go to files. They get sent to a logging router instead
(e.g., Fluentd), which send them to logging aggregators (e.g.,
Graylog)


And because it is very hard to parse those logs in a sensible way it 
is becoming increasingly popular for container apps to use a json 
based log format. This is "ugly" but still somewhat readable for human 
users and almost all central logging solutions have a built-in json 
parser. No regex-vodoo to parse multiline output required (e.g. 
Stacktraces). Switching from raw text to a structed log format also 
improves the posibilities for your search queries a lot.


Addressing questions 2 and 5: it is best practice to have only one 
application in your container and remove everything also. So having 
multiple applications would usually result in multiple different 
containers. They might share the same (Java+Tomcat) baseimage but 
include different apps. A container image is an immutable artefact. As 
such they should have a bit of "hardening" and I would not expect 
something like the manager app in a container. Changes should result 
in new

Re: Tomcat in distroless image

[Stefan Mayr]

Am 05.07.2022 um 23:36 schrieb Pawel Veselov:

Christopher, Stephan,

On Tue, Jul 5, 2022 at 11:18 PM Christopher Schultz
 wrote:


Stefan,

On 7/2/22 09:45, Stefan Mayr wrote:

Hi,

Am 01.07.2022 um 17:10 schrieb Christopher Schultz:

Thomas,

On 6/30/22 13:52, Thomas Meyer wrote:

Sadly currently Tomcat startup relies on shell script to bootstrap
JVM process.

In the light of distroless images (e.g.
https://blog.chainguard.dev/introducing-apko-bringing-distroless-nirvana-to-alpine-linux/)


What are you thoughts on packaging tomcat in distroless base OCI
images that doesn't even contain any shell anymore?

Would it be possible to provide an alternative start mechanism which
directly starts JVM process which setup/prepare env like the current
catalina.sh shell script does?

What are your thoughts on above topic?


Speaking for myself, here, of course.

The Tomcat team doesn't package anything other than the "standard
distributions" such as the source and the pre-built Java binaries, plus
the Windows binaries for various things. We don't package anything
specific like Docker base containers, etc. and I don't think we want to
get into that business.

If someone in the community wants to do something like that: go for it.
But we have enough to do already and don't want to play favorites.

The only reason catalina.sh exists is to figure out what's going on with
the hosting environment. If you want to build a launch-process that is
integrated into a specific environment, then you don't need all that
tooling to figure out what is what: you can just write one big command
line and launch the JVM with all the necessary arguments.


I agree with that. In my opinion this part of environment detection is
not necessary in a container because a container is immutable. So it
should be enough to run Tomcat the way you want to have it and copy &
paste the java command line into our container image generation manifest
(e.g. Dockerfile).

To use Tomcat in a more container friendly way you should consider also
some other things too:
- because containers are static there is no need to explode WAR files or
scan the directory for changes. The extracted application can be copied
into the image and server.xml can be tuned to disable those features.
- you should avoid logging into files and tune the logging configuration
to log everything to stdout and stderr


Logging to stdout and stderr is a sure way to get oneself into a
serious bottleneck.


Maybe, but this is the convention most have agreed on for stateless 
(container) applications. It is factor #11 of the 12 factor app 
principles (https://12factor.net/logs).
As always, your choices depend on the infrastructure you have. Runnning 
on a single docker host with an image customized for your needs? Then 
you can always mount the local filesystem into your container to write 
your logs on. Is this an image you deliver to your customers or that 
will run in kubernetes? So don't log into files and stick to the 
defaults: log to stdout and stderr



This is something that I really don't understand about containerized
applications running like this. Maybe I'm too old-skool and think that
application logs go into application-log files and access-logs go into
access-log files.

Assuming an ideal case, where should all of these things go if your
choices are "stdout" or "stderr"?

1. HTTP Access logs
2. HTTP access logs from multiple virtual hosts (or is the idea that one
container ~= 1 virtual host)
3. Server logs (e.g. DEBUG/INFO/WARN coming from the app server)
4. Application logs (e.g. DEBUG/INFO/WARN coming from the application)
5. Application logs for applications other than the primary (e.g. Tomcat
Manager)


They don't go to files. They get sent to a logging router instead
(e.g., Fluentd), which send them to logging aggregators (e.g.,
Graylog)


And because it is very hard to parse those logs in a sensible way it is 
becoming increasingly popular for container apps to use a json based log 
format. This is "ugly" but still somewhat readable for human users and 
almost all central logging solutions have a built-in json parser. No 
regex-vodoo to parse multiline output required (e.g. Stacktraces). 
Switching from raw text to a structed log format also improves the 
posibilities for your search queries a lot.


Addressing questions 2 and 5: it is best practice to have only one 
application in your container and remove everything also. So having 
multiple applications would usually result in multiple different 
containers. They might share the same (Java+Tomcat) baseimage but 
include different apps. A container image is an immutable artefact. As 
such they should have a bit of "hardening" and I would not expect 
something like the manager app in a container. Changes should result in 
new images.


There are many different scenarios how containers can be

Re: Tomcat in distroless image

[Stefan Mayr]

Hi,

Am 01.07.2022 um 17:10 schrieb Christopher Schultz:

Thomas,

On 6/30/22 13:52, Thomas Meyer wrote:

Sadly currently Tomcat startup relies on shell script to bootstrap
JVM process.

In the light of distroless images (e.g.
https://blog.chainguard.dev/introducing-apko-bringing-distroless-nirvana-to-alpine-linux/) 



What are you thoughts on packaging tomcat in distroless base OCI 
images that doesn't even contain any shell anymore?


Would it be possible to provide an alternative start mechanism which
directly starts JVM process which setup/prepare env like the current
catalina.sh shell script does?

What are your thoughts on above topic?


Speaking for myself, here, of course.

The Tomcat team doesn't package anything other than the "standard
distributions" such as the source and the pre-built Java binaries, plus
the Windows binaries for various things. We don't package anything
specific like Docker base containers, etc. and I don't think we want to
get into that business.

If someone in the community wants to do something like that: go for it.
But we have enough to do already and don't want to play favorites.

The only reason catalina.sh exists is to figure out what's going on with
the hosting environment. If you want to build a launch-process that is
integrated into a specific environment, then you don't need all that
tooling to figure out what is what: you can just write one big command
line and launch the JVM with all the necessary arguments.


I agree with that. In my opinion this part of environment detection is 
not necessary in a container because a container is immutable. So it 
should be enough to run Tomcat the way you want to have it and copy & 
paste the java command line into our container image generation manifest 
(e.g. Dockerfile).


To use Tomcat in a more container friendly way you should consider also 
some other things too:
- because containers are static there is no need to explode WAR files or 
scan the directory for changes. The extracted application can be copied 
into the image and server.xml can be tuned to disable those features.
- you should avoid logging into files and tune the logging configuration 
to log everything to stdout and stderr


Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hi,

i created a bugzilla entry 
(https://bz.apache.org/bugzilla/show_bug.cgi?id=65901) and included a 
first attempt to patch this issue.


Am 12.02.2022 um 14:24 schrieb Stefan Mayr:

Hello Tomcat users,

this week we were debugging a strange connection issue which I tracked 
down to an interference between Apache httpd and mod_jk.


For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
3. a Tomcat mit AJP-Connector

We have an application doing many different HEAD requests against an 
application running in the Tomcat server. The requests contain an 
Authorization header for Basic authentication. Expected response is a 
HTTP 200 OK or HTTP 401 if this particular user is not allowed to access 
that ressource. Because this is a HEAD request there must not be a 
response body according to RFC 2616.


If there is a response body in the response to the HEAD request our 
loadbalancer does strange things: aborts the connection if the clients 
uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its 
own which we might have to solve with the vendor.


Now comes the point where I need your help. We have a httpd 
configuration with mod_jk which generates these invalid response bodies 
on HEAD requests. I have a gut feeling this could be a bug with mod_jk.


For demonstration purpose i created a minimal demo app which only is a 
WEB-INF/web.xml


https://jakarta.ee/xml/ns/jakartaee;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
   https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd;
   version="5.0">
     
     
     Login
     /*
     
     
     manager
     
     
     
     manager
     
     
     BASIC
     


Then I place a JkMount in my Apache httpd configuration (+ minimal 
worker.properties):


JkMount /demo/* ajp13_worker

Testing this with curl works like expected:

root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
 > HEAD /demo/ HTTP/1.1
 > Host: localhost
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Connection #0 to host localhost left intact

But our default setup always includes custom error pages:

Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var

If both of those lines are added this results in a response body for the 
HEAD request.


root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
 > HEAD /demo/ HTTP/1.1
 > Host: localhost
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:56:27 GMT
Date: Sat, 12 Feb 2022 12:56:27 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Excess found: excess = 589 url = /demo/ (zero-length body)
* Connection #0 to host localhost left intact

Checking with tcpdump on port 8009 we see the expected response without 
a body from the Tomcat AJP connector. The tcpdump von port 80 reveals 
httpd is adding the configured ErrorDocument as response body.


If we comment out either the Alias or ErrorDocument directive the 
response is correct again.


Doing similar tests with CGI/PHP applications always show the correct 
response without a response body. This only affects requests which use 
mod_jk.


So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running 
Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the 
included mod_jk 1.2.43) at work. At home the same happens on a stock 
openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as 
on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.

Re: mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hello Thomas,

Am 15.02.2022 um 11:38 schrieb Thomas Hoffmann (Speed4Trade GmbH):

Hello Stefan,

by spec / RFC, a HEAD request is not allowed to return any body.

Greetings,
Thomas


This is true and that is why i'm writing to this list. In the described 
case mod_jk returns a response body although it should not (at least i 
think mod_jk is somehow responsible for that)



-Ursprüngliche Nachricht-
Von: Stefan Mayr 
Gesendet: Montag, 14. Februar 2022 23:07
An: users@tomcat.apache.org
Betreff: Re: mod_jk interference with ErrorDocument/Alias on HEAD request

Hello again,

a self-compiled mod_jk 1.2.48 shows the same issue.

Am 13.02.2022 um 18:37 schrieb Stefan Mayr:

Hi,

looking at the source code
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.
0/mod_jk.c#L2954#L2973
I did some more testing:

Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
Variant 2: JkMount /demo/* ajp13_worker

ignoring what variant 2 changes for regular request: reading the
source comment my understanding is, that for both variants a HEAD
request (by definition must have an empty response body) should let
Apache httpd handle the error code.

But the return code for jk_handler looks different:

Variant 1: s.http_response_status
Variant 2: r->status


Although this looks different on the first glance it seems to be the same.


The response only seems correct for variant 1 - which is configured to
let Apache httpd handle all responses for status codes >= 401. For
variant 2 mod_jk seems to handle the response itself - contrary to
what the comment explains.


This leads to the next assumption, that whenever there is a special handling 
for use_server_errors there should be something similar for the case with an 
empty/non-existing response body.

There is
https://github.com/apache/tomcat-connectors/blob/main/native/common/jk_ajp_common.c#L1991#L1993
with no corresponding (pseudo) code like

  if (!r->something_like_bodyct && r->http_response_status >= 
JK_HTTP_BAD_REQUEST){
  r->response_blocked = JK_TRUE;
  }

Adding code like this (sorry, i could not find out how to determine if there is 
a response body) fixes the issue with the wrong response body for a HEAD 
request. But we miss the WWW-Authenticate header now.

Digging further we find
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L331#L353
which has a special treatment for 401 HTTP Unauthorized. But again, only for 
use_server_errors.

This should be fixable by extending the condition like this

  if ((s->extension.use_server_error_pages &&
  status >= s->extension.use_server_error_pages) ||
  (!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {


But the WWW-Authenticate header is still missing. So i'm wrong, again.
Although it feels like i'm close.


Am 12.02.2022 um 14:24 schrieb Stefan Mayr:

Hello Tomcat users,

this week we were debugging a strange connection issue which I
tracked down to an interference between Apache httpd and mod_jk.

For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat
mit AJP-Connector

We have an application doing many different HEAD requests against an
application running in the Tomcat server. The requests contain an
Authorization header for Basic authentication. Expected response is a
HTTP 200 OK or HTTP 401 if this particular user is not allowed to
access that ressource. Because this is a HEAD request there must not
be a response body according to RFC 2616.

If there is a response body in the response to the HEAD request our
loadbalancer does strange things: aborts the connection if the
clients uses HTTP/2, SSL errors if using HTTP/1.1. But this is an
issue on its own which we might have to solve with the vendor.

Now comes the point where I need your help. We have a httpd
configuration with mod_jk which generates these invalid response
bodies on HEAD requests. I have a gut feeling this could be a bug
with mod_jk.

For demonstration purpose i created a minimal demo app which only is
a WEB-INF/web.xml  https://jakarta.ee/xml/ns/jakartaee;
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
    xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee

https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd;

    version="5.0">
  
  
  Login
  /*
  
  
  manager
  
  
  
  manager
  
  
  BASIC
  


Then I place a JkMount in my Apache httpd configuration (+ minimal
worker.properties):

JkMount /demo/* ajp13_worker

Testing this with curl works like

Re: mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hello again,

a self-compiled mod_jk 1.2.48 shows the same issue.

Am 13.02.2022 um 18:37 schrieb Stefan Mayr:

Hi,

looking at the source code 
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973 
I did some more testing:


Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
Variant 2: JkMount /demo/* ajp13_worker

ignoring what variant 2 changes for regular request: reading the source 
comment my understanding is, that for both variants a HEAD request (by 
definition must have an empty response body) should let Apache httpd 
handle the error code.


But the return code for jk_handler looks different:

Variant 1: s.http_response_status
Variant 2: r->status


Although this looks different on the first glance it seems to be the same.

The response only seems correct for variant 1 - which is configured to 
let Apache httpd handle all responses for status codes >= 401. For 
variant 2 mod_jk seems to handle the response itself - contrary to what 
the comment explains.


This leads to the next assumption, that whenever there is a special 
handling for use_server_errors there should be something similar for the 
case with an empty/non-existing response body.


There is 
https://github.com/apache/tomcat-connectors/blob/main/native/common/jk_ajp_common.c#L1991#L1993 
with no corresponding (pseudo) code like


if (!r->something_like_bodyct && r->http_response_status >= 
JK_HTTP_BAD_REQUEST){

r->response_blocked = JK_TRUE;
}

Adding code like this (sorry, i could not find out how to determine if 
there is a response body) fixes the issue with the wrong response body 
for a HEAD request. But we miss the WWW-Authenticate header now.


Digging further we find 
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L331#L353 
which has a special treatment for 401 HTTP Unauthorized. But again, only 
for use_server_errors.


This should be fixable by extending the condition like this

if ((s->extension.use_server_error_pages &&
status >= s->extension.use_server_error_pages) ||
(!r->sent_bodyct && r->status >= HTTP_BAD_REQUEST)) {


But the WWW-Authenticate header is still missing. So i'm wrong, again. 
Although it feels like i'm close.



Am 12.02.2022 um 14:24 schrieb Stefan Mayr:

Hello Tomcat users,

this week we were debugging a strange connection issue which I tracked 
down to an interference between Apache httpd and mod_jk.


For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
3. a Tomcat mit AJP-Connector

We have an application doing many different HEAD requests against an 
application running in the Tomcat server. The requests contain an 
Authorization header for Basic authentication. Expected response is a 
HTTP 200 OK or HTTP 401 if this particular user is not allowed to 
access that ressource. Because this is a HEAD request there must not 
be a response body according to RFC 2616.


If there is a response body in the response to the HEAD request our 
loadbalancer does strange things: aborts the connection if the clients 
uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its 
own which we might have to solve with the vendor.


Now comes the point where I need your help. We have a httpd 
configuration with mod_jk which generates these invalid response 
bodies on HEAD requests. I have a gut feeling this could be a bug with 
mod_jk.


For demonstration purpose i created a minimal demo app which only is a 
WEB-INF/web.xml


https://jakarta.ee/xml/ns/jakartaee;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
   
https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd;

   version="5.0">
 
 
 Login
 /*
 
 
 manager
 
 
 
 manager
 
 
 BASIC
 


Then I place a JkMount in my Apache httpd configuration (+ minimal 
worker.properties):


JkMount /demo/* ajp13_worker

Testing this with curl works like expected:

root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
 > HEAD /demo/ HTTP/1.1
 > Host: localhost
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm

Re: mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hi,

looking at the source code 
https://github.com/apache/tomcat-connectors/blob/main/native/apache-2.0/mod_jk.c#L2954#L2973 
I did some more testing:


Variant 1: JkMount /demo/* ajp13_worker;use_server_errors=401
Variant 2: JkMount /demo/* ajp13_worker

ignoring what variant 2 changes for regular request: reading the source 
comment my understanding is, that for both variants a HEAD request (by 
definition must have an empty response body) should let Apache httpd 
handle the error code.


But the return code for jk_handler looks different:

Variant 1: s.http_response_status
Variant 2: r->status

The response only seems correct for variant 1 - which is configured to 
let Apache httpd handle all responses for status codes >= 401. For 
variant 2 mod_jk seems to handle the response itself - contrary to what 
the comment explains.


Am 12.02.2022 um 14:24 schrieb Stefan Mayr:

Hello Tomcat users,

this week we were debugging a strange connection issue which I tracked 
down to an interference between Apache httpd and mod_jk.


For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
3. a Tomcat mit AJP-Connector

We have an application doing many different HEAD requests against an 
application running in the Tomcat server. The requests contain an 
Authorization header for Basic authentication. Expected response is a 
HTTP 200 OK or HTTP 401 if this particular user is not allowed to access 
that ressource. Because this is a HEAD request there must not be a 
response body according to RFC 2616.


If there is a response body in the response to the HEAD request our 
loadbalancer does strange things: aborts the connection if the clients 
uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its 
own which we might have to solve with the vendor.


Now comes the point where I need your help. We have a httpd 
configuration with mod_jk which generates these invalid response bodies 
on HEAD requests. I have a gut feeling this could be a bug with mod_jk.


For demonstration purpose i created a minimal demo app which only is a 
WEB-INF/web.xml


https://jakarta.ee/xml/ns/jakartaee;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
   https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd;
   version="5.0">
     
     
     Login
     /*
     
     
     manager
     
     
     
     manager
     
     
     BASIC
     


Then I place a JkMount in my Apache httpd configuration (+ minimal 
worker.properties):


JkMount /demo/* ajp13_worker

Testing this with curl works like expected:

root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
 > HEAD /demo/ HTTP/1.1
 > Host: localhost
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Connection #0 to host localhost left intact

But our default setup always includes custom error pages:

Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var

If both of those lines are added this results in a response body for the 
HEAD request.


root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
 > HEAD /demo/ HTTP/1.1
 > Host: localhost
 > User-Agent: curl/7.68.0
 > Accept: */*
 >
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:56:27 GMT
Date: Sat, 12 Feb 2022 12:56:27 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Excess found: excess = 589 url = /demo/ (zero-length body)
* Connection #0 to host localhost left intact

Checking with tcpdump on port 8009 we see the expected response without 
a b

Re: mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hi Thomas,

Am 13.02.2022 um 17:27 schrieb Thomas Hoffmann (Speed4Trade GmbH):

Hello,

maybe you can try to set an environment variable which skips interpreting the 
content-length:
https://tomcat.apache.org/connectors-doc/reference/apache.html#Advanced%20Environment%20Variables
--> JK_IGNORE_CL

To get more information, you can also set the logfile and log-level to debug: 
JkLogLevel
(same reference page as above).

Greetings,
Thomas


I added both options:

Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
SetEnv JK_IGNORE_CL
JkLogLevel Debug
JkMount /demo/* ajp13_worker

This produces the following debug output:

[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 005001 00 17 
74 65 78 74 2F 68 74 6D 6C 3B 63 68 61  - ...text/html;cha
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 006072 73 65 
74 3D 75 74 66 2D 38 00 A0 02 00 02 65  - rset=utf-8.e
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 00706E 00 00 
00 00 00 00 00 00 00 00 00 00 00 00 00  - n...
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (736): (ajp13_worker) status = 401
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (743): Number of headers is = 4
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[0] 
[Cache-Control] = [private]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[1] 
[WWW-Authenticate] = [Basic realm="Authentication required"]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[2] 
[Content-Type] = [text/html;charset=utf-8]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_unmarshal_response::jk_ajp_common.c (799): (ajp13_worker) Header[3] 
[Content-Language] = [en]
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): received from 
ajp13 pos=0 len=2 max=8192
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_connection_tcp_get_message::jk_ajp_common.c (1459): 05 01 00 
00 00 00 00 00 00 00 00 00 00 00 00 00  - 
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_process_callback::jk_ajp_common.c (2132): (ajp13_worker) AJP13 
protocol: Reuse is OK
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_reset_endpoint::jk_ajp_common.c (848): (ajp13_worker) resetting 
endpoint with socket 15
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [debug] 
ajp_done::jk_ajp_common.c (3288): recycling connection pool for worker 
ajp13_worker and socket 15
[Sun Feb 13 16:37:02.591 2022] [212:140388647335680] [info] 
jk_handler::mod_jk.c (2968): No body with status=401 for worker=ajp13_worker
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug] 
jk_servlet_normalize::jk_util.c (2185): URI on entering 
jk_servlet_normalize: [/error/HTTP_UNAUTHORIZED.html.var]
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug] 
jk_servlet_normalize::jk_util.c (2279): URI on exiting 
jk_servlet_normalize: [/error/HTTP_UNAUTHORIZED.html.var]
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug] 
map_uri_to_worker_ext::jk_uri_worker_map.c (1168): Attempting to map URI 
'/error/HTTP_UNAUTHORIZED.html.var' from 1 maps
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug] 
find_match::jk_uri_worker_map.c (978): Attempting to map context URI 
'/demo/*=ajp13_worker' source 'JkMount'
[Sun Feb 13 16:37:02.592 2022] [212:140388647335680] [debug] 
jk_map_to_storage::mod_jk.c (4058): no match for 
/error/HTTP_UNAUTHORIZED.html.var found


The last lines are irritating. They don't change when I try to add a 
JkUnMount:


JkMount /demo/* ajp13_worker
JkUnMount /error/* ajp13_worker



-Ursprüngliche Nachricht-
Von: Stefan Mayr 
Gesendet: Samstag, 12. Februar 2022 14:24
An: Tomcat Users List 
Betreff: mod_jk interference with ErrorDocument/Alias on HEAD request

Hello Tomcat users,

this week we were debugging a strange connection issue which I tracked down to 
an interference between Apache httpd and mod_jk.

For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk 3. a Tomcat mit 
AJP-Connector

We have an application doing many different HEAD requests against an 
application running in the Tomcat server. The requests contain an Authorization 
header for Basic authentication. Expected response is a HTTP 200 OK or HTTP 

mod_jk interference with ErrorDocument/Alias on HEAD request

[Stefan Mayr]

Hello Tomcat users,

this week we were debugging a strange connection issue which I tracked 
down to an interference between Apache httpd and mod_jk.


For the full picture, the infrastructure setup contains

1. a Loadbalancer providing HTTPS, HTTP/1.1 and HTTP/2 for Clients.
2. an Apache httpd 2.4 webserver (HTTP only) with mod_jk
3. a Tomcat mit AJP-Connector

We have an application doing many different HEAD requests against an 
application running in the Tomcat server. The requests contain an 
Authorization header for Basic authentication. Expected response is a 
HTTP 200 OK or HTTP 401 if this particular user is not allowed to access 
that ressource. Because this is a HEAD request there must not be a 
response body according to RFC 2616.


If there is a response body in the response to the HEAD request our 
loadbalancer does strange things: aborts the connection if the clients 
uses HTTP/2, SSL errors if using HTTP/1.1. But this is an issue on its 
own which we might have to solve with the vendor.


Now comes the point where I need your help. We have a httpd 
configuration with mod_jk which generates these invalid response bodies 
on HEAD requests. I have a gut feeling this could be a bug with mod_jk.


For demonstration purpose i created a minimal demo app which only is a 
WEB-INF/web.xml


https://jakarta.ee/xml/ns/jakartaee;
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
  xsi:schemaLocation="https://jakarta.ee/xml/ns/jakartaee
  https://jakarta.ee/xml/ns/jakartaee/web-app_5_0.xsd;
  version="5.0">


Login
/*


manager



manager


BASIC



Then I place a JkMount in my Apache httpd configuration (+ minimal 
worker.properties):


JkMount /demo/* ajp13_worker

Testing this with curl works like expected:

root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> HEAD /demo/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:57:33 GMT
Date: Sat, 12 Feb 2022 12:57:33 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Connection #0 to host localhost left intact

But our default setup always includes custom error pages:

Alias /error/ "/usr/share/apache2/error/"
ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var

If both of those lines are added this results in a response body for the 
HEAD request.


root@1ae8973f1b6b:~# curl -I -v localhost/demo/
*   Trying 127.0.0.1:80...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 80 (#0)
> HEAD /demo/ HTTP/1.1
> Host: localhost
> User-Agent: curl/7.68.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 401
HTTP/1.1 401 401
< Date: Sat, 12 Feb 2022 12:56:27 GMT
Date: Sat, 12 Feb 2022 12:56:27 GMT
< Server: Apache/2.4.41 (Ubuntu)
Server: Apache/2.4.41 (Ubuntu)
< Cache-Control: private
Cache-Control: private
< WWW-Authenticate: Basic realm="Authentication required"
WWW-Authenticate: Basic realm="Authentication required"
< Content-Language: en
Content-Language: en
< Content-Type: text/html;charset=utf-8
Content-Type: text/html;charset=utf-8

<
* Excess found: excess = 589 url = /demo/ (zero-length body)
* Connection #0 to host localhost left intact

Checking with tcpdump on port 8009 we see the expected response without 
a body from the Tomcat AJP connector. The tcpdump von port 80 reveals 
httpd is adding the configured ErrorDocument as response body.


If we comment out either the Alias or ErrorDocument directive the 
response is correct again.


Doing similar tests with CGI/PHP applications always show the correct 
response without a response body. This only affects requests which use 
mod_jk.


So far I could reproduce this on SLES12 SP5 and SLES15 SP3 running 
Apache httpd 2.4.51 and a self compile mod_jk 1.2.46 (same with the 
included mod_jk 1.2.43) at work. At home the same happens on a stock 
openSUSE Leap 15.3 with Apache httpd 2.4.51 and mod_jk 1.2.43 as well as 
on Ubuntu 20.04 with httpd 2.4.41 and mod_jk 1.2.46.
I didn't try to compile the latest mod_jk version yet because I didn't 
spot a relevant point in the changelog.


Can anyone confirm this behaviour or point me to a configuration 
directive i missed?


Thank you,

Re: Do I Need Network NameSpaces to Solve This Tomcat+Connector/J Problem?

[Stefan Mayr]

Am 30.12.2021 um 12:24 schrieb Mark Thomas:

On 29/12/2021 21:04, Eric Robinson wrote:




My question is, is there a better way?


I can only think of variations on a theme.

The ~64k limit assumes client IP, server IP and server port remain 
constant. i.e. just client port is varying.


That suggests there is a single IP for the database server and that it 
is listening on a single port.


You are currently varying client IP. Varying server IP is unlikely to be 
any different in terms of ease of management etc.


There may be more mileage in getting the database server to listen on 
more than one port. It depends how the database sever is structured. If 
it can have multiple listeners all passing connections to the same 
database instance then adding db listeners might be a simpler way to 
manage this.


Mark


A third option could be to add something between database client and 
server. Something on layer 4 like multiple HAProxy servers or simple NAT 
gateways. Or more complex on layer 7 specfic products like ProxySQL or 
MaxScale. They could even pool connections and reduce the load on the 
database server. But this all adds complexity and new ways to fail.


The easiest solution in terms of implementation and operation is the one 
Mark suggested: add multiple ip addresses and/or ports to your database 
listener.


Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Spring Security LDAPS authenticator won't trust TLS cert

[Stefan Mayr]

Am 25.01.2021 um 19:04 schrieb Christopher Schultz:
> All,
> 
> On 1/25/21 11:10, Christopher Schultz wrote:
>> All,
>>
>> Off-topic, but I know there are plenty of Spring users on this list
>> who can probably help me figure this out.
>>
>> Recently, Let's Encrypt switched from using their soon-to-be-expiring
>> intermediate certificate:
>>
>> Owner:  CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
>> Issuer: CN=DST Root CA X3, O=Digital Signature Trust Co.
>> Serial number: a014142015385736a0b85eca708
>> Valid from: Thu Mar 17 12:40:46 EDT 2016 until: Wed Mar 17 12:40:46
>> EDT 2021
>>
>> To this new one:
>>
>> Owner: CN=R3, O=Let's Encrypt, C=US
>> Issuer: CN=DST Root CA X3, O=Digital Signature Trust Co.
>> Serial number: 400175048314a4c8218c84a90c16cddf
>> Valid from: Wed Oct 07 15:21:40 EDT 2020 until: Wed Sep 29 15:21:40
>> EDT 2021
>>
...
> 
> But why had it worked before, when cacerts didn't include the *previous*
> intermediate certificate?
> 

Because you usually don't need to add intermediate certificates to your
truststore. Your SSL-ified services presents his public certificate and
the certificate chain (all intermediates) to a client. The client
verifies the certificate chain you provided and checks the last
certificate against its truststore containing all root CAs.

So for your old and new certificate this should all work out if DST Root
CA X3 is in your cacerts file.

For the next new cert you will have two options for the certificate
chain:
https://letsencrypt.org/2020/12/21/extending-android-compatibility.html
or for the complete view of chains: https://letsencrypt.org/certificates/

So for a future proof setup you should have ISRG Root X1 in your
truststore or keep an eye on the intermediate certificate you use.

My guess for your current problem would be the following: your LDAPS
didn't update the chain and still provides the X3 instead of the R3
intermediate. The old intermediate certificate is ignored and it now
only works when you add the intermediate certificate to your truststore.
Please verify which intermediate certificate is provided by your LDAPS

e.g. openssl s_client -connect ldaps.example.com:636 -showcerts

- Stefan


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [EXTERNAL] Re: Bouncy Castle FIPS on RHEL 7.3

[Stefan Mayr]

Hi,

Am 30.11.2020 um 17:09 schrieb Amit Pande:
> I guess I will have to investigate the RHEL 7.3 entropy issue separately 
> (possibly as hobby project) and look for other options to make progress.
> 
> I still find it odd that something related to randomness (entropy generation) 
> is so consistent (the slowness is equally slow or more on multiple RHEL 7.3 
> systems I have, maybe I need to look for machines from different data center 
> or a physical 7.3 server).
> 
> And yes, the 10 year certificate validity is just for testing purposes. 
> 
> Thank you for your inputs. Indeed helpful in evaluating our choices. 
> 
> Thanks,
> Amit

you might have a look at rng-tools (rngd) or haveged to boost your
entropy pool.

We use haveged in a VMware virtualized environment and this reduces a
plain tomcat startup from multiple minutes to just a few secondes.

I think Red Hat preferes rngd but there should be some articles on
access.redhat.com to help you depending on the used hypervisor.

Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Weirdest Tomcat Behavior Ever?

[Stefan Mayr]

Am 03.11.2020 um 16:05 schrieb Eric Robinson:
>> -Original Message-
>> From: Eric Robinson 
>> Sent: Tuesday, November 3, 2020 8:21 AM
>> To: Tomcat Users List 
>> Subject: RE: Weirdest Tomcat Behavior Ever?
>>
>>> From: Mark Thomas 
>>> Sent: Tuesday, November 3, 2020 2:06 AM
>>> To: Tomcat Users List 
>>> Subject: Re: Weirdest Tomcat Behavior Ever?
>>>
>>> On 02/11/2020 12:16, Eric Robinson wrote:
>>>
>>> 
>>>
 Gotcha, thanks for the clarification. Let's see what happens when
 the users
>>> start hitting it at 8:00 am Eastern.
>>>
>>> Progress. The first attempt to write to the socket triggers the
>>> following
>>> exception:
>>>
>>> 02-Nov-2020 14:33:54.083 FINE [http-bio-3016-exec-13]
>>> org.apache.tomcat.util.net.JIoEndpoint$DebugOutputStream.write
>>> [301361476]
>>>  java.net.SocketException: Bad file descriptor (Write failed)
>>> at java.net.SocketOutputStream.socketWrite0(Native Method)
>>> at
>>> java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:111)
>>> at java.net.SocketOutputStream.write(SocketOutputStream.java:155)
>>> at
>>>
...

>>> Because this is an instance of an IOException, Tomcat assumes it has
>>> been caused by the client dropping the connection and silently
>>> swallows it. I'll be changing that later today so the exception is
>>> logged as DEBUG level for all new Tomcat versions.
>>>
>>> Possible causes of "java.net.SocketException: Bad file descriptor"
>>> I've been able to find are:
>>>
>>> 1. OS running out of file descriptors.
>>>
>>> 2.Trying to use a closed socket.
>>>
>>> I want to review the source code to see if there are any others.
>>>
>>> I don't think we are seeing 2 as there is no indication of the Socket,
>>> InputStream or OutputStream being closed in the logs.
>>>
>>> That leaves 1. Possible causes here are a file descriptor leak or
>>> normal operations occasionally needing more than the current limit. I
>>> don't think it is a leak as I'd expect to see many more errors of this
>>> type after the first and we aren't seeing that. That leaves the
>>> possibility of the current limit being a little too low.
>>>
>>> My recommendation at this point is to increase the limit for file 
>>> descriptors.
>>> Meanwhile, I'll look at the JRE source to see if there are any other
>>> possible triggers for this exception.
>>>
>>> Mark
>>>
>>>
>>
>> On the tomcat server, max open file descriptors is currently 2853957
>>
>> [root@001app01a ~]# sysctl fs.file-max
>> fs.file-max = 2853957
>>
>> Most of the time, the number of open files appears to run about 600,000.
>>
>>  What do you think of watching the open file count and seeing if the number
>> gets up around the ceiling when the socket write failure occurs? Something
>> like...
>>
>> [root@001app01a ~]#  while [ TRUE ];do FILES=$(lsof|wc -l);echo "$(date
>> +%H:%M:%S) $FILES";done
>> 09:11:15 591671
>> 09:11:35 627347
>> 09:11:54 626851
>> 09:12:11 626429
>> 09:12:26 545748
>> 09:12:42 548578
>> 09:12:58 551487
>> 09:13:14 516700
>> 09:13:30 513312
>> 09:13:45 512830
>> 09:14:02 58
>> 09:14:18 568233
>> 09:14:35 570158
>> 09:14:51 566269
>> 09:15:07 547389
>> 09:15:23 544203
>> 09:15:38 546395
>>
>> It's not ideal; as it seems to take 15-20 seconds to count them using lsof.
>>
>>
>>
> 
> Wait, never mind. I realized the per-process limits are what matters. I 
> checked, and nofile was set to 4096 for the relevant java process.
> 
> I did...
> 
> # prlimit --pid 8730 --nofile=16384:16384
> 
> That should give java some extra breathing room if the issue is max open 
> files, right?

>From my experience you should see a different exception if you hit the
NOFILE limit: java.net.SocketException: Too many open files

But I've only seen that when you open or accept a new connection. Never
seen this later when something is written to that already open socket.

To me a bad file descriptor sounds more like a closed socket. This
reminds me of a database or http-client connection pool that handed out
connections with already closed sockets. I think this could be
suspicious because Mark wrote this happens on the first write to the socket.

- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat mod_jk rpm-build for CentOS-8 - exit with error.

[Stefan Mayr]

Hi Klaus,

Am 06.07.2020 um 15:13 schrieb Klaus Tachtler:
> Hi,
> 
> I'm trying to build a rpm package for CentOS-8 for mod_jk with the
> latest version 1.2.48. While building the rpm package, following error
> occurs:
> 
> 
> - %< -
> 
> ...
> make[1]: Entering directory
> '/root/rpmbuild/BUILD/tomcat-connectors-1.2.48-src/native/apache-2.0'
> /usr/lib64/apr-1/build/libtool --silent --mode=link gcc
> -I/usr/include/httpd -O2 -g -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
> -fstack-protector-strong -grecord-gcc-switches
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -DHAVE_CONFIG_H -DUSE_APACHE_MD5 -I../common -I ../common -DLINUX
> -D_REENTRANT -D_GNU_SOURCE -O2 -g -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
> -fstack-protector-strong -grecord-gcc-switches
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -pthread -DHAVE_APR  -I/usr/include/apr-1 -I/usr/include/apr-1 -O2 -g
> -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2
> -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong
> -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -DHAVE_CONFIG_H -O2 -g -pipe -Wall -Werror=format-security
> -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
> -fstack-protector-strong -grecord-gcc-switches
> -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
> -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
> -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
> -I/usr/include/apr-1  -Wl,-z,relro,-z,now -Wl,-z,relro -Wl,-z,now
> -Wl,-specs=/usr/lib/rpm/redhat/redhat-hardened-ld -o mod_jk.la -module
> -rpath /usr/lib64/httpd/modules -avoid-version mod_jk.lo
> ../common/jk_ajp12_worker.lo ../common/jk_connect.lo
> ../common/jk_msg_buff.lo ../common/jk_util.lo ../common/jk_ajp13.lo
> ../common/jk_pool.lo ../common/jk_worker.lo ../common/jk_ajp13_worker.lo
> ../common/jk_lb_worker.lo ../common/jk_sockbuf.lo ../common/jk_map.lo
> ../common/jk_uri_worker_map.lo ../common/jk_ajp14.lo
> ../common/jk_ajp14_worker.lo ../common/jk_md5.lo ../common/jk_shm.lo
> ../common/jk_ajp_common.lo ../common/jk_context.lo ../common/jk_url.lo
> ../common/jk_status.lo
> /usr/bin/ld: unrecognized option
> '-specs=/usr/lib/rpm/redhat/redhat-hardened-ld'
> /usr/bin/ld: use the --help option for usage information
> collect2: error: ld returned 1 exit status
> make[1]: *** [Makefile:83: mod_jk.la] Error 1
> make[1]: Leaving directory
> '/root/rpmbuild/BUILD/tomcat-connectors-1.2.48-src/native/apache-2.0'
> make: *** [Makefile:470: all-recursive] Error 1
> error: Bad exit status from /var/tmp/rpm-tmp.O4kRZI (%build)
> 
> - >% -
Does it build without the specfile? I tried to reproduce it in a simple
Dockerfile. This works so far - can you show us your .spec-file?

Sample Dockerfile to build mod_jk on CentOS 8

FROM centos:8
ARG JK_VERSION=1.2.48
ARG
REMOTE_URL=https://downloads.apache.org/tomcat/tomcat-connectors/jk/tomcat-connectors-${JK_VERSION}-src.tar.gz
RUN yum --assumeyes --quiet update && \
yum --assumeyes --quiet install \
diffutils \
file \
gcc \
httpd-devel \
libtool \
make \
redhat-rpm-config && \
yum clean all
RUN cd /usr/src && \
curl --silent --remote-name ${REMOTE_URL} && \
tar -xzf tomcat-connectors-${JK_VERSION}-src.tar.gz && \
cd tomcat-connectors-${JK_VERSION}-src/native && \
./configure --with-apxs=/usr/bin/apxs  && \
make && \
libtool --finish /usr/lib64/httpd/modules && \
make install

Regards,

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Wrong timezone in Date and Last-Modified-Headers

[Stefan Mayr]

Hi,

Am 10.06.2020 um 15:34 schrieb Mark Thomas:
> On 10/06/2020 14:07, Paul Carter-Brown wrote:
>> At runtime, any code can call TimeZone.setDefault to change the timezone of
>> the JVM.
>>
>> I'd suggest logging  TimeZone.getDefault().getDisplayName(Locale.ENGLISH);
>> intermittently and seeing if some code somewhere is changing the timezone.
>> Could be in any library...
> 
> Most likely this:
> 
> https://bz.apache.org/bugzilla/show_bug.cgi?id=64226
> 
> Mark
> 
> 
>>
>> Paul

You were right. I missed that one in the release notes. Using a request
with a If-Modified-Since-Header changed the timezone in the Tomcat-Response.

Luckily the latest Spring Boot release 2.1.14 contains Tomcat 9.0.34
which includes the bugfix.

Thanks,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Wrong timezone in Date and Last-Modified-Headers

[Stefan Mayr]

Hi,

today I've seen something I don't understand: our developers reported an
application that was returning a non-GMT timezone in Date and
Last-Modified headers.

$ curl -v http://localhost:8080
* Rebuilt URL to: http://localhost:8080/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200
< X-Correlation-Id: 68ef1063-dbbb-4216-a3a7-2acdb768abbd
< Last-Modified: Tue, 12 May 2020 14:04:15 YEKT
< Accept-Ranges: bytes
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Cache-Control: no-cache, no-store, max-age=0, must-revalidate
< Pragma: no-cache
< Expires: 0
< Strict-Transport-Security: max-age=31536000
< X-Frame-Options: DENY
< Content-Type: text/html;charset=UTF-8
< Content-Language: en-US
< Content-Length: 5409
< Date: Wed, 10 Jun 2020 20:42:26 JST
...

There you can see Date with JST and Last-Modified with YEKT instead of GMT

All other locales (LANG, LC_*, /etc/localtime) of this system are okay
and after a restart everything is correct again and I'm not able to
reproduce it

$ curl -vs http://localhost:8080 2>&1|egrep -i "Date|Last-Modified"
*  start date: Mar 25 10:50:00 2020 GMT
*  expire date: Mar 30 23:59:59 2021 GMT
< cache-control: no-cache, no-store, max-age=0, must-revalidate
< date: Wed, 10 Jun 2020 11:52:42 GMT
< last-modified: Tue, 12 May 2020 09:04:15 GMT


This is a Spring Boot application with an embedded Tomcat (9.0.16)
running in a Cloud Foundry Container.

Reading through the source code of
org.apache.tomcat.util.http.FastHttpDateFormat I don't understand how
this is even possible.

Has anybody ever seen this or an idea what could have happened there?
After the restart I can't reproduce it.

Thanks,

   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [SECURITY] CVE-2020-9484 Apache Tomcat Remote Code Execution via session persistence

[Stefan Mayr]

Hi,

Am 20.05.2020 um 17:19 schrieb Mark Thomas:
> CVE-2020-9484 Apache Tomcat Remote Code Execution via session persistence
> 
> Severity: High
> 
> Vendor: The Apache Software Foundation
> 
> Versions Affected:
> Apache Tomcat 10.0.0-M1 to 10.0.0-M4
> Apache Tomcat 9.0.0.M1 to 9.0.34
> Apache Tomcat 8.5.0 to 8.5.54
> Apache Tomcat 7.0.0 to 7.0.103
> 
> Description:
> If:
> a) an attacker is able to control the contents and name of a file on the
>server; and
> b) the server is configured to use the PersistenceManager with a
>FileStore; and
> c) the PersistenceManager is configured with
>sessionAttributeValueClassNameFilter="null" (the default unless a
>SecurityManager is used) or a sufficiently lax filter to allow the
>attacker provided object to be deserialized; and
> d) the attacker knows the relative file path from the storage location
>used by FileStore to the file the attacker has control over;
> then, using a specifically crafted request, the attacker will be able to
> trigger remote code execution via deserialization of the file under
> their control. Note that all of conditions a) to d) must be true for the
> attack to succeed.
> 

Assuming an attacker can do (a), (d) and the Tomcat instance is running
with a default configuration (c): is the StandardManager vulnerable or
not (b)?

Also a question about naming: is PersistenceManager the same
PersistentManager as in org.apache.catalina.session.PersistentManager?
So a vulnerable configuration would need to use something like


  


Regards,

  Stefan Mayr


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [SECURITY] CVE-2020-1938 AJP Request Injection and potential Remote Code Execution

[Stefan Mayr]

Am 24.02.2020 um 13:47 schrieb Mark Thomas:
> CVE-2020-1938 AJP Request Injection and potential Remote Code Execution
> 
> Severity: High
> 
> ...
> - returning arbitrary files from anywhere in the web application
>   including under the WEB-INF and META-INF directories or any other
>   location reachable via ServletContext.getResourceAsStream()
> - processing any file in the web application as a JSP
> Further, if the web application allowed file upload and stored those
> files within the web application (or the attacker was able to control
> the content of the web application by some other means) then this, along
> with the ability to process a file as a JSP, made remote code execution
> possible.

Is this a bug which is or will be fixed or is this a fundamental design
flaw of AJP which cannot be fixed? So to trust or not to trust are the
only options?

Thanks,

  Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [ANN] Apache Tomcat 9.0.31 available

[Stefan Mayr]

Hi Chris,

Am 13.02.2020 um 15:31 schrieb Christopher Schultz:
> [snip]
> The answer to the question "why change the default?" is: "because the
> default was essentially insecure, in a way that wasn't obvious to
> someone who wasn't paying close attention."
> 
> So we are forcing users to pay closer attention. If you want to open
> your AJP instance to the "whole world", then you can explicitly bind
> to 0.0.0.0/:: just as the previous default. Similarly, if you don't
> want to use the "secret" setting, then you can explicitly disable it.
> But the defaults will no longer let you be "insecure" without knowing it
> .
> 
> Obviously, there are ways to have a "secure" installation while using
> 0.0.0.0/:: and/or secretRequired="false". But having those things in
> the configuration right in front of you make it clear that some
> decision has been made, rather than hiding (potential) danger behind
> insecure defaults.

Okay, security is important. I'm a huge fan of secure by default and
minimal default setups instead all features enabled.
But I still don't understand what makes makes the AJP connector or the
protocol itself so insecure. I really don't know much about it. So far
this has been a technology that just works for me. Is really AJP the
culprit or are you just closing all windows on the "house of tomcat" so
everybody has to use the "front door" http?

Because if having an open connector is really the argument the http
connector would also (by default) have to bind to localhost only. Same
as with AJP: you have to make a clear decision in the configuration to
open it to the public. Is this something we have to expect in the future?

> [snip]
> 
> Will this disrupt some environments? Yes, it will. But the path to
> fixing it is to make one (or two) small edits to your configuration
> files which are surely under revision-control and automatically
> pushed-out to these hundreds-of-nodes clusters everyone is worried
> about, right? Well, then, change your configs and push them out there
> along with your upgrade of Tomcat and all will be well.
> [snip]

Automation is the right keyword. Sometimes it is a solution, sometimes
this also causes additional problems. In our specific case the
configuration management system generates server.xml from templates.
Currently it can only use different templates for different major
versions (7,8 or 9). Not having tried that yet I would add the new ajp
connector parameters in those templates to reflect the old defaults.
That would us gain some time to change the configuration management
while rolling out new Tomcat versions without breaking things. Now the
critical question: will this break the previous versions or will they
just ignore unkown parameters?

Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [ANN] Apache Tomcat 9.0.31 available

[Stefan Mayr]

Hi,

> - AJP defaults changed to listen the loopback address, require a secret
>   and to be disabled in the sample server.xml

What was the motivation behind this breaking change to require a secret
or to explitly disable it? What makes an open AJP connector more unsafe
than an open HTTP connector?

We have hundreds of Tomcats behind Apache httpd with mod_jk. My
interpretation is that upgrading Tomcat 8.5 or 9.0 will break that setup
until we disable the secret in all of them (or add a secret in mod_jk
and Tomcat).
I would understand that for a new major version 10.x but not in the
lifecycle of an existing major version.

Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: ECDSA Private Keys

[Stefan Mayr]

Am 09.01.2020 um 21:45 schrieb Christopher Schultz:
> DSA is almost never used. Nearly 100% of keys in the world are
> plain-RSA or EC. I know of no CA that uses DSA for signing. So pretty
> much every cert you will come across will be EC-with-RSA or
> RSA-with-RSA (that's keytype-with-signature-type).

I guess this might change. Let's Encrypt is quite popular and plans to
change to ECDSA for Root and Intermediate certificates

Source: https://letsencrypt.org/upcoming-features/

 - Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Jakarta EE 9

[Stefan Mayr]

Am 28.10.2019 um 22:07 schrieb Michael Osipov:

Am 2019-10-28 um 22:00 schrieb Stefan Mayr:

Am 28.10.2019 um 14:13 schrieb Rémy Maucherat:

On Mon, Oct 28, 2019 at 1:46 PM Johan Compagner 
wrote:


Hi



On Mon, 28 Oct 2019 at 13:15, Mark Thomas  wrote:


Hi all,

A frequent topic of discussion at ApacheCon EU was Jakarta EE 9. For

those

of you who aren't familiar with Jakarta EE the key points are:

- Oracle have donated Java EE to Eclipse
- Eclipse have released Jakarta EE 8 which is essentially identical to
Java EE 8
- Oracle have refused to allow changes to the APIs in the javax 
namespace

- The Jakarta EE community seem to be reaching consensus on releasing
Jakarta EE 9 which will rename all the Java packages from javax.* to
jakarta.*




what does this rename really mean?

import javax.servlet.http.HttpSession;
import javax.websocket.Session;

those are renamed?
If that is yes that would mean pretty much everything will break?



https://eclipse-foundation.blog/2019/05/03/jakarta-ee-java-trademarks/
I thought everyone knew about this. We were supposed to have a 
session on

this rename at ApacheCon EU, but unfortunately it didn't happen.

Rémy


This article mentions that javax.* package namespace is not allowed to
change. The API needs to remain compatible.
When javax.* is renamed to jakarta.* it should be sufficient to have a
javax.* shim library that translates everything to use jakarta.*. Or is
there any public information that Oracle prohibits that too?


This will only work if the API won't change. I consider such a shim to 
be dangerous at some point in time.


Michael


Time is the right cue: having both namespaces would allow a transition 
phase. The javax shim could (or should!) trigger a deprecation warning.


And I hope the API won't change too soon. It would be good for the 
Jakarta EE acceptance if they keeping existing interfaces stable for 
some time.


- Stefan







-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Jakarta EE 9

[Stefan Mayr]

Am 28.10.2019 um 14:13 schrieb Rémy Maucherat:
> On Mon, Oct 28, 2019 at 1:46 PM Johan Compagner 
> wrote:
> 
>> Hi
>>
>>
>>
>> On Mon, 28 Oct 2019 at 13:15, Mark Thomas  wrote:
>>
>>> Hi all,
>>>
>>> A frequent topic of discussion at ApacheCon EU was Jakarta EE 9. For
>> those
>>> of you who aren't familiar with Jakarta EE the key points are:
>>>
>>> - Oracle have donated Java EE to Eclipse
>>> - Eclipse have released Jakarta EE 8 which is essentially identical to
>>> Java EE 8
>>> - Oracle have refused to allow changes to the APIs in the javax namespace
>>> - The Jakarta EE community seem to be reaching consensus on releasing
>>> Jakarta EE 9 which will rename all the Java packages from javax.* to
>>> jakarta.*
>>>
>>>
>>
>> what does this rename really mean?
>>
>> import javax.servlet.http.HttpSession;
>> import javax.websocket.Session;
>>
>> those are renamed?
>> If that is yes that would mean pretty much everything will break?
>>
> 
> https://eclipse-foundation.blog/2019/05/03/jakarta-ee-java-trademarks/
> I thought everyone knew about this. We were supposed to have a session on
> this rename at ApacheCon EU, but unfortunately it didn't happen.
> 
> Rémy
> 
This article mentions that javax.* package namespace is not allowed to
change. The API needs to remain compatible.
When javax.* is renamed to jakarta.* it should be sufficient to have a
javax.* shim library that translates everything to use jakarta.*. Or is
there any public information that Oracle prohibits that too?

Regards,

  Stefan


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Multiple Hosts in server.xml - expected behaviour?

[Stefan Mayr]

Hi,

Am 31.08.2018 um 19:22 schrieb Christopher Schultz:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> Stefan,
> 
> On 8/30/18 23:27, Stefan Mayr wrote:
>> we got an application has multiple Service, Engine and Host entries
>> in server.xml. All Host sections share the same directory:
>>
>> appBase="webapps" unpackWARs="true" autoDeploy="true"
>>
>> Each host contains Context-Section using docBase="webapps/appX"
>> for diffenent applications appX.
>>
>> Is this a valid configuration (for Tomcat 7.0)? If yes, what should
>> we expect to happen?
> 
> Yes, this is valid. You should expect that each  will deploy a
> separate copy of each web application found in "webapps". If you have
> 3 s and 3 web applications in "webapps", you should expect
> Tomcat to start 9 separate contexts.
> 
>> We currently see and endless loop of application deployments that
>> fail because the ports of the Service Connector is already in use.
> 
> Something else is wrong, then. Connectors are separate from context
> (web app) deployments. Application deployments shouldn't attempt to
> configure connectors.
> 
> Can you post an error and/or stack trace?
> 
> - -chris

You were right. I started counting deployment log entries and failing
contexts. For each context we have n deploments, n being the number of
appBase definitions in Host entries. And because all of those apps seem
to bind additional local tcp ports we have n-1 failing contexts per
Host. Only the first one can bind that port. That led to a lot more
java.net.BindExceptions beeing thrown. I misinterpreted them to be
Connector ports.

Thank you,

   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Multiple Hosts in server.xml - expected behaviour?

[Stefan Mayr]

Hi,

we got an application has multiple Service, Engine and Host entries in
server.xml. All Host sections share the same directory:

appBase="webapps"
unpackWARs="true"
autoDeploy="true"

Each host contains Context-Section using docBase="webapps/appX" for
diffenent applications appX.

Is this a valid configuration (for Tomcat 7.0)? If yes, what should we
expect to happen?

We currently see and endless loop of application deployments that fail
because the ports of the Service Connector is already in use.

Thank you,

Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Documentation for Catalina Base

[Stefan Mayr]

Hi,

Am 30.07.2018 um 18:25 schrieb Christopher Schultz:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA256
> 
> Marek,
> 
> On 7/30/18 3:22 AM, Marek Czernek wrote:
>> Hi there,
>>
>> recently, we noticed the lack of documentation for Catalina Base,
>> and I wanted to find out more about it. Other than a strangely
>> obscure txt file [1], I did not find the concept and usage of
>> Catalina Base documented in the Tomcat docs.
> 
> Do you mean the CATALINA_BASE environment variable, which is copied
> into a running JVM as the "catalina.base" system property?
> 
>> My questions are:
>>
>> 1) Did I overlook it being documented somewhere?
> 
> Other than in RUNNING.txt (where it is thoroughly described), it is
> documented in:
> 
> 1. bin/catalina.sh and bin/catalina.bat
> 2. The "Directories and Files" section of each Tomcat X documentation
> (e.g.
> https://tomcat.apache.org/tomcat-9.0-doc/introduction.html#Directories_a
> nd_Files)
> 3. The "Introduction" section of each Tomcat X configuration guide for
> the "context" container (e.g.
> https://tomcat.apache.org/tomcat-9.0-doc/config/context.html#Introductio
> n)
> 
>> 2) If not, is there any reason for it not being documented? While
>> I don't want to duplicate text, I do not consider a txt file
>> somewhere a sufficient docs.
> 
> What, it needs to be in TeX, too?
> 
>> If the answer to both of my questions is no, we'll create a patch 
>> against the Tomcat docs.
>>
>> [1] https://tomcat.apache.org/tomcat-9.0-doc/RUNNING.txt
> 
> Patches are always welcome, but I believe it is currently
> properly-documented.
> 
> Is there something specific you were looking for and could not find?

That's the point to ask for some details I'm struggeling with

1. $CATALINA_BASE/bin/tomcat-juli.jar: when do I need to place this jar
in this directory? It is only clear to have it in CATALINA_BASE for the
documented switch to log4j. From reading catalina.sh I would say it is
only necessary to place it in CATALINA_BASE when we want to use a
different version. Is this assumption correct?

2. $CATALINA_BASE/conf: What files have to be in this directory? If they
are not present - are they loaded from CATALINA_HOME? RUNNING.txt only
mentions server.xml. But it looks like we need to have at least a copy
of web.xml in this directory too. Reading catalina.sh we also should
copy (or link) catalina.policy when running with a security manager. For
other files the situation is not clear, e.g. catalina.properties,
context.xml, ... Some stackoverflow answers to that question recommend
to copy the whole conf directory from CATALINA_HOME to CATALINA_BASE.
I'm not sure if this is a good idea if I don't need to change the
defaults. The documentation could be improved by listing the files that
are mandatory in CATALINA_BASE

Regards,

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: TomcatCon London slides and recordings

[Stefan Mayr]

Hi

Am 04.10.2017 um 19:27 schrieb Mark Thomas:
> ... are now (mostly) available:
> 
> http://tomcat.apache.org/presentations.html
> 
> I thought Jean-Frederic sent me his slides but I can't find the e-mail.
> I'm sure one of us will update that page shortly.

Reverse Proxies, Load-Balancing & Clustering - Part 1 and Part 2 link
the same slides.

Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [SECURITY] CVE-2017-5647 Apache Tomcat Information Disclosure

[Stefan Mayr]

Hi,

Am 10.04.2017 um 21:14 schrieb Mark Thomas:
> CVE-2017-5647 Apache Tomcat Information Disclosure
> 
> Severity: Important
> 
> Vendor: The Apache Software Foundation
> 
> Versions Affected:
> Apache Tomcat 9.0.0.M1 to 9.0.0.M18
> Apache Tomcat 8.5.0 to 8.5.12
> Apache Tomcat 8.0.0.RC1 to 8.0.42
> Apache Tomcat 7.0.0 to 7.0.76
> Apache Tomcat 6.0.0 to 6.0.52
> 
> Description
> A bug in the handling of the pipelined requests when send file was used
> resulted in the pipelined request being lost when send file processing
> of the previous request completed. This could result in responses
> appearing to be sent for the wrong request. For example, a user agent
> that sent requests A, B and C could see the correct response for request
> A, the response for request C for request B and no response for request C.

How about the pipelining: this reuses a TCP connection for a single
user. Is it possible for two different users (TCP connections) to see a
response for the other user?

> Mitigation:
> Users of the affected versions should apply one of the following
> mitigations:
> - Switch to the BIO HTTP where available
> - Disable send file
> - Upgrade to Apache Tomcat 9.0.0.M19 or later
> - Upgrade to Apache Tomcat 8.5.13 or later
> - Upgrade to Apache Tomcat 8.0.43 or later
> - Upgrade to Apache Tomcat 7.0.77 or later
> - Upgrade to Apache Tomcat 6.0.53 or later

Does this also affect requests to the AJP connector (Apache httpd +
mod_jk in front of Tomcat)?

> Credit:
> This issue was identified by the Tomcat security team.
> 
> History:
> 2017-04-10 Original advisory
> 
> References:
> [1] http://tomcat.apache.org/security-9.html
> [2] http://tomcat.apache.org/security-8.html
> [3] http://tomcat.apache.org/security-7.html
> [4] http://tomcat.apache.org/security-6.html
> 
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 

Thanks,

   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 8.5.5 (8.5+) Default Cookie Processor breaks persistent cookies for all IE versions

[Stefan Mayr]

Am 05.11.2016 um 23:58 schrieb Mark Thomas:
> On 04/11/2016 19:10, Hedrick, Brooke - 43 wrote:
>> Sorry if this has been already asked.   I searched the archives and
>> didn't find what I was looking for.
> 
> I don't recall anyone raising it before now.
> 
>> Has anyone else run into an issue with persistent cookies in Tomcat
>> 8.5+ and IE not working?
> 
> I can confirm I see the same issue.
> 
>> Does it make sense that the shipping configuration would not work
>> with IE for persistent cookies?
> 
> I'll turn that around. The shipping Tomcat configuration is RFC 6265
> compliant. Does it make sense that Microsoft would ship multiple
> versions of their browser for over a decade and fail to correctly
> implement any of the cookie specifications that were considered current
> throughout that period? (IE's cookie support is a sore point for me - I
> have been dealing with IE's spec non-compliance for almost as long as I
> have been working on Tomcat and it has always been unpleasant.)

When I read
https://blogs.msdn.microsoft.com/ieinternals/2009/08/20/internet-explorer-cookie-internals-faq/
and the last response to
https://developer.microsoft.com/en-us/microsoft-edge/platform/issues/8183708/
from the Microsoft Edge Team I fear full RFC6265 support is still some
years away in Microsoft world

> The default Tomcat community position in cases like this is that we do
> not implement workarounds for bugs in third-party code. You need to
> raise a bug with the provider of the buggy code.
> 
> We do make exceptions and they are typically for IE. Part of me thinks
> that if everyone refused to work-around Microsoft's poor implementations
> of various standards (WebDAV is another area that comes to mind) a)
> people would see just how bad some Microsoft code really is and b)
> Microsoft might come under pressure to actually fix it.
> 
> While we could make a stand on this particular point, I suspect that
> Microsoft won't even notice and all it will do is make life difficult
> for our users. As annoyed as I am with Microsoft about this, making life
> difficult for Tomcat users is not what this community is about. As much
> as it pains me to say it, I think we are going to have to work around this.
> 
> Maybe an new option:
> enableWorkaroundForBrokenMicrosoftCookieHandling
> 
> Seriously, we need to decide if this needs to be configurable or not.
> Given that RFC6265 allows both expires and max-age to be sent and the
> the legacy processor sends both by default I'm currently leaning towards
> just sending both in the RFC6265 processor.

+1 sending both headers

Assume the following: people upgrade Tomcat and the app stops working in
IE (most corporate users default browser). They will blame Tomcat - not
IE. Why should we risk to damage Tomcats reputation if sending both
headers is still standards compliant? This "hack" seems quite acceptable
for me. Adding a configuration option for a "strict" mode would make it
easier to test future browser implementations with real applications.

> Assuming no-one objects, I'll aim to get this fixed for the next release
> (not the one currently in progress but the one expected early next month).
> 
> We also need to update the note in the docs about IE versions.
> 
> Mark
> 
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
> 
> 

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Code signing WAR and verification

[Stefan Mayr]

Am 09.08.2016 um 19:48 schrieb Mark Thomas:

On 09/08/2016 18:29, Stefan Mayr wrote:

Hi,

two colleagues came with an idea that our new java platform should only
run signed code. In the java world I've only seen signed java applets.
From a bit of internet research it looks like any JAR, WAR or EAR can be
signed with jarsigner (maybe all zip files?).

Some sources indicate that this is supported or verified in WebLogic. So
how about Tomcat? Is there any verification of signed code or are there
any configuration flags to enable/enforce/disable this?

I would guess the signature is ignored. Am I wrong?


You are correct. Signatures on a WAR will be ignored.

https://bz.apache.org/bugzilla/show_bug.cgi?id=52489


I don't see a signature verification in the patch. But from the 
description it might be enough to trigger the SecurityManager somehow.



I'm far from convinced that the proposed patch on that issue is sufficient.

I'm also not convinced that there is a standard for signing WARs. Some
authoritative references (i.e. to official Java SE or Java EE docs)
would be very helpful.

Mark


Specs are hard to find. For jars a nice description can be found in [1]. 
The servlet spec [2] mentions that "Web applications can be packaged and 
signed into a Web ARchive format (WAR) file using the standard Java 
archive tools." But when I ran over the servlet spec I did not find a 
description how the servlet container should handle signed war files. Or 
is this delegated to the security manager? This is still a mystery to 
me. Especially when I think of think of JSPs and their on-demand 
compilation. What can be the magic phrase we should look for?


    Stefan Mayr

[1] 
http://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Signed_JAR_File
[2] 
http://download.oracle.com/otndocs/jcp/servlet-2.4-fr-spec-oth-JSpec/ 
see SRV.9.6



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Code signing WAR and verification

[Stefan Mayr]

Hi,

two colleagues came with an idea that our new java platform should only 
run signed code. In the java world I've only seen signed java applets. 
From a bit of internet research it looks like any JAR, WAR or EAR can 
be signed with jarsigner (maybe all zip files?).


Some sources indicate that this is supported or verified in WebLogic. So 
how about Tomcat? Is there any verification of signed code or are there 
any configuration flags to enable/enforce/disable this?


I would guess the signature is ignored. Am I wrong?

Thank you,

  Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Can tomcat serve MPI (parallel) applications?

[Stefan Mayr]

Am 29.11.2015 um 19:24 schrieb Martijn Slouter:

Thanks for your reply, comments below:
...
Any suggestion how I can accomplish the configuration, if I start
tomcat with the MPI web application using "mpirun -n 2 java ..." so
that only the first MPI process opens the tomcat communication ports,
while all other MPI processes disable their communicators?

As an alternative I can run the MPI application as a separate server
(tested across 16 hosts already), and use tomcat as a (serial) client
to this parallel server. The disadvantage is that huge amounts of data
need to be processed another time instead of being served directly
from the MPI application.



How does mpirun communicate to the started java process that it is the 
first process? Maybe it is easier to write a wrapper that can decide 
which tomcat configuration to use depending if this is your master 
process or not. If you write this wrapper in java you could use an 
embedded tomcat or jetty to startup a servlet container where needed.


Regards,

   Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Http 2 support in Tomcat

[Stefan Mayr]

Am 15.05.2015 um 20:23 schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 5/15/15 12:59 PM, Mark Thomas wrote:

On 15/05/2015 17:49, Christopher Schultz wrote:

Tony,

On 5/15/15 12:06 PM, PerfGuru wrote:

Good news thanks for the update. It may be some time before
servlet 4 support is released but good to know Tomcat will be
ready then. But does APR support http/2? I know other web
servers now do.


Mark's initial work on the HTTP/2 connector has been using APR.
Note that APR doesn't handle the HTTP protocol, it just helps
manage sockets, pools, etc. The protocol needs to be built on top
of those building blocks. Please see the dev list (archives) for
details. Just search for HTTP/2 subjects.


Correct. This is primarily because ALPN support is not there in
JSSE yet and wont be until Java 9. That said, one of the reasons
for the huge amount of connector refactoring that I did before
starting on HTTP/2 was so that the connector implementation
specific code was kept to a minimum (so far just ALPN). HTTP/2 will
work just fine with NIO and NIO2 just as soon as JSSE support ALPN
and we plumb it in.


Is this something that's workable with the current pre-release builds
of Java 9? Presumably, Servlet 4 will require Java 8. Does that mean
that HTTP/2 with the NIO connectors will only be available with a
higher JRE version?


h2c support (HTTP/2 over a cleartext channel via HTTP upgrade
rather than TLS) is also on the TODO list. It shouldn't be much
work to add (less than a day) and it wil be available for all
connectors.




Also, I think Tomcat supports websockets which has the
important capabilities that http/2 has and has been available
longer.


Tomcat supports Websocket since Tomcat 7, as long as you are
running on Java 7 or higher.


Depending upon your strategy you might want to go with that or
wait a couple of years for http/2. Remember the firewalls,
proxies and other network systems have to support http/2 before
it can reach Tomcat. So it may be by the time http/3 is
approved.


Tomcat has no choice (application developers do). Servlet 4.0 will
require HTTP/2 so Tomcat has to support it.


Correct. Using Websocket through certain proxies (e.g. httpd) is
even still sometimes tricky.


Yeah. We should touch base with jimjag and discuss this. I think
the httpd thinking around WebSockets and the Java world thinking
about WebSockets aren't quite lined up.


+1

Perhaps there isn't much use for Websockets in the Perl or PHP world,
where a lot of httpd installations are being used.

Jim Riggs's (reprise, I believe) presentation at ApacheCon about how
mod_php needs to die (paraphrasing) advocates more extensive use of
mod_proxy_* to physically separate the web tier from the application
tier, in a way that more closely resembles how Java webapps are
deployed. Perhaps if the world starts listening to the Jim Riggses of
the world, the shortcomings of mod_proxy_websocket and friends will
become more apparent.


Do you see any problems using a HTTP/2 enabled proxy in front of Tomcat 
HTTP or AJP connectors? Combining httpd with mod_h2 and mod_jk should 
give us enough time for Servlet 4.0 -  assuming that the critical slow 
bandwith, high latency path between the client and the proxy benefits 
most of the new procotol.


- Stefan


--
Mayr Stefan

Hausen - Gassenaecker 10
82269 Geltendorf

Tel.: 08193 - 9979469

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat Connection Pool Problems with XtraBackup

[Stefan Mayr]

Hi John,

Am 11.10.2014 23:30, schrieb John Smith:

I'm trying to workout a managed backup scheme on a MySQL production
database with XtraBackup. According to our DBA, XtraBackup doesn't lock the
database, but issues a series of SHOW TABLE STATUS commands and then works
on the file system level.

Still, just as XtraBackup runs my logs start to blow up with connection
pool errors:

Error getting database connection:[http-nio-8080-exec-5] Timeout: Pool
empty. Unable to fetch a connection in 10 seconds, none available[size:100;
busy:100; idle:0; lastwait:1].

There's some suggestion that turning off 'innodb-stats-on-metadata' might
help so we're trying that.

...
Anything else seem like it could be tweaked in relation to XtraBackup?

TIA,
John



XtraBackup can operate without locks if your database contains only 
InnoDB tables. For other storage engines like MyISAM it still uses 
locks. A problem we seen is high IO load during XtraBackup. Your DBA 
could check iostats while backup is in progress. If this makes your 
database server unresponsive he could try to throttle the backup: 
http://www.percona.com/doc/percona-xtrabackup/2.2/innobackupex/throttling_ibk.html


-Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Forward TLS connection information from AWS ELB - httpd - Tomcat

[Stefan Mayr]

Am 01.10.2014 19:18, schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-

...

What I'm mainly looking for is a way to say the incoming
connection (from ELB) is HTTP and I want to pretend that the
connection is HTTPS.


Then the easier solution seems using ELB for SSL termination and
using the X-Forwarded-Proto header, passing from apache to tomcat


Yes. Just looking for a way to say oh, the connection is also encrypted.


If I remember correctly this needs only one line in Apache httpd to 
forward it to Tomcat


SetEnvIf X-Forwarded-Proto https HTTPS=on

mod_jk should use this information and mark it as a secure connection 
for you. Then you can require a secure connection in your webapp web.xml 
or check it in httpd with the same environment variable:


Order Deny,Allow
Deny from all
Allow from env=HTTPS

If the httpd is only a helper process to pass this information to Tomcat 
you can also use the Proxy-Valves: 
http://tomcat.apache.org/tomcat-7.0-doc/config/valve.html#Proxies_Support


Something like this should serve your purpose:
Valve
 className=org.apache.catalina.valves.RemoteIpValve
 protocolHeader=x-forwarded-proto
 portHeader=x-forwarded-port
/

Togehter with transport-guarantee CONFIDENTIAL in your web.xml this 
would eliminate the need to configure anything on Apache httpd at all.


- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Forward TLS connection information from AWS ELB - httpd - Tomcat

[Stefan Mayr]

Hi Christopher,

Am 01.10.2014 20:40, schrieb Christopher Schultz:

-BEGIN PGP SIGNED MESSAGE-

...


I've been trying to get mod_remoteip to handle the client's IP address
for me -- especially for logging -- but I'm having some difficulty and
have asked a question over on the httpd users' list about that.

...

mod_remoteip can do more for you (if you need it). Just for the purpose 
of logging the X-Forwarded-For-Header you could define a custom 
LogFormat and add request headers like \%{X-Forwarded-For}i\ to it.


- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to create an Oracle XA Datasource within Tomcat 7

[Stefan Mayr]

Am 29.08.2014 14:23, schrieb Sylvain Goulmy:

Thank you for your feedbacks.

I had actually already tried a configuration with a url parameter. With
this configuration :

   Resource

factory=org.apache.tomcat.jdbc.naming.GenericNamingResourcesFactory
name=jdbc/mydatasource-xa password=password
type=oracle.jdbc.xa.client.OracleXADataSource user=userName
url=jdbc:oracle:thin:@myHost:1522:myDataBase /



Did you try the other option of adding driverType=thin and 
portNumber=1521 ?


Although I don't get why makeURL is called if an URL is set. From what I 
find via google this could be a bug in the oracle driver.


Regards,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to create an Oracle XA Datasource within Tomcat 7

[Stefan Mayr]

Am 27.08.2014 11:05, schrieb Sylvain Goulmy:

Hi all,

I'm trying to setup a XA Datasource configuration in Tomcat 7 with an
Oracle database.

I have spent a long time trying to set that configuration up without any
success.

I have read the  following blog entry which is i think the only ressource
that deals with this subject :

http://www.tomcatexpert.com/blog/2010/04/01/configuring-jdbc-pool-high-concurrency

with the Pooling connections from other data sources paragraph.

Here is the configuration i have done in my context.xml file :

   Resource

factory=org.apache.tomcat.jdbc.naming.GenericNamingResourcesFactory
name=jdbc/mydatasource-xa password=password
type=oracle.jdbc.xa.client.OracleXADataSource user=userName
databaseName=databaseName serverName=myDatabaseHost portNumber=1522 /


...

and here is the exception i have when the application starts :

2014-08-27 08:25:49,748 [default-startStop-1] ERROR

org.apache.tomcat.jdbc.pool.ConnectionPool- Unable to create initial
connections of pool.
java.sql.SQLException: Invalid Oracle URL specified:
OracleDataSource.makeURL
  at oracle.jdbc.pool.OracleDataSource.makeURL(OracleDataSource.java:1277)
at oracle.jdbc.pool.OracleDataSource.getURL(OracleDataSource.java:1025)


The JDBC-Driver cannot create the JDBC-URL from these parameters: use 
url instead of serverName, databaseName, portNumber e.g. 
url=jdbc:oracle:thin:@myhost:1521/orcl or add a driverType, e.g. 
driverType=thin


see http://www.orafaq.com/wiki/JDBC for URLs and 
http://docs.oracle.com/cd/E18283_01/appdev.112/e13995/oracle/jdbc/pool/OracleDataSource.html 
for the possible parameters and default values.


Regards,

Stefan

--
Mayr Stefan

Hausen - Gassenaecker 10
82269 Geltendorf

Tel.: 08193 - 9979469

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: How to configure persistence session cookie with apache tomcat

[Stefan Mayr]

Hi Sapna,

Am 15.08.2014 01:51, schrieb Sapna Kumari:

Hi All,

My apologies if the questions are too stupid or naive.

The problem is I have two servers under my vip (oia-vip.example.com) When I
have either of the one servers(server1 or server2) running  we are able to
login to my application(oracle Identity Analytics - OIA ) fine.

But when our application instance is running  on both the servers under a
VIP, then we face issues  post logging in to the application.


From the OIA logs, Initially when we hit the application url, it goes to

one server, we are able to login just fine but when we click on any link in
the application it goes to the other server under the VIP and logs us out
of the application.

  When we have only one instance running under the VIP we do not face any
login issues coz all the traffic is directed to one server only.

  Now My n/w guy says *the LTM is looking for a cookie being passed from
the application in the HTTP_RESPONSE headers which match this name:
App_passive_cookie*

  Now how  do I know what cookie is being passed by my application ?

My Apache tomcat version is 5.5.33

OIA version is 11.1.1.3



If I understand you correctly there is a f5 BigIP Loadbalancer in front 
of your Tomcat Servers. A usual application will not send this very 
specific response Header. You will need to customize your application. 
Another solution could be to change the persistence mode on the LTM.


According to 
http://support.f5.com/kb/en-us/products/big-ip_ltm/manuals/product/ltm_configuration_guide_10_0_0/ltm_persist_profiles.html 
the LTM seems to support the HTTP Cookie Insert Method. If your 
application already uses session cookies for persistence then another 
persistence cookie inserted by the loadbalancer should work without 
changes on your existing application.


To check for cookie you can use the developer tools included in IE, 
Firefox, Chrome, etc.


e.g. Firefox: open console with CRTL+Shift+J, Activate network and 
reload your webpage. When you see the server response click on the 
response code on the right side. A new window will show a table with 
Cookie and Response-Header information.


As Oracle seems to support cluster configuration of OIA you could also 
ask their support about the setup requirements.


Stefan

--
Mayr Stefan

Hausen - Gassenaecker 10
82269 Geltendorf

Tel.: 08193 - 9979469

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Packet misses in Tomcat

[Stefan Mayr]

Am 13.01.2014 07:22, schrieb Divyaprakash Y:

-Original Message-
From: Stefan Mayr [mailto:ste...@mayr-stefan.de]
Sent: 10 January 2014 23:26
To: users@tomcat.apache.org
Subject: Re: Packet misses in Tomcat

Hi

Am 09.01.2014 14:21, schrieb Divyaprakash Y:



-Original Message-
From: Divyaprakash Y
Sent: 08 January 2014 14:35
To: Tomcat Users List
Subject: RE: Packet misses in Tomcat


...

Strange that this is happening only to me.

Looks like something similar was reported on the dev list when voting for 
Tomcat 7.0.50 ..

Thanks.




I tried same setup today with the BIO connector, everything worked flawlessly. 
Will there be any issue with the APR connector(earlier setup) or are there any 
extra configurations which I missed in my server.xml?


This might be the issue seen in
https://issues.apache.org/bugzilla/show_bug.cgi?id=55976 . Looks like Mark 
fixed it today for 7.0.51 (not released yet)

- Stefan





Thanks Stefan for the information.

Should that(Fix on NIO Connector) fix the possible issue in APR Connector as 
well?


I guess not. But maybe you shoudl 7.0.50 give a try. APR was updated in 
one of the not released versions after your lastest 7.0.42.
Also a minimal testcase would help to check if this is reproducable on 
other peoples machine (like they did in the bugzilla ticket)


- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Packet misses in Tomcat

[Stefan Mayr]

Hi

Am 09.01.2014 14:21, schrieb Divyaprakash Y:



-Original Message-
From: Divyaprakash Y
Sent: 08 January 2014 14:35
To: Tomcat Users List
Subject: RE: Packet misses in Tomcat

-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: 08 January 2014 02:52
To: Tomcat Users List
Subject: Re: Packet misses in Tomcat

Christopher,

Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

André,

On 1/7/14, 5:09 AM, André Warnier wrote:

I do not pretend to know your system, nor your application, nor that
the following is a definite explanation.  But on the base of the
currently available data, I would say : - it is quite unlikely that
Tomcat 7 is randomly dropping requests. If it was, then I would
imagine that this list would be overflowing with cries for help.
There is quite a bit of traffic on this list related to Tomcat 7, but
I don't recall seeing any significant number of issues mentioning
dropped requests. - it also doesn't seem, from your
wireshark-related observations, that the requests are being lost
outside of Tomcat. - so I would say at this point, that the most
likely place for requests to disappear is in your own application.



...

Strange that this is happening only to me.
Looks like something similar was reported on the dev list when voting 
for Tomcat 7.0.50

..

Thanks.




I tried same setup today with the BIO connector, everything worked flawlessly. 
Will there be any issue with the APR connector(earlier setup) or are there any 
extra configurations which I missed in my server.xml?


This might be the issue seen in 
https://issues.apache.org/bugzilla/show_bug.cgi?id=55976 . Looks like 
Mark fixed it today for 7.0.51 (not released yet)


- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Mapping security role outside of servlet specification

[Stefan Mayr]

Am 11.06.2013 19:42, schrieb Felix Schumacher:

Am 05.06.2013 13:42, schrieb Ilya Kazakevich:

Hello,

I use probe app for several tomcats.
It's security model is based on servlet API security: security roles and
constraints are provided in web.xml and mapped for each servlet it has.
I use JNDIRealm to authenticate users via LDAP (actually AD is used).
I want to grant access to Probe on Tomcat1' installation for people from
Group1  and to Tomcat2 to people from Group2.
But manager group is hard-coded in web.xml inside of probe!

I want to map group Group1 from LDAP realm to role manager in tomcat.
How can I do that?
I can't use security-role-ref because I should put in servlet tag,
so I
can't
do that on tomcat level.
There is  security-role-mapping tag in some servers but  it is
proprietary
and not supported by tomcat.

So the only way for me to extract .war file and patch web.xml.

That is the way, we do it at work. For probe, that is.

For other applications we are using different Attributes in the ldap
group objects to map to different names for the role.

Greetings
  Felix


What is the best way to solve it?

Ilya.


What would be the best way to do this on the tomcat level? I would like 
to add a simple mapping.properties to our tomcat installations 
containing the individual mappings in the form of security-role=jndi-role.


I see multiple options:
1. create a nestable realm to wrap around the JNDIRealm (Tomcat7)
2. create an own realm extending JNDIRealm
3. provide a patch to JNDIRealm

pseudo-code for overriding hasRole could be like this

public boolean hasRole(..) {
  String mappedRole = mapping.getProperty(role);
  if((mappedRole != null)  super.hasRole(..,mappedRole)) {
return true;
  } else {
return false;
  }
}

I'm not sure wether it is also required to override the authenticate 
methods which return the principal object.


Any recommendations?

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Mapping security role outside of servlet specification

[Stefan Mayr]

Am 23.08.2013 20:10, schrieb Stefan Mayr:

Am 11.06.2013 19:42, schrieb Felix Schumacher:

Am 05.06.2013 13:42, schrieb Ilya Kazakevich:

Hello,

I use probe app for several tomcats.
It's security model is based on servlet API security: security roles and
constraints are provided in web.xml and mapped for each servlet it has.
I use JNDIRealm to authenticate users via LDAP (actually AD is used).
I want to grant access to Probe on Tomcat1' installation for people
from
Group1  and to Tomcat2 to people from Group2.
But manager group is hard-coded in web.xml inside of probe!

I want to map group Group1 from LDAP realm to role manager in
tomcat.
How can I do that?
I can't use security-role-ref because I should put in servlet tag,
so I
can't
do that on tomcat level.
There is  security-role-mapping tag in some servers but  it is
proprietary
and not supported by tomcat.

So the only way for me to extract .war file and patch web.xml.

That is the way, we do it at work. For probe, that is.

For other applications we are using different Attributes in the ldap
group objects to map to different names for the role.

Greetings
  Felix


What is the best way to solve it?

Ilya.


What would be the best way to do this on the tomcat level? I would like
to add a simple mapping.properties to our tomcat installations
containing the individual mappings in the form of
security-role=jndi-role.

I see multiple options:
1. create a nestable realm to wrap around the JNDIRealm (Tomcat7)
2. create an own realm extending JNDIRealm
3. provide a patch to JNDIRealm

pseudo-code for overriding hasRole could be like this

public boolean hasRole(..) {
   String mappedRole = mapping.getProperty(role);
   if((mappedRole != null)  super.hasRole(..,mappedRole)) {
 return true;
   } else {
 return false;
   }
}

I'm not sure wether it is also required to override the authenticate
methods which return the principal object.

Any recommendations?

   Stefan


I've created a bugzilla entry and attached a prototype for option 1: 
https://issues.apache.org/bugzilla/show_bug.cgi?id=55477


I only tested with a UserDatabaseRealm so far. I hope to find some time 
at work to test it with a JNDIRealm.


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: binary distribution - how to upgrade ?

[Stefan Mayr]

Hi Bernd,

Am 20.06.2013 18:45, schrieb Lentes, Bernd:

Hi,

i'm fairly new to tomcat. We have a SLES 10 SP4 64bit host, running Tomcat 5.5, 
which was provided as a rpm from the distributor.
Our developers need now a more current version, 6 or prefered 7.
I didn't find rpm's for these versions for my OS. So i have to install the 
binary version from the tomcat web page.
As a side note: general support for SLES10 ends with july 2013. If you 
don't want to buy extended support you should switch to SLES11 as soon 
as possible. This includes tomcat6 instead of tomcat5. SUSE uses version 
6.0.18 but backports security patches.


Though I would still recommend the official tomcat binaries from the 
apache website. It's much easier to support a well known state that your 
developers can also install on their workstation.


- Stefan


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Delayed WAR expansion, timeout on context startup?

[Stefan Mayr]

Am 14.05.2013 16:13, schrieb Jeffrey Janner:

-Original Message-
From: Mayr Stefan [mailto:ste...@mayr-stefan.de]
Sent: Tuesday, May 14, 2013 5:17 AM
To: Tomcat Users List
Subject: RE: Delayed WAR expansion, timeout on context startup?

  Hi Chuck,

  Am Montag, den 13.05.2013, 15:44 +0200 schrieb Caldarale, Charles R
  chuck.caldar...@unisys.com:

From: Stefan Mayr [mailto:ste...@mayr-stefan.de]
Subject: Delayed WAR expansion, timeout on context startup?



Are there any parameters to adjust deployment timeouts for these
contexts with large WAR files?


Want to give us a hint about what Tomcat version, JVM level, and
platform you're running on?  It's rather difficult to answer _any_
question without the basics.


  Oh, that's a wonderful mix of applications and versions. Most common
are

  OS: SLES10 SP4, SLES11 SP2, both using TrendMicro ServerProtect 3
  Java: Java 6 U20,U33,U37,U45; Java 7 U09, U21  Tomcat 6.0.20, 6.0.35,
6.0.35; 7.0.26, 7.0.27, 7.0.32

  Developers are crying when we move applications to the newer os with
an  active antivirus scanner. This affects all version combinations.

  The specific application failing is alfresco 4 running on Java 6 U33,
Tomcat 6.0.35



Sorry, this would have been a better reply to the OP, but here goes.
Have you thought about adding the webapps directory to the anti-virus' list of 
ignored directories?
I assume that the system(s) these WAR files are coming from also have 
anti-virus problems, but if they don't or are otherwise untrustworthy, modify 
your deployment procedure to first move the WAR file to a system/directory 
where it would be scanned, then move it from there to the production webapps 
directory.
Jeff

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Yes we did but hundreds of tomcats with different applications on 
different servers and enterprise procedures for changing this settings ...


Wrong list for this type of problem :-)

Thanks,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Delayed WAR expansion, timeout on context startup?

[Stefan Mayr]

Hi,

we've upgraded our linux systems and experience some serious slowdowns 
from our antivirus solutions. Expanding large WAR files (e.g. Alfresco) 
is blocked by the virus scanner for minutes (!). Now we get tomcats 
failing to deploy contexts on startup. Everything was fine when the 
deployment/WAR expansion is/was faster (av off or just older linux 
plattform).
Are there any parameters to adjust deployment timeouts for these 
contexts with large WAR files?


I know we need to get the av scanner fixed. But at the moment we just 
need a temporary workaround for the longer expansions times.


Thanks,

  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Configuring Tomcat for generating log file (catalina.out) (Tomcat 5.0)

[Stefan Mayr]

Am 18.01.2013 20:06, schrieb Gabriel Huerta Araujo:

Hi all:

I did not notice the fact that my server application does not generate log 
file. Someone else changed configuration files and tomcat is not generating 
this log file (catalina.out). How can I restablish tomcat for generating this 
file?

Regards.


catalina.out is the output catched from stdout. In catalina.sh there 
should be an entry with org.apache.catalina.startup.Bootstrap piping the 
output to catalina.out. Maybe someone started your appserver with 
different parameters. In some cases (run/debug) Tomcat does not create 
catalina.out by itself


Good luck,

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat slow during startup = 7.0.28

[Stefan Mayr]

Am 14.01.2013 15:11, schrieb Conrad Kostecki:

Hi!
I've noticed, that Tomcat got much slower during startup. When using a Version 
 7.0.27 (+ Oracle JDK 7), it takes much longer.
...
After this Step, with a newer version, it can take several minutes, until it 
continues.
During this pause, I can see, that the disk i/o is pretty high.
When I switch back to an older version  7.0.27, this goes much faster. It just 
takes seconds.

I am using a stock default config, only added URIEncoding=UTF-8. Tested 
with Windows 7 Professional and Ubuntu 12.04.

I don't know where to start for debugging this problem. Any Ideas?


Is this a large webapp with many classes? I think around 7.0.29 
something changed with annotation scanning. There was much discussion 
about memory issues through some changed behaviour as well.
Maybe you could try to check with metadata-complete from 
https://tomcat.apache.org/migration-7.html#Annotation_scanning



Bye,

  Stefan







-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 7.x and Internet Explorer Adobe Reader plugin

[Stefan Mayr]

Am 22.08.2012 22:31, schrieb Miguel González Castaños:

We are having what sounds like a similar problem (although 7.0.26
works for us) and can provide our details.

We are using Solaris 10, Tomcat 7.0.26, Apache/2.2.16, mod_jk/1.2.35
and Java(TM) SE Runtime Environment (build 1.6.0_30-b12) in our
production environment. We are using the same except for the latest
Tomcat 7.0.29 in our test environment.

Recently one of our testers found that when using IE8 against our test
environment she could not download PDFs (available as links on web
pages) and would instead see the error A network error occurred while
accessing this document on the Internet but it worked fine on the
older Tomcat in production.  Indeed if I roll our testing environment
back to Tomcat 7.0.26, this problem does not occur.  It also doesn't
occur in any other browser we tried (IE9, FireFox 14.0.1 and Safari
5.1.7).

I had remembered that something similar had been discussed on this
list, but the archived thread looks like it went stale when it
couldn't be reproduced:
PDF Download problem tomcat = 7.0.27
(http://mail-archives.apache.org/mod_mbox/tomcat-users/201207.mbox/%3CCAC=HunuF5zqKf5s_D9syTZ1J2nFXdCjEWKj=zq+hc8bgnn1...@mail.gmail.com%3E)


A good portion of our visitors use IE8 with the 9.5.1 Adobe Reader and
we have a number of PDF documents available for download so I'm hoping
perhaps our different configuration to the ones previously reported
might eventually lead to a fix.


As I have answered to Chris, it came down that the downgrade to 7.0.26
fixed it. Only a person reports problems that with a particular PDF file
so we have assumed it's an issue with his setup. I couldn't reproduce
the issue as I did with 7.0.29, so it seems there is something wrong
with that version.


Is your setup using tcnative? Version 7.0.27 introduced an update

Changelog: Update the native component of the Tomcat APR/native 
connector to 1.1.23 and take advantage of the simplified distribution. 
(mturk)


Maybe you could change your HTTP connector from protocol=http (has 
some autodetection for apr) to 
protocol=org.apache.coyote.http11.Http11Protocol and retest with 7.0.29.


Bye,

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 7.x and Internet Explorer Adobe Reader plugin

[Stefan Mayr]

Am 22.08.2012 23:04, schrieb Kari Scott:


On Aug 22, 2012, at 3:55 PM, Stefan Mayr wrote:

Am 22.08.2012 22:31, schrieb Miguel González Castaños:
We are having what sounds like a similar problem (although 7.0.26
works for us) and can provide our details.

We are using Solaris 10, Tomcat 7.0.26, Apache/2.2.16, mod_jk/1.2.35
and Java(TM) SE Runtime Environment (build 1.6.0_30-b12) in our
production environment. We are using the same except for the latest
Tomcat 7.0.29 in our test environment.

Recently one of our testers found that when using IE8 against our test
environment she could not download PDFs (available as links on web
pages) and would instead see the error A network error occurred while
accessing this document on the Internet but it worked fine on the
older Tomcat in production.  Indeed if I roll our testing environment
back to Tomcat 7.0.26, this problem does not occur.  It also doesn't
occur in any other browser we tried (IE9, FireFox 14.0.1 and Safari
5.1.7).

I had remembered that something similar had been discussed on this
list, but the archived thread looks like it went stale when it
couldn't be reproduced:
PDF Download problem tomcat = 7.0.27
(http://mail-archives.apache.org/mod_mbox/tomcat-users/201207.mbox/%3CCAC=HunuF5zqKf5s_D9syTZ1J2nFXdCjEWKj=zq+hc8bgnn1...@mail.gmail.com%3E)


A good portion of our visitors use IE8 with the 9.5.1 Adobe Reader and
we have a number of PDF documents available for download so I'm hoping
perhaps our different configuration to the ones previously reported
might eventually lead to a fix.

As I have answered to Chris, it came down that the downgrade to 7.0.26
fixed it. Only a person reports problems that with a particular PDF file
so we have assumed it's an issue with his setup. I couldn't reproduce
the issue as I did with 7.0.29, so it seems there is something wrong
with that version.

Is your setup using tcnative? Version 7.0.27 introduced an update

Changelog: Update the native component of the Tomcat APR/native connector to 
1.1.23 and take advantage of the simplified distribution. (mturk)

Maybe you could change your HTTP connector from protocol=http (has some autodetection 
for apr) to protocol=org.apache.coyote.http11.Http11Protocol and retest with 7.0.29.

Bye,

Stefan


No, we are using the AJP connector.

 Connector port=8009 protocol=AJP/1.3 connectionTimeout=1 
maxParameterCount=1 maxThre
ads=400 redirectPort=8443 /


Same same but different: this one is also using autodetection.

So you should test protocol=org.apache.coyote.ajp.AjpProtocol

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: PDF Download problem tomcat = 7.0.27

[Stefan Mayr]

Am 31.07.2012 23:28, schrieb André Warnier:

To be more explicit : my bet at this stage would be a bug in the
XP+IE+Acrobat9 combination (as being the usual suspects), but a bug
that gets triggered only because Tomcat 7.0.27+ send the response just a
bit differently than 7.0.26.



How about APR? The changelog includes following information in 7.0.27:

Update the native component of the Tomcat APR/native connector to 
1.1.23 and take advantage of the simplified distribution. (mturk) 


The initial thread starter reported to use Connector 
protocol=HTTP/1.1 ... 


If the autodetection finds the neccessary tcnative-lib we might search 
for a change in the APR. At the end of last year there was CVE-2011-3192 
for a vulnerability in httpd and HTTP range requests.


Maybe one should test protocol=org.apache.coyote.http11.Http11Protocol 
vs. protocol=org.apache.coyote.http11.Http11AprProtocol to ensure 
which implementation causes this problem.


- Stefan



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Issue with sudo + init script

[Stefan Mayr]

Am 01.08.2012 19:10, schrieb Shaw, Ray V CTR (US):

I'm starting up Tomcat 7.0 on RHEL6 with the following init script:

#!/bin/bash
#
# chkconfig: 235 80 20
# description: Takes care of starting and stopping Tomcat.

CATALINA_HOME=/opt/tomcat7
export JAVA_HOME=/usr/java/jdk6-64/

case $1 in
 'start')
 /bin/su tomcat -c $CATALINA_HOME/bin/startup.sh -security
 ;;
 'stop')
 /bin/su tomcat -c $CATALINA_HOME/bin/shutdown.sh
 ;;
 *)
 echo usage: $0 {start|stop}
 ;;
esac

If I use sudo -s (or -i) to get a root shell, I can use this script to start 
and stop Tomcat.

If I invoke this script directly via sudo, i.e. sudo /sbin/service tomcat7 
stop, stopping Tomcat works.  Starting does not; Tomcat gives the usual output to 
the terminal, creates or touches catalina.out (but writes nothing to it), and then 
instantly exits.  This is a problem, because I need to allow the Web admin to start/stop 
Tomcat without giving them everything.

Has anyone seen anything similar to this before?  /var/log/secure isn't showing 
anything out of the ordinary (and obviously, the commands are being run).



Could you post your tomcat output when you try to start it? You should 
also check differences in your environment variables when using 
different methods.


- Stefan



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Howot debug network connections from JVM?

[Stefan Mayr]

Good morning,

I could need some ideas how to debug a very specific network problem. It 
is not directly Tomcat related (more JVM) but maybe you have some pointers.


The situation: We have an application running in Tomcat on two different 
servers (TomcatA, TomcatB). Both are doing HTTP calls to an application 
in a different network (RemoteApp).
In the middle of the night the RemoteApp does not respond to our HTTP 
calls an we get (expected) timeouts for some minutes. This is okay so 
far. After RemoteApp has some CPU cycles left to answer our requests 
again, it becomes strange:
one of the two Tomcats can communicate with RemoteApp. The other can not 
(still getting timeouts). This does not happen every night and it is not 
fixed to TomcatA or TomcatB. Last night I used tcpdump and I don't know 
how to interpret it:


TomcatA connects to RemoteApp (SYN, SYN/ACK, ACK) and immediately closes 
the connection (FIN/ACK) in no time (1ms). This does not look like a 
timeout as reported in the logfiles and there is no HTTP request on the 
wire.


Any ideas howto narrow this down? How can I find out who closes the 
connection? Candidates are the webapp, the Tomcat/JVM-Process, OS-Limits 
or the OS (Linux) itself?

As this only happens in production debugging is not easy.

Any ideas welcome,

   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: isHexDigit error problems and upgrading Tomcat and jdk

[Stefan Mayr]

Am 07.06.2012 10:13, schrieb Miguel González Castaños:

...

  Do you suggest me to upgrade to Tomcat 6 or 7? What about jdk? 1.6
or 1.7?

1.6 is more widely tested (many years), but for a new system I would
go with 1.7.

It's not a new system, it's been running for 3 years already. I don't
want to risk that any library has changed its behavior with tomcat 7 or
something similar.


I heard that Oracle plans to stop provide free updates for 1.6 this
summer. So they will be available only as Java for Business.

Does it mean that it is probably better to go for 1.7 instead?


Maybe you should have a look at oracles lifecycle policy: 
http://www.oracle.com/technetwork/java/eol-135779.html


If you need to provide a secure system with current patches you should 
take the effort to upgrade to java 7. This provides you with 3 more 
years of Oracle updates.


   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: ROOT.xml problem

[Stefan Mayr]

Am 03.06.2012 09:17, schrieb Kevin Marx:

OK, so now, how is this working?  How to fix it?

ROOT.xml is located in conf/Catalina/localhost

File contents are thus:

?xml version='1.0' encoding='utf-8'?
Context
docBase=C:\Corda\CenterView4\Server\corda\
reloadable=true
/

in the browser I am entering http://localhost:8080

the URL is comes up with is thus:

http://localhost:8080/dashboards/welcome/main.dashxml

However it should be:

http://localhost:8080/corda/dashboards/welcome/main.dashxml

This URL will work when entered.  The /corda is necessary

Why is the /corda missing from the URL?  What needs to be in the ROOT.xml file 
to make it show up?


So you have a third-party app which needs to be called in the /corda 
context. There seems no way to make it accessible as ROOT applications.


I think you just need a redirect pointing to your applications location.

Steps:
1. remove everything you did to make it work as ROOT; back to the 
working /corda version

2. Create a redirect in your ROOT context

Options to make no. 2 work:
create webapps/ROOT/index.jsp with content
%
response.sendRedirect(http://yourhostname:8080/corda/;);
%

or a simple html-only version index.html:
html
head
meta http-equiv=refresh
  content=0; URL=/corda/
head
body
a href=/corda/Go to corda/a
/body
/html

hope this fulfills your needs

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: IE 9 ignoring my custom error pages with 404 codes

[Stefan Mayr]

Hello,

Am 13.05.2012 00:24, schrieb Konstantin Kolinko:

2012/5/13 Kiran Badiki...@poonam.org:

Hi,

I am trying to setup custom error page and has done below modification to
web.xml

error-page
error-code500/error-code
location/errorback.jsp/location
/error-page
error-page
error-code404/error-code
location/errorback.jsp/location

Then in errorback.jsp

I wrote the default hello jsp page and triggered 404 condition to check if
setting are working fine,the above setting seems to be working fine in
chrome/firefox but for some reasons IE is still showing up its own 404
message and aborting the further calls to my 404 page which I styled it some
images and other things.

Now after further investigation, I came across a setting in IE9 where in
there is checkbox to show friendly http error messages, if I turn this off,
I get my error page else I dont get it.

Now I want to show custom error page to my users. Is their any workaround
for this ? Ideally I would have expected IE to deliver what tomcat serves
and in this case custom error page but its not happening when checkbox is
turned on.

Is their any work around for this or I have to accept the whatever IE is
doing ?


Ask Microsoft? It is their beast.

I remember seeing someone adding a few KBs of HTML comments into the
error page, because that suppresses this behaviour of IE.


The usual limit is 512kb for 404 pages (Attention: some people report 
this limit also accounts for compressed content).


Eric Law documented some details in his blog post: 
http://blogs.msdn.com/b/ieinternals/archive/2010/08/19/http-error-pages-in-internet-explorer.aspx


He also has details where to find the limits for different error codes 
in the registry.


Bye,

  Stefan



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Bug in Tomcat AJP Connector?

[Stefan Mayr]

Am 08.04.2012 18:41, schrieb Ofer Israeli:

2012/4/6 Pidp...@pidster.com:

On 05/04/2012 22:17, Ofer Israeli wrote:

Y

On 5 באפר 2012, at 18:58, Konstantin Kolinkoknst.koli...@gmail.com

wrote:



2012/4/5 Ofer Israeliof...@checkpoint.com:

Mark Thomas wrote:

On 04/04/2012 17:02, Ofer Israeli wrote:

Once you have an OOME all bets are off. The JVM needs to be

restarted.

There is no guarantee of reliable operation after an OOME.

Mark


Hi Mark,
I agree that there in such a situation the JVM should be restarted, but it

isn't restarted by Tomcat.  On the other hand, Tomcat does take some
precautious actions and kills the accepting thread, but in such a case it should
also close the socket that thread is listening on otherwise it is leaving 
garbage
around after the thread's death.

Do you see any reason as not to close the listening socket?



1. Tomcat does not start JVM  thus it cannot restart it.

You need some external tool or script or admin to perform monitoring
and (re)starts.

2. OOM can happen at a random place. Once it happens, it is likely
that other places will also start to fail randomly. It is also likely
that your attempts to recover will fail as well.

Mark already mentioned it: all bets are off.

Best regards,
Konstantin Kolinko


Hi Konstantin,

I agree regarding the OOM bringing TC to a state where it must be

restored, but my point remains: if there is code that handles catching this
exception and terminating the thread, why not terminate gracefully by
closing the listening socket before killing the thread?

And your point has been answered.  After an OOM the JVM is in an
unknown, unsafe state so a restart MUST occur to restore service.

Closing a socket gracefully after an OOM is a bit like trying to shut one of the
portholes on the Titanic, shortly after hearing a large crashing sound.


There's only one place I know of where Tomcat attempts to interact with
OOM conditions and this is not one of them, so I don't believe it's safe to say
that Tomcat is deliberately handling this exception.

NB an OOM is an Error, not an Exception - it is a subclass of
VirtualMachineError, which is thrown to indicate that the Java Virtual
Machine is broken or has run out of resources necessary for it to continue
operating.

An Error is a subclass of Throwable that indicates serious problems that a
reasonable application should not try to catch.
/end-quote

If anything, the locations where Tomcat catches a Throwable should be
modified so it does *not* catch Errors, rather than continuing to do so and
then attempting a trivial tidy-up.


p


Thanks for your input - you're right regarding the error and the fact that 
Tomcat is indeed catching a Throwable and not an Exception.  I assume that if 
the Throwable were not caught, then the thread would die in any case.  Although 
stated before that Tomcat could not kill itself in such a situation, I still 
wonder if it would be possible to do so.  Or taking a different perspective on 
this: if the JVM specification is such that it cannot be trusted to continue 
running after an OOM, then why does it not kill itself or restart itself?



I guess you can do this with some vendor specific JVM arguments as 
SUNs/Oracles -XX:OnOutOfMemoryError: 
http://www.oracle.com/technetwork/java/javase/tech/vmoptions-jsp-140102.html


Different findings like kill -9 %p let me suspect that you can use %p 
as a variable for your current pid. With that you can either kill your 
current instance and let your monitoring handle the rest or try to 
initiate the restart by yourself.


Give it a try

Stefan





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Java 7 + Tomcat 6.0.35 + Win2k3 Problem

[Stefan Mayr]

Am 11.03.2012 11:33, schrieb Pid:

On 11/03/2012 06:25, pricyber wrote:

Hi I recently upgrade JDK from 1.6 to 1.7u3. And tomcat no longer work on
Windows 2003, ie not sending response to any request either with https or
plain http. But the same code/setting works on Windows 7 and my ubuntu
machine.


Error logs?
Access logs?



Are there known issues with Java 7 and Win2k3 with Tomcat 6.0.35?


None I've heard of.


p


Thanks alot.



You should supply some mode setup details:
- is tc-native used? Binaries or compiled by yourself?
- What type of connector is used? Java BIO?
- Can you enable the access log valve to see if the request was 
processed by your tomcat (and check the status code)


  Stefan


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Odd NIO connector behavior

[Stefan Mayr]

Am 28.12.2011 10:04, schrieb ma...@apache.org:

Matthew Tysonmatthewcarlty...@gmail.com  wrote:


That's right, there is an f5 load balancer.  The valve is used to keep
track of whether the request was via HTTPS or not.


What happens if you go direct to Tomcat and bypass the F5?


tcpdump seems to confirm the same.  What are you thinking?


Probably, like me, that the F5 isn't handling the Comet requests correctly.


This is what I would guess. We have a loadbalancing device that handles 
n client-lb connections with m lb-server connections in its HTTP mode. 
There we have to switch to TCP proxy mode to keep 1:1 relations.


Your F5 is where to do start crosschecking with tcpdump: client - F5 
vs F5 - server


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Odd NIO connector behavior

[Stefan Mayr]

Am 24.12.2011 19:33, schrieb Matthew Tyson:

On Sat, Dec 24, 2011 at 1:06 AM, Mark Thomasma...@apache.org  wrote:


On 23/12/2011 23:39, Matthew Tyson wrote:

Hello,

We have been having quite a few problems with using long-polling
connections in Tomcat, via the NIO connector.  Upgrading to Tomcat 7.0.23
definitely improved things, but we are still seeing major issues.


Glad to hear things are getting better. No so glad to hear you are still
having problems.


The problems only crop up after a couple minutes under some load (modest
load, around 2-3 connections per second).


That's pretty low load.



It is.  We have just a small portion of connections routed to this server.





One very clear problem I am looking at right now is that the service

method

on a servlet is continually being called, although there is no traffic
coming into tomcat from that remote IP (we verified this at the ethernet
device).


Hmm. Very strange that the service method is being called. There needs
to be a complete and valid set of HTTP headers for that to happen and
the request/response gets recycled afterwards so the data shouldn't get
processed twice.



It is very strange.



The logging statement at the beginning of the service method is being
executed every so often, like so:



logger.info(REQUEST:  + request.getRemoteAddr() ++
request.getMethod() ++ request.getQueryString());

INFO  2011-12-23 15:30:50,860 org.cometd.server.CometdServlet REQUEST:
75.149.42.46  POST  null
INFO  2011-12-23 15:31:02,484 org.cometd.server.CometdServlet REQUEST:
75.149.42.46  GET


  
message=%5B%7B%22channel%22%3A%22%2Fmeta%2Fconnect%22%2C%22connectionType%22%3A%22callback-polling%22%2C%22advice%22%3A%7B%22timeout%22%3A0%7D%2C%22id%22%3A%22354%22%2C%22clientId%22%3A%222b611tiekwk6p2mfh5bye3bm6y7l%22%7D%5Djsonp=dojo.io.script.jsonp_dojoIoScript135._jsonpCallback

INFO  2011-12-23 15:31:28,512 org.cometd.server.CometdServlet REQUEST:
75.149.42.46  POST  null
INFO  2011-12-23 15:31:36,571 org.cometd.server.CometdServlet REQUEST:
75.149.42.46  POST  null


Odd. So there are at least two different requests being processed here.


But again, there is no traffic from that IP.  I'm not sure if this is

some

sort of loop, a very long delay, or other connections being mixed up.


I'm not aware of any connection mix up issues that might explain this.


  Probably the last, since I don't see any loop pattern, and it has
continued without any traffic for almost a half an hour now.

Thoughts?


We need more information :)

If you can create a simple web application that reproduces this I'd be
happy to take a look. I suspect that is non-trivial so I'll suggest a
couple of other options.

1. The simple thing is to add a stack trace to that log message so we
can see exactly what code path is triggered this.



Here is a couple stack traces from this when the problem is occurring:

INFO  2011-12-24 10:25:35,578 COMET  REQUEST: 75.149.42.46  POST  null |
TRACE:
java.lang.Throwable
 at org.cometd.server.CometdServlet.service(CometdServlet.java:149)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:722)
 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 at
com.company.util.filter.MonitoringFilter.doFilter(MonitoringFilter.java:47)


I guess com.company is something your company developed. Monitoring 
sounds optional. Can you reproduce your error without it?



 at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
 at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
 at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:224)
 at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:169)
 at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
 at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:168)
 at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:98)
 at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:928)
 at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
 at
com.company.util.tomcat.SecureProxyValve.invoke(SecureProxyValve.java:57)


Again com.company: What is this valve doing? Can you reproduce the 
problem without it?



 at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
 at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:987)
 at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:539)
 at

Re: Maximum memory that can be assigned to Tomcat on windows platform

[Stefan Mayr]

Am 30.11.2011 12:08, schrieb j...@gniffelnieuws.net:

On Wed, 30 Nov 2011 16:14:45 +0530, Choudhury wrote

Hello ,
 The question is not why I would use 32 bit JVM , the
question is whether there is any maximum limit on memory for Tomcat
and if yes why ? Regards,


The limit is the JVM, not Tomcat itself


It depends on the windows version used actually. From what I remember this is
limited to maximum 2GB for user processes. The easiest way to test is, is with
a small app that shows some memory information like:
public class MaxMemory {
 public static void main(String[] args) {
 Runtime rt = Runtime.getRuntime();
 long totalMem = rt.totalMemory();
 long maxMem = rt.maxMemory();
 long freeMem = rt.freeMemory();
 double megs = 1048576.0;

 System.out.println (Total Memory:  + totalMem +  ( +
(totalMem/megs) +  MiB));
 System.out.println (Max Memory:+ maxMem +  ( + (maxMem/megs) +
 MiB));
 System.out.println (Free Memory:   + freeMem +  ( + (freeMem/megs)
+  MiB));
 }
}

Try to run it with different options for the heap. I would be surprised if you
would get over 1600MB.


This is what we experienced: up to 1GB Heap (Xmx) was stable on all servers.
Heaps larger were very prone to crashes. We've seen a maximum of 1.2GB 
Heap on Windows with 32bit JVMs


That's why we have switched to 64bit JVMs on all machines. No need to 
swap JVM editions when applications require more memory - just increase 
Xmx as long as RAM is available.


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: WebApp access to a LAN share

[Stefan Mayr]

Am 16.10.2011 10:31, schrieb André Warnier:

Léa Massiot wrote:

Hello,

Thank you for reading my post.

Here is my problem:
- I have two machines S and M on the same LAN.
- S is a Debian machine running a Tomcat server.
- And I have a WebApp W deployed on this Tomcat server.
- M is a Windows machine which hosts some files for W. - S and M
belong to the same Samba domain.
- On M, the WebApp files are stored in a directory: C:\p1\p2\.
- p2 is a share.
- Somewhere in the WebApp Java code, I have declared a final to
store the
files path.
Here is what I wrote:
public static final String s_path = //M/p2/;
- Now, when I manipulate the WebApp in such a way a file f.xml
located in
C:\p1\p2\p3\ has to be opened and read, I get the following error:
java.io.FileNotFoundException: /M/p2/p3/f.xml (No such file or directory)
My question is: how do I have to declare s_path to properly access,
from S
(W), those files which are stored on M?
(I basically want to have the data on one machine and Tomcat on another
machine).


Without discussing the pros and cons of the underlying logic, the issue
here is that Java (on S) does not understand that //M/p2/ means a
network share in Microsoft notation.
(Only Windows understands that).
What you need to to is mounting that network share to a local (Linux)
mountpoint on S.
Then in Java you access that mountpoint, not the original share.

This is quite Off-Topic regarding Tomcat and even Java, but here is an
idea :

On the Linux machine,

1) create an empty directory /mnt/M/p2

2) in the file /etc/fstab, add the following line (all on one line):
//M/p2 /mnt/M/p2 cifs
rw,username=xxx,password=yyy,uid=uuu,gid=ggg,dir_mode=0775,file_mode=0774 0
2

(xxx and yyy are the (Windows) user-id and password needed to access
that share; uuu and ggg are a Linux user-id and group under which the
remote files will look like under Linux). And you will need to get
someone to help you with the other parameters, to adjust them to your
needs.

3) under Linux, isssue the command mount -a. This will read the file
/etc/fstab, and mount all the mountpoint that ar not already mounted
(like the new one above).

After this, under Linux, whenever you access the directory /mnt/M/p2,
you will see the files on the remote M nachine.
And in Java, you then access this the same way, as /mnt/M/p2.



So machine M is essentially a fileserver that is accessed by 
application W.


I see two options:
1. Abstract it on OS level: Linux machine S mounts the fileshare into 
the local directory /M/p2 via CIFS/SMBFS. Details were just outlined by 
André
2. Solve it in your application using jCIFS (or another component) to 
access a remote Windows fileshare


Independent recommendation: make s_path a configuration variable. 
Hardcoded paths are no good idea


- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: TC on Windows Server Core

[Stefan Mayr]

Am 24.08.2011 23:01, schrieb André Warnier:

David kerber wrote:

Will TC run on a Windows Server Core installation? For those of you
not familiar with that term, it's a windows server installation with
no GUI, and minimal other pieces. The idea is a reduced disk, memory
and cpu footprint (rather Unix/Linux like).


One might be forgiven if one were to wonder why one would then not use a
perfectly serviceable minimal Linux base then, no ?
Once you take away the GUI, what is left that is so compelling about
Windows Server ?


I would say the very same reasons apply
- reduced ressource utilization
- less vulnerable
- best tool for the job (if windows is your everyday OS)

Not your favourite OS? s/windows/os_your_favorite_os/g. If you can 
afford it there is an unlimited freedom of choice - but I guess this is 
not the right list for os flame wars.


Back to your question. I recommend to read 
http://blogs.technet.com/b/jorke/archive/2008/09/17/cat-power-tomcat-on-server-2008-core-with-iis7.aspx 
. It is a bit dated but lists an ugly pitfall: MSVCR71.dll is required 
but not included. (don't know if this still applies)


Also many 3rd party libraries use awt for some internal purpose and 
expect a display (even on linux). So use java.awt.headless=true.


Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: TC on Windows Server Core

[Stefan Mayr]

Am 25.08.2011 22:55, schrieb Stefan Mayr:

...
Back to your question. I recommend to read
http://blogs.technet.com/b/jorke/archive/2008/09/17/cat-power-tomcat-on-server-2008-core-with-iis7.aspx
. It is a bit dated but lists an ugly pitfall: MSVCR71.dll is required
but not included. (don't know if this still applies)


I've found the explaination for this behaviour in Oracles bug database: 
http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=6509291


It is marked as won't fix. So you still need to copy the dll

Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 5.5.28 on 64 bit windows server 2008.

[Stefan Mayr]

Hello,

Am 21.08.2011 15:01, schrieb Venkata Surapaneni:

Verlag,

 Both the suggestions worked and We are going to add address line to 
the server.xml file.

Is the support for IP 4 dropped by plan or by accident ? If it is by plan, 
isn't it early ? There are still lot of applications running on IP 4 and it 
will be while before every one is ready for IP 6.

If it is by accident, I believe the versions from 5.5.29 have this issue.


I guess this is not a Tomcat issue. In your server.xml you habe the 
org.apache.catalina.core.AprLifecycleListener enabled which uses the dll 
named by Eurotrans-Verlag. It should be enough to remove this line from 
your config to disable native APR. Another solution might be to keep 
your old Tomcat version and update only the binaries für your operating 
system. Maybe your dlls are not meant for 64bit Windows. See 
http://tomcat.apache.org/download-native.cgi.


For further JVM IPv6 documentation check first 
http://download.oracle.com/javase/6/docs/technotes/guides/net/ipv6_guide/index.html
Especially java.net.preferIPv4Stack is interesting to enforce a certain 
protocol behaviour.


Stefan






-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: response headers missing ?

[Stefan Mayr]

Am 14.07.2011 13:25, schrieb André Warnier:

Mark Thomas wrote:

On 14/07/2011 11:29, André Warnier wrote:

Hi.

This is a bit of a side question, or let's say a question-by-proxy.

...

I think for this problem, I have to treat tomcat as a little, rather
inefficient, black box and try to fixup on the apache front ends, hence
the direction of my original idea...


If you have no chance to get these applications fixed - keep it this 
way. Treat the tomcat as black box and add cache-control headers on the 
apache-frontend where possible.


Use Location(Match) and mod_headers to append whatever is needed to the 
server response to make it browser-cachable



Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Tomcat 6.0.32 on 64-bit 2008 server - Event 1000, Application Error

[Stefan Mayr]

Am 04.05.2011 18:34, schrieb André Warnier:

fsman...@netscape.net wrote:


Also, do the out-of-order timestamps (and the server startup time of
-1161496934 ms) hint at anything?


Not that it has anything to do with the problem, but I would say that
the startup time looks very much like a formatting error. Or else that
hardware of yours is *really* fast.
May even explain the out-of-order timestamps, when you think about it.
;-)


out-of-order timestamps on a virtual machine are not uncommon. VMware 
ESX for example does not provide a stable timesource. Most know problems 
are clocks that are running too fast. Depending on your time 
synchronization ist is now possible that system time is set backwards. 
Many programms calculating time differences don't like that. Startup 
time faster then light speed anyone?
Also it is possible that a certain system time is skipped: when the 
system clock is too slow the VMware Tools can skip a period of time and 
set the current time.
We once had MySQL not closing idle-connections because the time this 
should happen was skipped.


You could try monitoring the system time in relation to a stable time 
source and check if there are any coherences.



- Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Simulating HTTPS in terminated SSL/Apache 2/Tomcat 6 cluster

[Stefan Mayr]

Hi

Am 29.03.2011 12:28, schrieb Rainer Jung:

On 29.03.2011 12:07, Richard Levy wrote:

...
The current setup involves two intelligent load-balancers that
terminates SSL then hands over to Oracle 10g. The 10g stack has
custom Oracle versions of Apache which have configuration options not
found in standard Apache and appear to have made configuration
simpler. For instance, there is a simulateHttps directive that you
can give to a virtual host in Oracle Apache which does not exist in
the standard version. Using that option, we simply have 2 virtual
hosts defined in Apache, one for http traffic and one for https
traffic (with simulateHttps set to on). They both forward to OC4J and
it works fine.


You can achieve this with standard Apache and mod_jk esily to.


That's what we have in production:
1. Our intelligent Loadbalancer controls an additional 
HTTP-Request-Header Front-End-Https (default off, on when using SSL).
2. Our Apache is one VirtualHost using the following block for SSL 
detection:

  IfModule env_module
# Fake SSL if Loadbalancer does SSL-Offload
SetEnvIf Front-End-Https ^on$ HTTPS=on
  /IfModule

Advantage:
1. as we use the same backend for ssl and non-ssl-traffic the session 
affinity still works switching between HTTP and HTTPS
2. Reduced Apache configuration as we don't need to double VirtualHosts 
for SSL
3. Transparent to the Java Applications. The Application won't notice 
we've cheated :-)



With the new architecture we have two load-balancer which round-robin
to two Apache servers. These servers connect to two Tomcat servers
which are configured with a mod_jk loadbalancer using session
affinity. Through config we have separated HTTP HTTPS traffic. We
have done this because the application needs to identify when it is
secure, which we do by simulating HTTP using settings in the connector
in the Tomcat server.xml. This is detailed further down.

Obviously we want to keep the same functionality on the Apache/Tomcat
stack, and also retain the termination of SSL at the load-balancer,
but are having trouble when the application switches to HTTPS. It
works fine if we have a single Tomcat instance running, but once the
2nd Tomcat instance is enabled, moving to HTTPS fails because a new
session is created.


I went only brievly thourgh your configuration list and communication
behavior, but thanks for the details!

I suggest you try the following:

- use only one connector per Tomcat instance, so
only one jvmRoute and only one worker per Tomcat

- let Apache via mod_jk inform the connector,
whether the request came via HTTP or (fake) HTTPS

- still do the redirect, but now because of only one connector you
have only one jvmRoute and only one worker per Tomcat.
So stickyness should work again.

How can Apache forward the HTTP vs. (fake) HTTPS info via mod_jk?

mod_jk automatically forwards the info whether the request was HTTPS.
When the application checks it, the Tomcat AJP connector retrieves the
info from what mod_jk forwarded it was and presents it to the
application. AJP itself is never encrypted.

Where does mod_jk get the info from? By default it checks an Apache
environment variable named HTTPS, which is set by Apache whenever it
handles an HTTPS request.

You can also set the HTTPS environment variable by your Apache config,
but it is better to understand not to overload the meaning of the variable.

Instead:

- tell mod_jk to look for another, private variable which contains the
info whether the rquest was HTTP or (fake) HTTPS:

JkHTTPSIndicator MyHTTPSVariable

(you can choose the name MyHTTPSVariable)

- set the variable MyHTTPSVariable to On in the Apache
Virtual Host that receives your fake HTTPS requests

SetEnv JkHTTPSIndicator On

(for this mod_env must be enabled)

That's it :)


Guess that's cleaner then what we did. I'll put swapping HTTPS with 
JkHTTPSIndicator on my todo-list.


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Simulating HTTPS in terminated SSL/Apache 2/Tomcat 6 cluster

[Stefan Mayr]

Am 29.03.2011 16:19, schrieb Jeffrey Janner:

OK, I'm sure some of you guys can weigh in better on this than me, but:

If he already has two intelligent load-balancers terminating the SSL,
couldn't he simplify the configuration a good bit by removing the
Apache servers all together?  That is, couldn't he just run Tomcat as a
stand-alone cluster with HTTP/HTTPS connectors instead of AJP?

--
Jeff


To answer my own question:  Unless he needs the Apache httpd setup to insure session 
stickyness because his intelligent load-balancers cannot provide it.


There are other reasons too:
- adding addition access restricions not handled in tomcat
- adding custom errorpages
- offloading static content from tomcat
- doing some rewriting
- ...

Apache httpd may not be very lightweight but it is very feature rich. 
Often it is the only point you can additionaly influence closed 
application running in tomcat


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Windows Authentication: Issue 49318 vs 47679

[Stefan Mayr]

Hi Mark,

Am 28.03.2011 10:49, schrieb Mark Thomas:

On 28/03/2011 08:42, Borut Hadžialić wrote:

Hellos Stefan,

if you can't fix your problem with configuration and decide that you
want to solve the problem by programming, then this might help you
http://blog.springsource.com/2009/09/28/spring-security-kerberos/
After understanding that article a developer should be able to add a
SPNEGO implementation (probably not the whole protocol, just as much
it is needed for your app) to your Tomcat application by adding some
filters.


Or you could just add Spring Security to your app. I'll add that as an
option to the new How-To.


I guess this is the classic kerberos/keytab approach (no NTLM-fallback) 
that many solutions offer.



Today I've found Bug 49318 - add a Negotiate (Kerberos/NTLM) authenticator /
integrate Waffle (https://issues.apache.org/bugzilla/show_bug.cgi?id=49318).
The last comment links a new Windows Authentication How-To from Mark Thomas.
Looks like we have already tried almost all proposed solutions:


Thanks for the great feedback on the options. I put the existing how-to
together pretty much entirely on some Google searches. I'll add your
feedback to the how-to / maybe remove some options that don't look viable.


- IIS + mod_jk:
  tried but stuck in Bug 47679. Also tried ARR to pass the user name
  as a request header from IIS to Tomcat without success
- Apache mod_ntlm: used it and we replaced it by the much more stable
  mod_auth_ntlm_winbind. NTLMv1 is also disabled on Windows 7 (default)
- Apache mod_auth_ntlm: in heavy use but stuck to Apache 2.0 and 32bit
  plattform - we couldn't get stability problems solved on Apache 2.2
  and 64bit Linux. No ongoing development.
- Apache mod_auth_sspi: till now in internal use for a very small
  project (works just fine), not sure about the future. Although
  there seems to be some new activity on 1.0.5 beta
- Waffle: found it on thursday and it is on my our todo-list for
  testing it next week

Any chances to get Bug 47679 solved? How can we help (we are admins, no
devs)?
What solutions have you deployed? Recommendations?


It is tricky to recommend something right now. I'm guessing you want
something that a) works reliably and b) is likely to be supported for
the long term. Right now Waffle probably comes closest to that. It you
can wait a little while, I should have SPNEGO support in Tomcat 7 fairly
soon. It may - or may not - get back-ported to Tomcat 6. It will depend
on the eventual solution.


You're definitely right. We search for the holy grail of intranet 
authentication. a+b is a must.


The idea of using IIS with ARR in reverse proxy mode passing a username 
was dead end: Microsoft pointed us to a nice article describing HTTP 
request processing order. Rewriting a request comes before the 
authentication modul - so nothing to append to a header or request in 
the first place.
See 
http://learn.iis.net/page.aspx/501/iis-70-request-filtering-and-url-rewriting/

Leaves IIS with mod_jk if you can live with Bug 47679.

Our first test with Waffle is promising. Now it needs to be integrated 
and in our application for further testing.


Native SPNEGO in Tomcat sounds great. Waiting a little while depends on 
your scale of little. Is there already some development we can follow? 
Will this use Java GSS? I never figured out how to configure this with 
Tomcat.


   Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Windows Authentication: Issue 49318 vs 47679

[Stefan Mayr]

Hello everybody,

as many others before we wanted to do single-sign-on for intranet web 
applications using integrated windows authentication (negotiate because 
IE sometimes tries NTLM instead of using plain kerberos - breaking all 
our kerberos-only experiments).


We thought that IIS would be the best choice for integrated windows 
authentication and we could pass the user via AJP (using mod_jk) to our 
tomcat instances.


Our setup:
- Windows 2008 R2 using IIS 7.5 (64bit)
- mod_jk 1.2.31
- Oracle Java 1.6 U24
- Tomcat 6.0.32

At first glance using tomcatAuthentication=false worked as expected. We 
got the remote user and started deploying an application. End of 
happiness - the application complained about a missing user-agent. That 
header was not passed to tomcat when authentication was enabled on IIS.


Some research revealed Bug 47679 - Not all headers get passed to Tomcat 
server from isapi_redirect.dll 
(https://issues.apache.org/bugzilla/show_bug.cgi?id=47679)


Today I've found Bug 49318 - add a Negotiate (Kerberos/NTLM) 
authenticator / integrate Waffle 
(https://issues.apache.org/bugzilla/show_bug.cgi?id=49318). The last 
comment links a new Windows Authentication How-To from Mark Thomas. 
Looks like we have already tried almost all proposed solutions:


- IIS + mod_jk:
  tried but stuck in Bug 47679. Also tried ARR to pass the user name
  as a request header from IIS to Tomcat without success
- Apache mod_ntlm: used it and we replaced it by the much more stable
  mod_auth_ntlm_winbind. NTLMv1 is also disabled on Windows 7 (default)
- Apache mod_auth_ntlm: in heavy use but stuck to Apache 2.0 and 32bit
  plattform - we couldn't get stability problems solved on Apache 2.2
  and 64bit Linux. No ongoing development.
- Apache mod_auth_sspi: till now in internal use for a very small
  project (works just fine), not sure about the future. Although
  there seems to be some new activity on 1.0.5 beta
- Waffle: found it on thursday and it is on my our todo-list for
  testing it next week

Any chances to get Bug 47679 solved? How can we help (we are admins, no 
devs)?

What solutions have you deployed? Recommendations?

Thank you,

Stefan Mayr

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org