Re: Database related performance degradation after upgrading from Tomcat 9.0.33 to Tomcat 9.0.69

[Terence M. Bandoian]

On 2/21/2023 10:58 PM, Konstantin Kolinko wrote:

ср, 22 февр. 2023 г. в 01:31, Artur Tomusiak - Hannon Hill
:

After upgrading from Tomcat 9.0.33 to Tomcat 9.0.69,

Note that using a binary search (bisection) one could limit the version range.



If I understand the point you're making:

If testing for this problem was conducted with versions between 9.0.33 
and 9.0.69, the number of versions that could have caused the problem 
would be more limited which would simplify finding the cause.


-Terence Bandoian

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Database related performance degradation after upgrading from Tomcat 9.0.33 to Tomcat 9.0.69

[Konstantin Kolinko]

ср, 22 февр. 2023 г. в 01:31, Artur Tomusiak - Hannon Hill
:
>
> After upgrading from Tomcat 9.0.33 to Tomcat 9.0.69,

Note that using a binary search (bisection) one could limit the version range.

Alternatively, it is possible to reconfigure the pool to use Apache
Commons DBCP 2 and Apache Commons Pool 2 directly (instead of
package-renamed version used by Tomcat), and bisect their version
ranges.

> [...]
>
> Here is our database connection configuration in context.xml file:
>
>name="jdbc/CascadeDS"
> auth="Container"
> type="javax.sql.DataSource"
> username="@{dbusername}"
> password="@{dbpassword}"
> driverClassName="com.mysql.jdbc.Driver"
>
> url="jdbc:mysql://@{dbhostport}/@{dbname}?useUnicode=truecharacterEncoding=UTF-8useSSL=false"
> maxTotal="250"
> maxIdle="10"

Only 10 idle connections are allowed?

So if you have 250 active connections, only 10 of them can be returned
to the pool, and the other 240 of them have to be closed and reopened.

How many active connections do you usually have? Are those numbers adequate?

> maxWaitMillis="3000"
> removeAbandonedOnBorrow="true"
> removeAbandonedOnMaintenance="true"
> removeAbandonedTimeout="300"
> logAbandoned="true"
> testOnBorrow="true"
> testOnCreate="true"

I do not see a "validationQuery", so you are relying on
connection.isValid() for those tests...

Why do you need a "testOnCreate" ?

>   />

Best regards,
Konstantin Kolinko

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Database related performance degradation after upgrading from Tomcat 9.0.33 to Tomcat 9.0.69

[John.E.Gregg]

Artur,

> -Original Message-
> From: Artur Tomusiak - Hannon Hill 
> Sent: Tuesday, February 21, 2023 4:31 PM
> To: users@tomcat.apache.org
> Subject: Database related performance degradation after upgrading from
> Tomcat 9.0.33 to Tomcat 9.0.69
> 
> After upgrading from Tomcat 9.0.33 to Tomcat 9.0.69, jobs in our application
> that execute lots of quick database queries end up being visibly slower - 16%
> slower on average in a typical setup where a database is on a local area
> network. Here is additional information we have
> confirmed:
> 
>- This is not specific to operating system (tested on Linux and MacOS)
>- This is not related to database vendors (tested on MySQL and Oracle)
>- This is not related to our software (identical code runs on different
>versions of Tomcat)
>- This is related to database connection latency as opposed to the speed
>of the database or the app - the longer the database latency, the more
>significant the slowdown is. When testing with a local database on the same
>machine, there is no performance hit between the two versions of Tomcat.
>When testing with a database on another network across the Internet (very
>high latency), the job running on Tomcat 9.0.69 is about 50% slower than
>Tomcat 9.0.33.
>- This might be coincidental, but based on the number of queries the job
>executes and the database connection latency, it appears as if each
>database query required 2 additional network trips to the database on
>Tomcat 9.0.69 as compared to Tomcat 9.0.33.
>   - For example, if a job executes about 37,000 queries, and the
>   database connection latency is about 0.15 ms, the job ends up being
> about
>   11 seconds slower on Tomcat 9.0.69 than Tomcat 9.0.33 based on our
> tests.
>   This seems to add up to 2 extra network trips per query because 37,000
>   queries * 0.15 ms/trip * 2 extra trips/query = 11,100 ms = 11.1 s.
>   - Another example is when testing with a db connection over the
>   Internet (25 ms latency) and a job that executes 1,231 queries, that 
> job is
>   more or less 60 seconds slower on Tomcat 9.0.69 than Tomcat
> 9.0.33 based on
>   our tests. Again, if we assumed that there are extra 2 trips to the
>   database per query, this adds up: 1,231 queries * 25 ms/trip * 2 extra
>   trips/query = 61,550 ms = 61.55 s.
> 
> We are suspecting that the slowness comes from around getting a database
> connection from the connection pool, though we spotted an occasional
> slowness around transaction committing as well.
> 
> Here is our database connection configuration in context.xml file:
> 
>name="jdbc/CascadeDS"
> auth="Container"
> type="javax.sql.DataSource"
> username="@{dbusername}"
> password="@{dbpassword}"
> driverClassName="com.mysql.jdbc.Driver"
> 
> url="jdbc:mysql://@{dbhostport}/@{dbname}?useUnicode=truechar
> acterEncoding=UTF-8useSSL=false"
> maxTotal="250"
> maxIdle="10"
> maxWaitMillis="3000"
> removeAbandonedOnBorrow="true"
> removeAbandonedOnMaintenance="true"
> removeAbandonedTimeout="300"
> logAbandoned="true"
> testOnBorrow="true"
> testOnCreate="true"
>   />
> 
> Is this a known issue? If not, is there any additional information I could
> provide to help troubleshoot or replicate the problem?
> 
> Thank you,
> Artur Tomusiak

I don't know anything about version differences between those versions of 
Tomcat, but I do know a thing or two about JDBC and network round trips.

Assuming the connections are open and in the pool, here are the possible round 
trips that I can think of:

1. Check the health of the connection upon checkout (testOnBorrow=true)
2. Disable autocommit
3. Prepare JDBC statement
4. Execute statement & read initial results
5. Read additional results
6. Commit/roll back transaction
7. Enable auto commit

Statement caching should help with #3.

4 and 5 are a little mushy.  I know with Oracle the first 10 rows by default 
come back in the response to the query.  If there are more results, they will 
require additional round trips unless you've increased the JDBC prefetch size.

7 might surprise you but typically pools store connections with autocommit 
enabled because that is the default state for a connection.  Some pools allow 
you to disable this functionality.  IOW, the autocommit state doesn't change 
unless you explicitly change it.

Also, you need to think about what's going on at the TCP level.  If there are 
enough packets in the response, the DB can't just send them all at once.  It 
might have to pause to wait for the ACKs from the client.  So even if the row 
count is low but the rows are wide, there might be extra pauses in there.  As 
long as the data was the same in your tests, I don't think this should be an 
issue, though.

Hope this helps.

Database related performance degradation after upgrading from Tomcat 9.0.33 to Tomcat 9.0.69

[Artur Tomusiak - Hannon Hill]

After upgrading from Tomcat 9.0.33 to Tomcat 9.0.69, jobs in our
application that execute lots of quick database queries end up being
visibly slower - 16% slower on average in a typical setup where a database
is on a local area network. Here is additional information we have
confirmed:

   - This is not specific to operating system (tested on Linux and MacOS)
   - This is not related to database vendors (tested on MySQL and Oracle)
   - This is not related to our software (identical code runs on different
   versions of Tomcat)
   - This is related to database connection latency as opposed to the speed
   of the database or the app - the longer the database latency, the more
   significant the slowdown is. When testing with a local database on the same
   machine, there is no performance hit between the two versions of Tomcat.
   When testing with a database on another network across the Internet (very
   high latency), the job running on Tomcat 9.0.69 is about 50% slower than
   Tomcat 9.0.33.
   - This might be coincidental, but based on the number of queries the job
   executes and the database connection latency, it appears as if each
   database query required 2 additional network trips to the database on
   Tomcat 9.0.69 as compared to Tomcat 9.0.33.
  - For example, if a job executes about 37,000 queries, and the
  database connection latency is about 0.15 ms, the job ends up being about
  11 seconds slower on Tomcat 9.0.69 than Tomcat 9.0.33 based on our tests.
  This seems to add up to 2 extra network trips per query because 37,000
  queries * 0.15 ms/trip * 2 extra trips/query = 11,100 ms = 11.1 s.
  - Another example is when testing with a db connection over the
  Internet (25 ms latency) and a job that executes 1,231 queries,
that job is
  more or less 60 seconds slower on Tomcat 9.0.69 than Tomcat
9.0.33 based on
  our tests. Again, if we assumed that there are extra 2 trips to the
  database per query, this adds up: 1,231 queries * 25 ms/trip * 2 extra
  trips/query = 61,550 ms = 61.55 s.

We are suspecting that the slowness comes from around getting a database
connection from the connection pool, though we spotted an occasional
slowness around transaction committing as well.

Here is our database connection configuration in context.xml file:

  

Is this a known issue? If not, is there any additional information I could
provide to help troubleshoot or replicate the problem?

Thank you,
Artur Tomusiak

Re: Tomcat 10.0.x

[Christopher Schultz]

Amn,

On 2/18/23 14:22, Amn Ojee Uw wrote:

But, Debian has no upgrades for Tomcat 11.
I am new to Linux and I am using Debian.
Should I uninstall Tomcat and then install version 11 of it?


Tomcat 11 is a pre-release version of Tomcat. The best version of Tomcat 
to use depends upon your needs. I would pick one of the following:


1. If you use javax.servlet and have a Java EE-style webapp, I would use 
Tomcat 9.


2. If you use jakarta.servet and hava a Jakarta EE webapp, I would use 
Tomcat 10.1.


I would only use Tomcat 11 for experimental purposes.

Hope that helps,
-chris


On Tue, 2023-02-07 at 19:04 +,
jonmcalexan...@wellsfargo.com.INVALID wrote:

Thank you

Dream * Excel * Explore * Inspire
Jon McAlexander
Senior Infrastructure Engineer
Asst. Vice President
He/His

Middleware Product Engineering
Enterprise CIO | EAS | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508

jonmcalexan...@wellsfargo.com
This message may contain confidential and/or privileged information.
If you are not the addressee or authorized to receive this for the
addressee, you must not use, copy, disclose, or take any action based
on this message or any information herein. If you have received this
message in error, please advise the sender immediately by reply e-
mail and delete this message. Thank you for your cooperation.


-Original Message-
From: Mark Thomas 
Sent: Tuesday, February 7, 2023 12:58 PM
To: Tomcat Users List 
Subject: Re: Tomcat 10.0.x


7 Feb 2023 18:38:31 jonmcalexan...@wellsfargo.com.INVALID:


Hi Mark, Christopher, Remy, et-al,

In regards to the Apache Tomcat(r) - End of life for Apache
Tomcat
10.0.x<
https://urldefense.com/v3/__https://tomcat.apache.org/tomcat-

10

.0-

eol.html__;!!F9svGWnIaVPGSwU!pmqqpyFZRZ4wSiu9dnkBXWrGqWkXTBM
ONR-cAp

fu4qcCKC6Lk9k3_sAk5_4IdJN7cn1-qyhgVXTjJpVm53L9kA$ > page, is this

and END OF LIFE event or just an END OF SUPPORT event for the
10.0.x
releases of Tomcat 10x?

Is there a difference?

The EOL page describes what we expect to happen.
Which reminds me, there is some clean up to do now we are past 31st
Jan.

Mark




Thanks,

Dream * Excel * Explore * Inspire
Jon McAlexander
Senior Infrastructure Engineer
Asst. Vice President
He/His

Middleware Product Engineering
Enterprise CIO | EAS | Middleware | Infrastructure Solutions

8080 Cobblestone Rd | Urbandale, IA 50322
MAC: F4469-010
Tel 515-988-2508 | Cell 515-988-2508



jonmcalexan...@wellsfargo.com

This message may contain confidential and/or privileged
information.
If you are not the addressee or authorized to receive this for
the
addressee, you must not use, copy, disclose, or take any action
based
on this message or any information herein. If you have received
this
message in error, please advise the sender immediately by reply
e-mail
and delete this message. Thank you for your cooperation.


---
--
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Query about support for OpenSSL 1.1.1

[Mark Thomas]

On 21/02/2023 10:08, Vivek Naruka (EXT-NSB) wrote:

Hi,

Currently we use APCAHE TOMCAT 9.0.69 which supports OpenSSL1.1.1 version.


It is not that simple.

Tomcat 9.0.x can be used with Tomcat Native 1.2.x which provides:
 - TLS support when using the HTTP APR/native connector
 - an alternative to JSSE to provide TLS support for the HTTP NIO and
 NIO2 connectors

Tomcat 9.0.x can be used with Tomcat Native 2.0.x which provides:
 - an alternative to JSSE to provide TLS support for the HTTP NIO and
 NIO2 connectors
  - (the APR/native connector is NOT available with Tomcat Native 2.0.x)

Tomcat Native 1.2.x can be compiled with any current OpenSSL version 
(1.1.1, 3.0.x, 3.1.x)


Tomcat Native 2.0.x requires OpenSSL 3.0.x onwards.


We would like to know the APACHE TOMCAT version that supports openSSL3.0 with 
Java 8 version?


Tomcat 8.5.x
 - requires minimum of Java 7
 - can use Tomcat Native 1.2.x (which can be compiled with OpenSSL 3.0)
 - can use Tomcat Native 2.0.x (which can be compiled with OpenSSL 3.0)

Tomcat 9.0.x
 - requires minimum of Java 8
 - can use Tomcat Native 1.2.x (which can be compiled with OpenSSL 3.0)
 - can use Tomcat Native 2.0.x (which can be compiled with OpenSSL 3.0)

Tomcat 10.1.x
 - requires minimum of Java 11 so fails to meet your requirements

Tomcat 11.0.x
 - requires minimum of Java 17 (may rise to 21) so fails to meet your
   requirements


Does TOMCAT depend on OS (like RHEL, Windows, etc) for OpenSSL support or does 
it package OpenSSL on its own?


Tomcat does not depend (directly) on OpenSSL.

Tomcat depends on Tomcat Native to provide OpenSSL functionality.

Tomcat Native and OpenSSL support are optional. Tomcat will run quite 
happily without them.


Tomcat Native can be compiled with OpenSSL provided statically or 
dynamically.


The Tomcat Native binaries for Windows are compiled with OpenSSL 
provided statically.


The Tomcat team only provide source for other platforms. It is expected 
that users compile it themselves. By default, the compilation process 
will use the OpenSSL version provided by the OS but that can be 
overridden if desired.


Mark





Regards
Vivek Singh


-Original Message-
From: Mark Thomas 
Sent: 15 February 2023 16:43
To: users@tomcat.apache.org
Subject: Re: Query about support for OpenSSL 1.1.1

On 15/02/2023 10:30, Vivek Naruka (EXT-NSB) wrote:

Hi Tomcat Support Team,

There is new version of Openssl i.e. Openssl 3.0 available for which tomcat 
provide support in its newly released versions.
We are using Openssl version 1.1.1 in our project and need to know that if 
tomcat will continue its support towards Openssl 1.1.1 as well till year 2030.


Yes and no.

For Tomcat 9.0.x and earlier, OpenSSL provides the following optional features 
via Tomcat Native 1.2.x:
- TLS support when using the HTTP APR/native connector
- an alternative to JSSE to provide TLS support for the HTTP NIO and
NIO2 connectors

For Tomcat 10.1.x and later, the APR/native connector has been removed and 
OpenSSL provides the following features via Tomcat Native 2.0.x:
- an alternative to JSSE to provide TLS support for the HTTP NIO and
NIO2 connectors

Tomcat Native 1.2.x currently supports OpenSSL 1.0.2 onwards (including 3.0.x). 
The minimum OpenSSL version could be increased to OpenSSL 1.1.1 onwards (along 
with a version bump to Tomcat Native 1.3.x) but that work is fairly low 
priority. Whether / when that update happens doesn't really change the answer 
to your question.

Tomcat Native 2.0.x currently supports OpenSSL 3.0.x onwards.

End of Life for Tomcat 8.5.x has been announced as 31 March 2024.

No End of Life date has been announced for 9.0.x but major Tomcat versions 
typically reach End of Life at ~3 year intervals so a reasonable guess for the 
End of Life date for Tomcat 9.0.x is 31 March 2027.

Once Tomcat 9.0.x reaches End of Life, there will be no requirement to continue 
supporting Tomcat Native 1.2.x so it seems likely that Tomcat Native 1.2.x will 
reach End of Life at the same point.

Tomcat 9.x is a special case for End of Life as it is the final version that 
supports Java EE. As such, once 9.0.x reaches end of life there will be 9.10.x 
but that will pick up all the changes from 10.1.x apart from the switch from 
the Java EE API to the Jakarta EE API. This means Tomcat 9.10.x will depend on 
Tomcat Native 2.0.x (and OpenSSL 3.0.x).

So, from the ASF's perspective, Tomcat Native 1.2.x (including support for 
OpenSSL 1.1.1) is expected to end some around March 2027. It might be as much 
as 18 months later but I don't see it extending as far as 2030.


All of that said, there are also downstream distributions of Apache Tomcat 
provided by various Linux distributions. If you obtain Tomcat and Tomcat Native 
via one of these distributions, it will remain supported by the distribution 
for the standard support timescales for that distribution - irrespective of 
whether or not the ASF has declared that version to have 

RE: Query about support for OpenSSL 1.1.1

[Vivek Naruka (EXT-NSB)]

Hi, 

Currently we use APCAHE TOMCAT 9.0.69 which supports OpenSSL1.1.1 version. We 
would like to know the APACHE TOMCAT version that supports openSSL3.0 with Java 
8 version? 

Does TOMCAT depend on OS (like RHEL, Windows, etc) for OpenSSL support or does 
it package OpenSSL on its own?

Regards
Vivek Singh


-Original Message-
From: Mark Thomas  
Sent: 15 February 2023 16:43
To: users@tomcat.apache.org
Subject: Re: Query about support for OpenSSL 1.1.1

On 15/02/2023 10:30, Vivek Naruka (EXT-NSB) wrote:
> Hi Tomcat Support Team,
> 
> There is new version of Openssl i.e. Openssl 3.0 available for which tomcat 
> provide support in its newly released versions.
> We are using Openssl version 1.1.1 in our project and need to know that if 
> tomcat will continue its support towards Openssl 1.1.1 as well till year 2030.

Yes and no.

For Tomcat 9.0.x and earlier, OpenSSL provides the following optional features 
via Tomcat Native 1.2.x:
- TLS support when using the HTTP APR/native connector
- an alternative to JSSE to provide TLS support for the HTTP NIO and
   NIO2 connectors

For Tomcat 10.1.x and later, the APR/native connector has been removed and 
OpenSSL provides the following features via Tomcat Native 2.0.x:
- an alternative to JSSE to provide TLS support for the HTTP NIO and
   NIO2 connectors

Tomcat Native 1.2.x currently supports OpenSSL 1.0.2 onwards (including 3.0.x). 
The minimum OpenSSL version could be increased to OpenSSL 1.1.1 onwards (along 
with a version bump to Tomcat Native 1.3.x) but that work is fairly low 
priority. Whether / when that update happens doesn't really change the answer 
to your question.

Tomcat Native 2.0.x currently supports OpenSSL 3.0.x onwards.

End of Life for Tomcat 8.5.x has been announced as 31 March 2024.

No End of Life date has been announced for 9.0.x but major Tomcat versions 
typically reach End of Life at ~3 year intervals so a reasonable guess for the 
End of Life date for Tomcat 9.0.x is 31 March 2027.

Once Tomcat 9.0.x reaches End of Life, there will be no requirement to continue 
supporting Tomcat Native 1.2.x so it seems likely that Tomcat Native 1.2.x will 
reach End of Life at the same point.

Tomcat 9.x is a special case for End of Life as it is the final version that 
supports Java EE. As such, once 9.0.x reaches end of life there will be 9.10.x 
but that will pick up all the changes from 10.1.x apart from the switch from 
the Java EE API to the Jakarta EE API. This means Tomcat 9.10.x will depend on 
Tomcat Native 2.0.x (and OpenSSL 3.0.x).

So, from the ASF's perspective, Tomcat Native 1.2.x (including support for 
OpenSSL 1.1.1) is expected to end some around March 2027. It might be as much 
as 18 months later but I don't see it extending as far as 2030.


All of that said, there are also downstream distributions of Apache Tomcat 
provided by various Linux distributions. If you obtain Tomcat and Tomcat Native 
via one of these distributions, it will remain supported by the distribution 
for the standard support timescales for that distribution - irrespective of 
whether or not the ASF has declared that version to have reached End of Life.


Finally, there are companies that provided commercial support for Tomcat that 
may be prepared to offer support beyond that provided by the ASF. 
My only word of caution is that if you opt to use such support, you should 
assure yourself that the provider has the in-house expertise necessary to 
back-port security fixes and produce updated Tomcat releases.

HTH,

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org