Dan,
On 8/12/25 11:33 PM, Daniel Schwartz wrote:
-----Original Message-----
From: Robert Turner <rtur...@e-djuster.ca.INVALID>
Sent: Tuesday, August 12, 2025 11:09 PM
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service
> [snip]>
For clarity, the 13 ms is the time it takes the pool to return to the caller
(Java code) a connection when requested (as opposed to how long the connection
is used for). Your other comments are accurate -- you could have many idle
connections, however, 300 seconds is fairly reasonable for an idle timeout.
What you want to be looking for is why you are peeking so high (maybe a benign
connection that does nothing is triggering DB connections -- if so, this could
lead to a DoS of your server with minimal effort). Something is either holding
on to connections for a long period, or you are getting lots of connections at
once. You should be able to tell easily which one it is because you will have
access logs, and you can annotate your code with diagnostics as needed. Also,
you can simply test on your development computer and see what's going
on...without exposing your testing to the wide world of the Internet and the
associated risks.
DGS: Yes, I have access logs. I will look more closely and see if anything
stands out. Within nginx you can block requests from specific IP addresses,
but I haven't resorted to this yet.
DGS: Am not sure what you mean by "you can annotate your code with diagnostics as needed"
or "you can simply test on your development computer and see what's going on". What sort
of diagnostics, and what sort of test?
I suspect what Robert means by "annotate your code with diagnostics as
needed" just means "put something in your code to record additional
information. For example, logging the request URI, number of db
connections obtained, the amount of time each connection took to obtain
(~13ms ain't bad, but it's not exactly lightning-fast, either), and *the
amount of time you spend doing your queries before the connections are
returned to the pool* which seems to be something worth knowing.
As for "what sort of test", I would recommend setting up a load-test in
a development environment like your own workstation. Get your
application running locally with the smallest pool you can muster (8
connections? Weird that there is a minimum number of connections in this
pool... I'm not impressed). Make a few requests manually to ensure that
the application is working "normally". Then, unleash a load test on the
application.
There are a number of ways to do this.
First, you can mash RELOAD in your web browser and see what happens.
That's usually not very helpful and doesn't give you much useful data.
Another simple load-test is to use curl from the command line like this:
$ while true ; do curl http://localhost:8080/your/application/page >
/dev/null; done
That will pound your application harder than you probably can through
your web browser. Pretty CTRL-C to stop the madness.
When you are ready for proper load-testing, consider a simple tool like
ab [1]. Getting ApacheBench is different depending upon your development
environment. If you let me know your OS, etc. I can point you in the
right direction.
When you are ready to go down a very deep rabbit hole, you can use
JMeter[2] but I suspect you will be able to learn a lot using both curl
and ab, with ab being the much better choice.
I'm hoping you are able to get some questions answered about your
application like:
1. If I configure the pool with 8 connections, how many requests (in
series) are required to get the pool to break?
2. If I configure the pool with 8 connections, how many requests (in
parallel) are required to get the pool to break?
3. If I configure the pool with 100 connections, how many requests (in
series) are required to get the pool to break?
4. If I configure the pool with 100 connections, how many requests (in
parallel are required to get the pool to break?
5. What's the average response-time for a large number of serial
requests? Hopefully, it's "fast". For most users, a second of latency is
about when people start getting bored waiting for your site and consider
switching to a competitor. Most users are also willing to wait a lot
longer for some specific things. But loading your home page isn't one of
them. "Generate a complex report for me?" Sure, they'll wait. But "click
on the item to get details" is not something you want to make people
wait for.
6. What's the average response-time for a large number of parallel
requests, mixed-in with serial requests at the same time. This simulates
*real world traffic*. And your application needs to be able to meet
*peak-load* at any given time. So if you expect 100 simultaneous
requests, your application must be able to handle those requests without
falling over. I can tell you that if you have a fairly well-written
application, running on Tomcat, with a single connection in the database
connection pool, and if your application can respond to requests in 1
second, and you have a 60 second wait-for-connection configuration on
that pool, then your application can handle a burst of traffic up to 60
simultaneous requests without rejecting a single one. Once the 61st
request is made (on top of the other 60 simultaneous requests), one of
them is going to throw an error. But the application will not crash.
Tomcat will not crash. The JVM will not crash. A single error for a
single request will be returned to a single client. All 60 others will
get the expected response. If you crank that up to 100 simultaneous
requests, over and over again, 60% of them will return successfully and
the other 40% will get errors. Which is exactly what you want. If your
database (which is the real bottleneck, here) can really only take
(e.g.) 60 simultaneous requests, then don't configure your application
to give it 100 or 1000. It's better to return errors to some clients
than to slowly grind everything to a halt. If your database starts
thrashing, then you have nothing.
The "in series" questions are easy to answer with curl, and hopefully
the answers are "I can make a large number of requests in series and
never break the pool" regardless of the configuration.
The "in parallel" questions will get down to what happens when the pool
is empty, how long are your db connections being used (i.e. duration
between pool-check-out and pool-return), etc., and will require that you
use a tool like ab or JMeter. ab is was easier, so, again, I'd recommend
it for your uses.
ab gives you great tabular data when it terminates, and will answer
basically all the questions above if you run it with various options.
Make sure to keep an eye on the application server's log files. For
Tomcat, that means catalina.out plus any application-log file you may
have configured in your application. You are looking for any errors and
especially stack-traces. For the "simple tests" (like you at the browser
or trying 100 requests back-to-back) you should see no errors.
But you should start seeing errors at some point. And the server
(Glassfish, or Tomcat) should *not* crash. Both the JVM and the server
should stay running, and so should your application. Your application
and configuration *do* have a breaking point: that's the point of a
load-test: to understand what that breaking point actually is. But you
shouldn't be able to take-down the application server or the JVM.
Hope that helps,
-chris
[1] https://httpd.apache.org/docs/2.4/programs/ab.html
[2] https://jmeter.apache.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
