Chris, Thanks for the detailed response. I'll keep this on file for future reference, but it's a bigger project than what I am prepared to undertake at this time.
Dan -----Original Message----- From: Christopher Schultz <ch...@christopherschultz.net> Sent: Thursday, August 14, 2025 2:39 PM To: users@tomcat.apache.org Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service Daniel, On 8/13/25 8:59 PM, Daniel Schwartz wrote: > Chris, > > You have given me a lot to think about. > > Re your comment: "Getting ApacheBench is different depending upon your > development environment. If you let me know your OS, etc. I can point you in > the right direction." > > I found the following via Google: > https://urldefense.proofpoint.com/v2/url?u=https-3A__httpd.apache.org_ > docs_2.4_programs_ab.html&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_C > dpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=FYTiEEGpxyI > 35g9WJ8y3fZ5Je8vX4M0GpF72IoEhQPue-CY-PAjC4EZQkilLkh6l&s=RB3ipIsWbOeYvC > xSoWA_XG74cg6FprWIk4WS3qqZheI&e= This indicates that it comes as a > part of the Apache HTTP server and is useful only for this. Is there a > version that works with Glassfish? Is there any documentation? ab is a utility that you can run against any web server, including both Glassfish and Tomcat. It *does* come with Apache httpd, but you can get it separately and don't have to install the web server to use it. > I'm running on Windows on AWS. Apache Lounge [1] has binary versions of Apache httpd which include the utility. I didn't see any option to just get ab.exe from there, but you can download the ZIP package for Apache httpd and just take what you need. ChatGPT tells me that you will need these files from the ZIP archive: ab.exe Required DLLs (usually): libhttpd.dll libapr-1.dll libaprutil-1.dll pcre.dll or pcre2-*.dll (depending on version) ssleay32.dll, libeay32.dll, or OpenSSL DLLs (if SSL support is needed) Just extract all those files into the same directory and run "ab" from the command line. It's a CLI tool, so you'll do a lot of typing. I would refer to the documentation[2] to see what all the options are and what they mean. I would start with something simple like this: C:> ab -n 10 https://urldefense.proofpoint.com/v2/url?u=https-3A__example.com_yourapp_page&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=FYTiEEGpxyI35g9WJ8y3fZ5Je8vX4M0GpF72IoEhQPue-CY-PAjC4EZQkilLkh6l&s=g3Ca5FqEeCz6u0NYTSKj2F5wNT_ELJ4A7Ndjx6Mu1Lw&e= This will make 10 requests and give you some statistics. Then check your Glassfish pool counters. If you get a weird error like "apr_socket_connect(): Invalid argument (22)" or something, try using "127.0.0.1" instead of "localhost". -chris [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__apachelounge.com_downloads&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=FYTiEEGpxyI35g9WJ8y3fZ5Je8vX4M0GpF72IoEhQPue-CY-PAjC4EZQkilLkh6l&s=NOeF_ACBzetTE-QvTk1zfjY5rdjlIEjHcTq3IG4MzSQ&e= [2] (https://urldefense.proofpoint.com/v2/url?u=https-3A__apachelounge.com_downloads&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=FYTiEEGpxyI35g9WJ8y3fZ5Je8vX4M0GpF72IoEhQPue-CY-PAjC4EZQkilLkh6l&s=NOeF_ACBzetTE-QvTk1zfjY5rdjlIEjHcTq3IG4MzSQ&e=) > -----Original Message----- > From: Christopher Schultz <ch...@christopherschultz.net> > Sent: Wednesday, August 13, 2025 2:49 PM > To: users@tomcat.apache.org > Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service > > Dan, > > On 8/12/25 11:33 PM, Daniel Schwartz wrote: >> -----Original Message----- >> From: Robert Turner <rtur...@e-djuster.ca.INVALID> >> Sent: Tuesday, August 12, 2025 11:09 PM >> To: Tomcat Users List <users@tomcat.apache.org> >> Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service >> > > [snip]> >> For clarity, the 13 ms is the time it takes the pool to return to the caller >> (Java code) a connection when requested (as opposed to how long the >> connection is used for). Your other comments are accurate -- you could have >> many idle connections, however, 300 seconds is fairly reasonable for an idle >> timeout. What you want to be looking for is why you are peeking so high >> (maybe a benign connection that does nothing is triggering DB connections -- >> if so, this could lead to a DoS of your server with minimal effort). >> Something is either holding on to connections for a long period, or you are >> getting lots of connections at once. You should be able to tell easily which >> one it is because you will have access logs, and you can annotate your code >> with diagnostics as needed. Also, you can simply test on your development >> computer and see what's going on...without exposing your testing to the wide >> world of the Internet and the associated risks. >> >> DGS: Yes, I have access logs. I will look more closely and see if anything >> stands out. Within nginx you can block requests from specific IP addresses, >> but I haven't resorted to this yet. >> >> DGS: Am not sure what you mean by "you can annotate your code with >> diagnostics as needed" or "you can simply test on your development computer >> and see what's going on". What sort of diagnostics, and what sort of test? > > I suspect what Robert means by "annotate your code with diagnostics as > needed" just means "put something in your code to record additional > information. For example, logging the request URI, number of db connections > obtained, the amount of time each connection took to obtain (~13ms ain't bad, > but it's not exactly lightning-fast, either), and *the amount of time you > spend doing your queries before the connections are returned to the pool* > which seems to be something worth knowing. > > As for "what sort of test", I would recommend setting up a load-test in a > development environment like your own workstation. Get your application > running locally with the smallest pool you can muster (8 connections? Weird > that there is a minimum number of connections in this pool... I'm not > impressed). Make a few requests manually to ensure that the application is > working "normally". Then, unleash a load test on the application. > > There are a number of ways to do this. > > First, you can mash RELOAD in your web browser and see what happens. > That's usually not very helpful and doesn't give you much useful data. > > Another simple load-test is to use curl from the command line like this: > > $ while true ; do curl > https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8080_y > our_application_page&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnV > fiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHscrNmr4 > jn0Y-7PvWGT5hMBTsJ5_E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=di3ilaaf964xSIwNBTX > tkVqVtJr5iPLPVNWhYYVb9Xw&e= > /dev/null; done > > That will pound your application harder than you probably can through your > web browser. Pretty CTRL-C to stop the madness. > > When you are ready for proper load-testing, consider a simple tool like ab > [1]. Getting ApacheBench is different depending upon your development > environment. If you let me know your OS, etc. I can point you in the right > direction. > > When you are ready to go down a very deep rabbit hole, you can use JMeter[2] > but I suspect you will be able to learn a lot using both curl and ab, with ab > being the much better choice. > > I'm hoping you are able to get some questions answered about your application > like: > > 1. If I configure the pool with 8 connections, how many requests (in > series) are required to get the pool to break? > > 2. If I configure the pool with 8 connections, how many requests (in > parallel) are required to get the pool to break? > > 3. If I configure the pool with 100 connections, how many requests (in > series) are required to get the pool to break? > > 4. If I configure the pool with 100 connections, how many requests (in > parallel are required to get the pool to break? > > 5. What's the average response-time for a large number of serial requests? > Hopefully, it's "fast". For most users, a second of latency is about when > people start getting bored waiting for your site and consider switching to a > competitor. Most users are also willing to wait a lot longer for some > specific things. But loading your home page isn't one of them. "Generate a > complex report for me?" Sure, they'll wait. But "click on the item to get > details" is not something you want to make people wait for. > > 6. What's the average response-time for a large number of parallel > requests, mixed-in with serial requests at the same time. This > simulates *real world traffic*. And your application needs to be able > to meet > *peak-load* at any given time. So if you expect 100 simultaneous requests, > your application must be able to handle those requests without falling over. > I can tell you that if you have a fairly well-written application, running on > Tomcat, with a single connection in the database connection pool, and if your > application can respond to requests in 1 second, and you have a 60 second > wait-for-connection configuration on that pool, then your application can > handle a burst of traffic up to 60 simultaneous requests without rejecting a > single one. Once the 61st request is made (on top of the other 60 > simultaneous requests), one of them is going to throw an error. But the > application will not crash. > Tomcat will not crash. The JVM will not crash. A single error for a > single request will be returned to a single client. All 60 others will > get the expected response. If you crank that up to 100 simultaneous > requests, over and over again, 60% of them will return successfully > and the other 40% will get errors. Which is exactly what you want. If > your database (which is the real bottleneck, here) can really only > take > (e.g.) 60 simultaneous requests, then don't configure your application to > give it 100 or 1000. It's better to return errors to some clients than to > slowly grind everything to a halt. If your database starts thrashing, then > you have nothing. > > The "in series" questions are easy to answer with curl, and hopefully the > answers are "I can make a large number of requests in series and never break > the pool" regardless of the configuration. > > The "in parallel" questions will get down to what happens when the pool is > empty, how long are your db connections being used (i.e. duration between > pool-check-out and pool-return), etc., and will require that you use a tool > like ab or JMeter. ab is was easier, so, again, I'd recommend it for your > uses. > > ab gives you great tabular data when it terminates, and will answer basically > all the questions above if you run it with various options. > > Make sure to keep an eye on the application server's log files. For Tomcat, > that means catalina.out plus any application-log file you may have configured > in your application. You are looking for any errors and especially > stack-traces. For the "simple tests" (like you at the browser or trying 100 > requests back-to-back) you should see no errors. > > But you should start seeing errors at some point. And the server > (Glassfish, or Tomcat) should *not* crash. Both the JVM and the server > should stay running, and so should your application. Your application > and configuration *do* have a breaking point: that's the point of a > load-test: to understand what that breaking point actually is. But you > shouldn't be able to take-down the application server or the JVM. > > Hope that helps, > -chris > > [1] > https://urldefense.proofpoint.com/v2/url?u=https-3A__httpd.apache.org_ > docs_2.4_programs_ab.html&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_C > dpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHsc > rNmr4jn0Y-7PvWGT5hMBTsJ5_E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=JCzN9UqN9UNmQk > eTGL_eK59RndvTL3wnS3hkMKeqR8s&e= [2] > https://urldefense.proofpoint.com/v2/url?u=https-3A__jmeter.apache.org > _&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopg > QUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHscrNmr4jn0Y-7PvWGT5hMBTsJ5 > _E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=AzXpplYNVaVb3NtWqdjjZqsfT36_PTaOGOe6-C > pbpBo&e= > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
