Hi Robert, Thanks for your thoughts. See my replies marked DSG.
Dan -----Original Message----- From: Robert Turner <rtur...@e-djuster.ca.INVALID> Sent: Monday, August 11, 2025 9:37 PM To: Tomcat Users List <users@tomcat.apache.org> Subject: Re: How to access a REST service Dan, According to your attached document, your high water mark was 269, which means at some point you had 269 connections in the pool at once, suggesting around that many in use at once. You likely could lower your limit to 300-500 as a result. DGS: I could try this and see if it still works. But it seems that a higher number wouldn't hurt. The system just wouldn't need that many at any given time, so they wouldn't be created. As a few of us have said, that is quite a high number and somewhat surprising, and as Chris said, 3 connections per request is also "odd" (unless you are accessing 3 different databases, or some other reason that requires distinctly different connections). DGS: A single simple query to my interface requires three database accesses, one to retrieve a list of countries, then, when the use selects a country, another database access to get the lists of states and cities associated with that country, and then assuming the simplest case where there are no states or cities, then a third database access when the user enters a year and requests the holidays for that country in that year. If there are states selected, then a fourth database access is required to get the cites for that state or to determine that there aren't any. From the attached document, it doesn't look like you have routine leaks, but as noted before, and by others, the code could leak in certain circumstances, although it generally doesn't seem to be in your sample period. There are very likely design / performance issues with the application related to the use of connections, but given what you've said before, this is not a priority for you at this time. I think for a lot of us, we see a fairly concerning issue and would like to help you resolve it, but of course, it's your prerogative to do what makes sense to you. DGS: I appreciate this, but at this point I am not really sure that there is a problem or what it might be if there is one. My current code seems to be working and Glassfish says that there aren't any leaks. DGS: Thanks again. I greatly appreciate everyone's taking the time to respond. Best of luck, Robert On Mon, Aug 11, 2025, 21:23 Daniel Schwartz <d...@danielgschwartz.com> wrote: > Hi Chris, > > Please see my replies after each of your comments. I have attached a > copy of the Glassfish page that shows the Connection Pool monitoring > data. I don't know what all this means, whether it is good, bad, or normal. > However, I do note the one item "NumPotentialConnLeak" with value "O > count" and description "Number of potential connection leaks", which > clearly means that it is not detecting any leaks. So this doesn't > seem to be an issue. > > I still don't know why the system was crashing previously. The > Glassfish log file was showing a message saying that it was unable to > create any more connections, and then it crashed. At that time the > maximum pool size was the default 32. The person I talked to at > Omnifish said that maximum pool size is the maximum possible number of > connections, so I tried raising this to 100, and the system still kept > crashing, so then I raised it to 1000, and now it is working fine. > > When I initially wrote to this list, I was considering switching to > Tomcat because of the problems I was having with Glassfish. For the > time being, however, since everything seems to be working, I'm not > prepared to undertake that project. Have many other things to do. > > Other replies below are marked DGS. But the answer to most of your > questions is that "I don't know". I probably should try to find a > Glassfish reference manual and try to learn what all those data items > mean. > > Dan > > -----Original Message----- > From: Christopher Schultz <ch...@christopherschultz.net> > Sent: Monday, August 11, 2025 3:41 PM > To: users@tomcat.apache.org > Subject: Re: How to access a REST service > > > Dan, > > On 8/8/25 12:21 PM, Daniel Schwartz wrote: > > From your and other people's replies, I'm learning that tracking > > down memory leaks can be quite daunting and possibly beyond me. > No, you can definitely figure this out, and it will be well worth the > trouble. > > > I'm aware of the try-with-resources instruction and will look into it. > > I wrote this code before learning about this. > > > > In any case, I will follow your advice and add a "finally" block to > > the "try" statement. I'm currently closing the connection in case > > an SQL exception is thrown, but I hadn't considered the possibility > > of other kinds of exceptions. I suppose that there could be such > > exceptions, but I'm mystified regarding what those might be. > > > > However, there has been a new development. After "fishing" around > > in the Glassfish administration system, I found that there is a way > > to monitor the JDBC connection pool and get all sorts of information. > > Based on this, I really don't think that there is any memory leak at > > all. > > > > When running a simple query on my website, I see that > > NumConnAcquired increases by three (which is what I would expect), > > and immediately thereafter NumConnReleased increases by the same number. > Wait, a single request obtains 3 connections? That sounds odd to me, > unless you are executing more than one of these types of operations > per request. It's not particularly odd to borrow more than one > connection per request, but I just wanted to make sure that we were > all understanding what it means for a "normal request". > > > So every connection that is being acquired is also being released. I > > think that, if there were a memory leak, the number of connections > > acquired would be increasing at a faster pace than the number being > > released. > >> Also, there is a NumConnUsed statistic that has a "High Water Mark" > > which is described as the "maximum number of connections that were > > used", and this is currently 160, which is not a lot. > That sounds like a lot to me. Is that reporting the total number of > connections requested from the pool, or the current number of > connections that are still "checked out" of the pool? > > DGS: Don't know. > > And are you sure that pooling actually is being used? > > DGS: Yes. > > > So it seems that the system is behaving normally. And I have > > noticed that since setting the maximum pool size to 1000, the system > > has not crashed for several days. Probably, 1000 is more than > > necessary, and you just need make sure you have more possible > > connections available than the anticipated future high-water mark. > > But this seems to be working. > > > > So maybe I'm just imagining problems that no longer exist. As long > > as the system keeps running, I'm fine with this. > > > > Thanks to everyone for their replies. > If you let this run for a few days, what does the NumConnUsed counter > look like? Is it counting pool-check-outs, or is it counting > real-connections-created? > > DGS: Don't know. > > -chris > > > -----Original Message----- > > From: Robert Turner <rtur...@e-djuster.ca.INVALID> > > Sent: Friday, August 8, 2025 1:32 AM > > To: Tomcat Users List <users@tomcat.apache.org> > > Subject: Re: How to access a REST service > > > > (sorry for any errors, I'm bashing this out quickly on my phone > > before going to bed) > > > > On Fri, Aug 8, 2025, 00:59 Daniel Schwartz <d...@danielgschwartz.com> > wrote: > > > >> Hello Chuck, > >> > >> You are correct, I only catch SQL exceptions. I have modified my > >> code so that the Connection object will be closed if an SQL > >> exception is thrown, but I wasn’t aware that there are other types > >> of exceptions that I should look for. I will look into this. > >> > > > > > > I would encourage entering "avoid java resource leak" into Google > > and > reading the "AI" generated details or any other articles that come up. > The "AI" description it gave me was a reasonable overview of what we > are trying to convey to you and how to resolve it. > > > > Unchecked exceptions can be thrown and the compiler will not warn > > about > it. > > This may be the issue, but it it not guaranteed, but it definitely > > is a > potential and obvious problem with the code. > > > > You need to ensure that everything related to the connection > (connection, ResultSet, Statement, PreparedStatement) are also > released > (closed) or they will retain references to the connection and the > connection may not be returned to the pool. > > > > As previously mentioned by others, using try-with-resources for all > > of > these objects is strongly recommended to avoid leaks. If you are uaing > older Java (consider upgrading), you will need to use finally blocks > instead (as previously mentioned by others). > > > > Unfortunately, you have only posted a portion of the code, and it > > may be > the case that it the problems stems from elsewhere (like where you > make the queries, or anywhere else that obtains a connection). > > > > It should be easy to get a tool to analyze the code (say your IDE > > (if > you use one), PMD, Spotbugs, Sonarqube, etc.) and they would likely > point out all the potential resource leaks. Many IDEs, such as > Intellij IDEA, also provide such analysis built in. They basically > look for any object derived from AutoClosable and ensure you are > closing the object in a finally clause (or in all potential code > paths). If you aren't, you have a leak. > > > > > >> However, I don’t think that this is causing a memory link, since my > >> code has always run perfectly without throwing any exceptions at > >> all, as far as I know. > >> > > > > You may not know. The container (Glassfish or Tomcat) often catches > > them > and unless you are checking their logs, you may not see any evidence > of them. > > > > > >> But I’m beginning to wonder if my “close” operation is actually > >> being recognized by Glassfish. It seems possible that this is not > >> being communicated to Glassfish, and the Connection object > >> continues to be marked as being in use. This would be a leak. But > >> I don’t know if this is happening. > >> > > > > It's certainly possible that some behaviour of the container and > > it's > pooling system is exasperating your issue, but I strongly doubt the > container is at fault. It is almost guaranteed to be your code. > > > > > > There are other possibly issues with your code, but these are less > "critical" than your leaks, but could also be a potential source of issues. > > Namely you have static Global's that are not protected against > concurrency, and the containers typically run each request in separate > threads. You could be creating more than one instance of your "singleton" > as a result. > > > > > > > >> Dan > >> > >> From: Chuck Caldarale <n82...@gmail.com> > >> Sent: Thursday, August 7, 2025 11:52 PM > >> To: Tomcat Users List <users@tomcat.apache.org> > >> Subject: Re: How to access a REST service > >> > >>> On 2025 Aug 7, at 21:43, Daniel Schwartz <d...@danielgschwartz.com > >> <mailto:d...@danielgschwartz.com>> wrote: > > I have just posted > >> everything again, this time with a few modifications. You say, > >> "Last time you posted it, it was prone to leaking connection." > >> NkdkJdXPPEBannerStart > >> Be Careful With This Message > >> From (Chuck Caldarale <n82...@gmail.com>)< > >> https://godaddy1.cloud-protect.net/email-details/?k=k1&payload=5361 > >> 6c > >> 7 > >> 465645f5fed27a22a24a04d2b4474850fcd4085279f5deb0a4b45aff271dc217bd5 > >> b8 > >> a > >> c8e65b868f686679b9d5ae69c216b8828c6ab976fd096f84ebdac1ecb38a6f62a2e > >> 05 > >> b > >> 1ce3cda3ef1fc6615c2d5d63b3db6ac3d93719e10964cf6ecb92b637712794fd981 > >> 4a > >> 3 > >> 6d8e401e9510d22d52433dc8526a1fd76ffd29c927dd092a286a002fa6edad24fdc > >> 28 > >> 9 > >> 956069ff246bd9f54272522f4c2b34608a52f7c8a6db9c157660efc922fcaa993a2 > >> 7c > >> e > >> 11dcae209b3f911b99a3e50e92edf0e120af447100e9d80cd45918b2f85aa8673ef > >> b1 > >> 4 e6bddb38cdfaa2e1ba5496bad5db8df0857d443aeb0399f46406f > >>> > >> Learn More< > >> https://godaddy1.cloud-protect.net/email-details/?k=k1&payload=5361 > >> 6c > >> 7 > >> 465645f5fed27a22a24a04d2b4474850fcd4085279f5deb0a4b45aff271dc217bd5 > >> b8 > >> a > >> c8e65b868f686679b9d5ae69c216b8828c6ab976fd096f84ebdac1ecb38a6f62a2e > >> 05 > >> b > >> 1ce3cda3ef1fc6615c2d5d63b3db6ac3d93719e10964cf6ecb92b637712794fd981 > >> 4a > >> 3 > >> 6d8e401e9510d22d52433dc8526a1fd76ffd29c927dd092a286a002fa6edad24fdc > >> 28 > >> 9 > >> 956069ff246bd9f54272522f4c2b34608a52f7c8a6db9c157660efc922fcaa993a2 > >> 7c > >> e > >> 11dcae209b3f911b99a3e50e92edf0e120af447100e9d80cd45918b2f85aa8673ef > >> b1 > >> 4 e6bddb38cdfaa2e1ba5496bad5db8df0857d443aeb0399f46406f > >>> > >> Potential Impersonation > >> The sender's identity could not be verified and someone may be > >> impersonating the sender. Take caution when interacting with this > message. > >> > >> NkdkJdXPPEBannerEnd > >> > >> > >> > >>> On 2025 Aug 7, at 21:43, Daniel Schwartz <d...@danielgschwartz.com > >> <mailto:d...@danielgschwartz.com>> wrote: > >> > >>> > >> > >>> I have just posted everything again, this time with a few > >> modifications. You say, "Last time you posted it, it was prone to > >> leaking connection." Could you say more exactly where you see saw > >> the > leak? > >> > >> > >> > >> > >> > >> You only catch errors of the type SQLException, which means > >> anything else will lose the connection. This is the reason you > >> should always use try - catch - finally (or the newer > >> with-resources syntax) around database manipulations, as recommended in > >> Chris’ blog post. > >> > >> > >> > >> - Chuck > >> > >> > >> > >> > >> > >>> Thanks, > >> > >>> > >> > >>> Dan Schwartz > >> > >>> > >> > >>> -----Original Message----- > >> > >>> From: Robert Turner <rtur...@e-djuster.ca.INVALID<mailto: > >> rtur...@e-djuster.ca.INVALID>> > >> > >>> Sent: Thursday, August 7, 2025 5:07 PM > >> > >>> To: Tomcat Users List <users@tomcat.apache.org<mailto: > >> users@tomcat.apache.org>> > >> > >>> Subject: Re: How to access a REST service > >> > >>> > >> > >>> Dan, > >> > >>> > >> > >>> On Thu, Aug 7, 2025 at 5:01 PM Daniel Schwartz > >>> <d...@danielgschwartz.com > >> <mailto:d...@danielgschwartz.com>> > >> > >>> wrote: > >> > >>> > >> > >>>> Hello Chris, > >> > >>>> > >> > >>>> Thank you for your reply, but I'm still unsure. You seem to be > >> > >>>> implying that I have a memory leak, i.e., many connection objects > >> > >>>> being created that are not being closed. However, I really don't > >> > >>>> think this is happening. My code closes each connection > >>>> immediately > >> after using it. > >> > >>> > >> > >>> > >> > >>> Maybe post your code again. Last time you posted it, it was prone > >>> to > >> leaking connections. If it hasn't changed, that is likely your problem. > >> > >>> > >> > >>> > >> > >>>> > >> > >>>> My understanding is that the only way the maximum pool size of X, > >> > >>>> whatever that is, would be a limitation is if there was an > >>>> attempt to > >> > >>>> create X+1 simultaneous connections. When you do this in > >>>> Glassfish, > >> > >>>> it outputs an error message saying that no more connections can > >>>> be > >> > >>>> created and then crashes. You have to go back in and manually > >>>> restart > >> it. > >> > >>>> > >> > >>>> I believe that the essential problem, as explained in a previous > >>>> email > >> > >>>> to Rob Sargent, is that I'm getting several hundred database > >>>> requests > >> > >>>> per day from web crawlers. I just spent some time reading > >>>> through my > >> > >>>> ngnix access.log and found that the vast majority of these are > >>>> from > >> GoogleBot. > >> > >>>> My guess it that, due to a time lag between opening and closing > >> > >>>> connections, many connections will be opened simultaneously. > >>>> This is > >> > >>>> why a small pool size won't work. > >> > >>>> > >> > >>>> Also, I'm advised to not block the web crawlers because this > >>>> assists > >> > >>>> with SEO. My understanding is that you just have to live with this. > >> > >>>> > >> > >>>> I don't think there is an issue with my code. The only answer I > >>>> can > >> > >>>> come up with is to have a large maximum pool size, larger that > >>>> the > >> > >>>> expected number of simultaneous accesses. > >> > >>>> > >> > >>> There is almost definitely a problem with your code > >>> (unfortunately), or > >> your database requests are very slow and triggered by any connection. > >> > >>> > >> > >>> We run servers that handle much more traffic than you are > >>> describing and > >> make thousands of DB requests per minute, and we rarely go over 10 > >> DB connections being used at a time. > >> > >>> > >> > >>> There is almost for sure something leaking in your code. This is > >>> very > >> unlikely to be a problem with the pooling ("select isn't broken"). > >> You are looking for unlikely causes to the problem. > >> > >>> > >> > >>> > >> > >>>> > >> > >>>> I originally wrote to this email list because I was thinking of > >> > >>>> shifting from Glassfish to Tomcat, and was trying to learn how to > >>>> do > >> > >>>> this. I think I do know how to do this now, and might try doing this. > >> > >>>> My understanding is that the connection pooling that works with > >>>> Tomcat > >> > >>>> doesn't have that same limitation as Glassfish, and one can have > >> > >>>> connections that exist outside the pool. This would resolve the > >>>> issue > >> > >>>> I'm currently having with Glassfish. > >> > >>>> > >> > >>>> Best regards, > >> > >>>> > >> > >>>> Dan Schwartz > >> > >>>> ...snip.. > >> > >>> > >> > >>> ------------------------------------------------------------------ > >>> -- > >>> - > >> > >>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org<mailto: > >> users-unsubscr...@tomcat.apache.org> > >> > >>> For additional commands, e-mail: users-h...@tomcat.apache.org<mailto: > >> users-h...@tomcat.apache.org> > >> > >>> > >> > >> > >> > >> > >> > >> ------------------------------------------------------------------- > >> -- > >> > >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org<mailto: > >> users-unsubscr...@tomcat.apache.org> > >> > >> For additional commands, e-mail: users-h...@tomcat.apache.org<mailto: > >> users-h...@tomcat.apache.org> > >> > >> > >> > > > > -------------------------------------------------------------------- > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > > For additional commands, e-mail: users-h...@tomcat.apache.org > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org
