Re: Does anyone have an approach to checking if Tomcat instance is UP?
In what context?? over the network?, over the web?, on the server? There is a portion of the shell script used in a Linux implementation for Tomcat 5.5 that checks for the running process: if start-stop-daemon --test --start --pidfile $CATALINA_PID \ --user $TOMCAT5_USER --startas $JAVA_HOME/bin/java \ /dev/null; then if [ -f $CATALINA_PID ]; then echo $DESC is not running, but pid file exists. exit 1 else echo $DESC is not running. exit 3 fi else echo -n $DESC is running with Java pid cat $CATALINA_PID exit 0 fi Smith Norton wrote: No - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
I have a script that checks to make sure that Tomcat is listening on the appropriate port. #!/bin/bash ... main() { ... chktomcat tomcat1 chktomcat tomcat2 ... } ... chktomcat() { netstat -a | grep $1 /dev/null 21 RC=$? if (( RC == 0 )) then echo Tomcat instance $1 is up. else echo Tomcat instance $1 is down. fi } ... main exit 0 I needed to add tomcat1 tomcat2 to the /etc/services file indicating the ports that they listen on. This script just shows that tomcat is listening on the network port. Thank you, Barry Goldstein [EMAIL PROTECTED] *** The information contained in this message may be privileged and/or confidential and protected from disclosure. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. Note that any views or opinions presented in this message are solely those of the author and do not necessarily represent those of Ameren. All emails are subject to monitoring and archival. Finally, the recipient should check this message and any attachments for the presence of viruses. Ameren accepts no liability for any damage caused by any virus transmitted by this email. If you have received this in error, please notify the sender immediately by replying to the message and deleting the material from any computer. Ameren Corporation *** - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Kim Do you have reason to believe that the OutOfMemory exception is due to: * some cumulative effect (e.g. memory leak), in which case you need to restart, preferably (just) before it happens * the qty of active sessions, so you need to load-balance when near some threshold (offload to a less busy server iff poss else please come back later) * the complexity of some user's request, in which case your app isn't really down at all... Paul Kim Albee wrote: Tracy, The JSP does a call to a method in our app -- which if it runs, that means the app is up and available -- the method does a simple query against the DB and then returns a status of OK if the method runs through just fine. In our example from this weekend -- the health.jsp (which is the one that does this check) ran and returned a good result, but the main index.jspreturned the 500 error with the OutOfMemory exception. So that is what is confusing here. thanks, Kim :-) On 8/20/07, Nelson, Tracy M. [EMAIL PROTECTED] wrote: How is your JSP checking your application? Are you issuing a request to your app and checking the HTTP status? If so, why isn't it recognizing the 500? Or is the JSP in your application which is failing? | -Original Message- | From: Kim Albee [mailto:[EMAIL PROTECTED] | Sent: Monday, 20 August, 2007 09:48 | | Repeatedly, that JSP will work, but the site is down because Tomcat hit an | OutOfMemory exception -- but our JSP (which is very small) still runs | through it's process and returns that everything is happy. Our | application | is up, but the 500 error is an OutOFMemory exception. - The information contained in this message is confidential proprietary property of Nelnet, Inc. and its affiliated companies (Nelnet) and is intended for the recipient only. Any reproduction, forwarding, or copying without the express permission of Nelnet is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this e-mail. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] -- Paul Singleton Jambusters Ltd tel: 01782 750821 fax: 08707 628609 VAT: 777 3904 85 Company no. 04150146 - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Jeff Hoffmann wrote: Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan, Dan Armbrust wrote: A simple cron job that points to a URL using lynx, and greps the output for what it should see will do the trick... I would use wget instead of Lynx, but that's just me. Don't forget that the OP said that his JSPs appear to run correctly even after the OOME. If what you're concerned about is an OOME, you can have a JSP that queries the runtime memory usage and outputs something easily parsable to alert you to a (pending) problem. IE: Runtime rt = Runtime.getRuntime(); double used = rt.totalMemory()-rt.freeMemory(); double free = rt.freeMemory(); double available = rt.maxMemory()-rt.totalMemory(); double usedpercent = (used/rt.maxMemory()) * 100; double freepercent = (free/rt.maxMemory()) * 100; double availablepercent = (available/rt.maxMemory()) * 100; Thanks for this - is there a similar (or any) way to anticipate out of PermGen? Paul S. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kim, Kim Albee wrote: The JSP does a call to a method in our app -- which if it runs, that means the app is up and available -- the method does a simple query against the DB and then returns a status of OK if the method runs through just fine. In our example from this weekend -- the health.jsp (which is the one that does this check) ran and returned a good result, but the main index.jspreturned the 500 error with the OutOfMemory exception. So that is what is confusing here. Two things are wrong: 1. Your health check is flawed ; otherwise, it would catch the fact that you have a dead server. 2. index.jsp is causing its own OOME, not reporting an existing condition. What does index.jsp do that health.jsp does not? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGyuhJ9CaO5/Lv0PARAoq1AJ45SG2Qa1qF/4BEJAoFoWG7yv4mrACdERCp 6CJVZUI8DlpWojvHP0+HgBM= =sPT9 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Everyone -- thanks for all the ideas and feedback. We've attempted to take the approach with our health.jsp to check the major functions in our application -- so if we can do a database request, that checks a bunch of things - and returns without error lets us know that our application is functioning. We figured that since it was running through Tomcat (as a .JSP) that tomcat would have to be up to have the page respond... so we didn't worry about Tomcat itself. Should we? In this instance the health.jsp continued to work and report all was good, while the main index.jsp got an OutOfMemory exception. If I query the runtime memory, will that have caught the exception happening in in the index.jsp? So if I check the available memory or percentage and it's lower than whtever threshhold we establish, then we could return a 'down' condition... would that be a solid way to catch any further memory errors? We use a monitoring tool that has the automated checks for the application JVM and we can set different threshholds there -- but I've got to be able to have the check run by the load balancer know that the system is down -- and it does only a simple check against this JSP page, and then knows to fail over -- so while we are working to establish threshhold alerts with our monitoring application, we also want to ensure the load balancer fails over accurately as well... So all suggestions are welcome. Kim :-) On 8/21/07, Christopher Schultz [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Kim, Kim Albee wrote: The JSP does a call to a method in our app -- which if it runs, that means the app is up and available -- the method does a simple query against the DB and then returns a status of OK if the method runs through just fine. In our example from this weekend -- the health.jsp (which is the one that does this check) ran and returned a good result, but the main index.jspreturned the 500 error with the OutOfMemory exception. So that is what is confusing here. Two things are wrong: 1. Your health check is flawed ; otherwise, it would catch the fact that you have a dead server. 2. index.jsp is causing its own OOME, not reporting an existing condition. What does index.jsp do that health.jsp does not? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGyuhJ9CaO5/Lv0PARAoq1AJ45SG2Qa1qF/4BEJAoFoWG7yv4mrACdERCp 6CJVZUI8DlpWojvHP0+HgBM= =sPT9 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Does anyone have an approach to checking if Tomcat instance is UP?
| From: Kim Albee [mailto:[EMAIL PROTECTED] | Sent: Tuesday, 21 August, 2007 12:49 | | We use a monitoring tool that has the automated checks for the application | JVM and we can set different threshholds there -- but I've got to be able | to | have the check run by the load balancer know that the system is down -- | and | it does only a simple check against this JSP page, and then knows to fail | over -- so while we are working to establish threshhold alerts with our | monitoring application, we also want to ensure the load balancer fails | over | accurately as well... | | So all suggestions are welcome. I'm not entirely sure how feasible this is, but can you have an output filter or valve catch (or at least note) the 500 status? If so, then maybe it could log an error to the database. Since you said your status page was able to hit the database even after the app crashed, you should hopefully be able to retrieve the error indication. - The information contained in this message is confidential proprietary property of Nelnet, Inc. and its affiliated companies (Nelnet) and is intended for the recipient only. Any reproduction, forwarding, or copying without the express permission of Nelnet is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this e-mail. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
No - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Does anyone have an approach to checking if Tomcat instance is UP?
Hello -- We have a load balanced situation, and we have a JSP that runs and checks our application to ensure it's up and returns a string that the monitor app is looking for if all is well. Repeatedly, that JSP will work, but the site is down because Tomcat hit an OutOfMemory exception -- but our JSP (which is very small) still runs through it's process and returns that everything is happy. Our application is up, but the 500 error is an OutOFMemory exception. We need a fool-proof way of knowing that the site is up or not, specifically so the load balancer will know to stop routing traffic to a server when it's down, and we can have people taking a look at what happened and bring the server back online without loss of service from a user perspective. Any suggestions on how to accomplish this? thanks, Kim :-)
Re: Does anyone have an approach to checking if Tomcat instance is UP?
A simple cron job that points to a URL using lynx, and greps the output for what it should see will do the trick... Dan On 8/20/07, Kim Albee [EMAIL PROTECTED] wrote: Hello -- We have a load balanced situation, and we have a JSP that runs and checks our application to ensure it's up and returns a string that the monitor app is looking for if all is well. Repeatedly, that JSP will work, but the site is down because Tomcat hit an OutOfMemory exception -- but our JSP (which is very small) still runs through it's process and returns that everything is happy. Our application is up, but the 500 error is an OutOFMemory exception. We need a fool-proof way of knowing that the site is up or not, specifically so the load balancer will know to stop routing traffic to a server when it's down, and we can have people taking a look at what happened and bring the server back online without loss of service from a user perspective. Any suggestions on how to accomplish this? thanks, Kim :-) - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Does anyone have an approach to checking if Tomcat instance is UP?
How is your JSP checking your application? Are you issuing a request to your app and checking the HTTP status? If so, why isn't it recognizing the 500? Or is the JSP in your application which is failing? | -Original Message- | From: Kim Albee [mailto:[EMAIL PROTECTED] | Sent: Monday, 20 August, 2007 09:48 | | Repeatedly, that JSP will work, but the site is down because Tomcat hit an | OutOfMemory exception -- but our JSP (which is very small) still runs | through it's process and returns that everything is happy. Our | application | is up, but the 500 error is an OutOFMemory exception. - The information contained in this message is confidential proprietary property of Nelnet, Inc. and its affiliated companies (Nelnet) and is intended for the recipient only. Any reproduction, forwarding, or copying without the express permission of Nelnet is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this e-mail. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan, Dan Armbrust wrote: A simple cron job that points to a URL using lynx, and greps the output for what it should see will do the trick... I would use wget instead of Lynx, but that's just me. Don't forget that the OP said that his JSPs appear to run correctly even after the OOME. Kim, how can /you/ tell that the server is dead, even when the pages still display? If you can write a JSP or a simple servlet to test for that detectable condition, then maybe you can report it in a web page, then use the above technique for testing your server's health. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGybyb9CaO5/Lv0PARAsW1AJ9BcjU7Wz/7CKqkSHuEzGblnJ9v/QCeOgJo P/x5UZzZ65U+nOjesskNaKY= =85U3 -END PGP SIGNATURE- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dan, Dan Armbrust wrote: A simple cron job that points to a URL using lynx, and greps the output for what it should see will do the trick... I would use wget instead of Lynx, but that's just me. Don't forget that the OP said that his JSPs appear to run correctly even after the OOME. If what you're concerned about is an OOME, you can have a JSP that queries the runtime memory usage and outputs something easily parsable to alert you to a (pending) problem. IE: Runtime rt = Runtime.getRuntime(); double used = rt.totalMemory()-rt.freeMemory(); double free = rt.freeMemory(); double available = rt.maxMemory()-rt.totalMemory(); double usedpercent = (used/rt.maxMemory()) * 100; double freepercent = (free/rt.maxMemory()) * 100; double availablepercent = (available/rt.maxMemory()) * 100; If your normal state is used % is 50%, you could send out a warning when it goes to 70% or restart if it goes to 90%. -- Jeff Hoffmann Head Plate Spinner PropertyKey.com - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: Does anyone have an approach to checking if Tomcat instance is UP?
Kim, You mentiond fool-proof... Perhaps a multi-pronged approach is best, if you have the time and inclination to implement it. 1) Apps can have issues for lots of reasons (running out of memory, db load and/or locks, thread deadlocks, etc, etc.) In lots of cases the VM/Tomcat are OK, but the app is not. With this in mind, if you can get your monitoring software to actually hit the app's core functionality itself, you can see if the app is responding, no matter if the VM/Tomcat are OK. Can you get your monitoring software to hit a non-trivial, non-access-controlled page? Can you get the monitor to login/logout every so often (say every few minutes)? How about building a health-check page in the app itself that queries all major resources like RDBMSs, and full text search engines, and LDAPs, etc and prints out status for all those? Depending on our access control setup, you can restrict access to the status page to your monitor (perpahs via IP address) or simply have that URL innaccessable from the public network. Also consider your tolerence for slowness. If the site is up but taking 20 seconds to respond, maybe that is broken. So of the monitor can check for response times that will help. 2) For the OOME condition, Jeff beat me to it. You can also, for example, use JMX and an http adapter to query an Mbean for the current memory state and alert if you are above 90%. The alert can be an email. This pro-active approach (and a fast response to the alert ;-) means your app won't get to the point of being unresponsive. 3) Perhaps this is too obvious, but why is the app running out of memory? Is there a leak or is the number of simultaneous requests too much? Have you given the app enough headroom in terms of max heap? We've had apps that just suck memory like nuts and have many simultaneous users, but if the VM is sized large enough they are happy. 4) For (free) monitoring software that can help with some of this, check out http://www.nagios.org/. Nagios and Splunk are cool. It is awesome what this stuff does for the price! Hope this helps at least a little, Brian Horblit Brian D. Horblit Senior Principal Engineer Thomson Healthcare (303) 486-6697 (800) 525-9083 x 6697 www.thomsonhealthcare.com [EMAIL PROTECTED] ---Original Message- --From: Jeff Hoffmann [mailto:[EMAIL PROTECTED] --Sent: Monday, August 20, 2007 10:25 AM --To: Tomcat Users List --Subject: Re: Does anyone have an approach to checking if --Tomcat instance is UP? [stuff deleted...] --If what you're concerned about is an OOME, you can have a --JSP that queries the runtime memory usage and outputs --something easily parsable to alert you to a (pending) problem. IE: -- --Runtime rt = Runtime.getRuntime(); --double used = rt.totalMemory()-rt.freeMemory(); double free --= rt.freeMemory(); double available = --rt.maxMemory()-rt.totalMemory(); double usedpercent = --(used/rt.maxMemory()) * 100; double freepercent = --(free/rt.maxMemory()) * 100; double availablepercent = --(available/rt.maxMemory()) * 100; -- --If your normal state is used % is 50%, you could send out --a warning when it goes to 70% or restart if it goes to 90%. -- --Jeff Hoffmann --Head Plate Spinner --PropertyKey.com -- --- --To start a new topic, e-mail: users@tomcat.apache.org To --unsubscribe, e-mail: [EMAIL PROTECTED] --For additional commands, e-mail: [EMAIL PROTECTED] -- -- - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Dan, True enough, except then those queries would get held as a user session, and we don't want that -- which is why we have a 'skinny' health.jsp that checks our app -- and 'should' crash if there are any issues with tomcat or the application -- but in this case, the main pages were getting out of memory exceptions, but the skinny health.jsp was running just fine... which it shouldn't be if there are failures in either Tomcat or the App. We're using Application Monitor to monitor the app and tomcat JVM instances as well as the health.jsp response. But for the load balancer, which only uses health.jsp, that's what needs to pick up the problem and report accordingly so the load balancer will take that server out of the load balanced cluster. Kim :-) On 8/20/07, Dan Armbrust [EMAIL PROTECTED] wrote: A simple cron job that points to a URL using lynx, and greps the output for what it should see will do the trick... Dan On 8/20/07, Kim Albee [EMAIL PROTECTED] wrote: Hello -- We have a load balanced situation, and we have a JSP that runs and checks our application to ensure it's up and returns a string that the monitor app is looking for if all is well. Repeatedly, that JSP will work, but the site is down because Tomcat hit an OutOfMemory exception -- but our JSP (which is very small) still runs through it's process and returns that everything is happy. Our application is up, but the 500 error is an OutOFMemory exception. We need a fool-proof way of knowing that the site is up or not, specifically so the load balancer will know to stop routing traffic to a server when it's down, and we can have people taking a look at what happened and bring the server back online without loss of service from a user perspective. Any suggestions on how to accomplish this? thanks, Kim :-) - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Does anyone have an approach to checking if Tomcat instance is UP?
Tracy, The JSP does a call to a method in our app -- which if it runs, that means the app is up and available -- the method does a simple query against the DB and then returns a status of OK if the method runs through just fine. In our example from this weekend -- the health.jsp (which is the one that does this check) ran and returned a good result, but the main index.jspreturned the 500 error with the OutOfMemory exception. So that is what is confusing here. thanks, Kim :-) On 8/20/07, Nelson, Tracy M. [EMAIL PROTECTED] wrote: How is your JSP checking your application? Are you issuing a request to your app and checking the HTTP status? If so, why isn't it recognizing the 500? Or is the JSP in your application which is failing? | -Original Message- | From: Kim Albee [mailto:[EMAIL PROTECTED] | Sent: Monday, 20 August, 2007 09:48 | | Repeatedly, that JSP will work, but the site is down because Tomcat hit an | OutOfMemory exception -- but our JSP (which is very small) still runs | through it's process and returns that everything is happy. Our | application | is up, but the 500 error is an OutOFMemory exception. - The information contained in this message is confidential proprietary property of Nelnet, Inc. and its affiliated companies (Nelnet) and is intended for the recipient only. Any reproduction, forwarding, or copying without the express permission of Nelnet is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this e-mail. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]