RE: java.net.SocketException: Too many open files
Greetings All, I had the exact same issue at one time a while ago, my resolution was the setting of the max number of files that was aloowed to be opened by the OS which affected the tomcat application. After I increased the max-files setting, all was resolved. Kind regards / Met vriendelijke groet, Lawrence Lamprecht -Original Message- From: Caldarale, Charles R [mailto:chuck.caldar...@unisys.com] Sent: Wednesday, June 03, 2009 10:20 PM To: Tomcat Users List Subject: RE: java.net.SocketException: Too many open files From: John Cartwright [mailto:john.c.cartwri...@noaa.gov] Subject: Re: java.net.SocketException: Too many open files I'll look into modifying the memory parameters. That would be a complete waste of time. The heap size has nothing to do with the problem you're seeing, and the discussion cited by Martin is full of erroneous information. Please ignore it. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Three tomcat instances
If you want to run 3 tomcats then you need to have 3 tomcat installations. One tomcat per instance that you want to have server, this in turn will mean that you will have 3 separate server.xml files. You can then load balance the 3 tomcats to be referenced as required. You can even create workers for each instance and then not loadbalance. Call exach required worker as you need it directly. Kind regards / Met vriendelijke groet, Lawrence Lamprecht -Original Message- From: xalia...@freemail.gr [mailto:xalia...@freemail.gr] Sent: Wednesday, June 03, 2009 9:54 AM To: users@tomcat.apache.org Subject: Three tomcat instances I write a java web service and depending on a result i want it to call other web service (http://localhost:80/axis/serv.jws). But, i want serv.jws to be handled by three different servers depending on where i want to send it. How can i run three tomcats simultaneously?I read the server documentation,but mentions that only one server i can set into server.xml Any help appreciated. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: jk Status not showing errors
What you could do is tail -f mod_jk.log file. Then take down the tomcat, see if the errors appear. You should see something like the following. Good Entries to Track Attempting to map context URI '/search-engine*' ajp_unmarshal_response::jk_ajp_common.c (621): status = 302 Maintaining worker loadbalancer1 Maintaining worker prod_se1 Maintaining worker prod_se2 Maintaining worker prod_sea Maintaining worker prod_seb service::jk_lb_worker.c (612): service worker=prod_sea jvm_route=prod_sea service::jk_lb_worker.c (612): service worker=prod_seb jvm_route=prod_seb service::jk_lb_worker.c (612): service worker=prod_sea jvm_route=prod_se1 service::jk_lb_worker.c (612): service worker=prod_seb jvm_route=prod_se2 Possible Error Entries Error connecting to tomcat. Tomcat is probably not started or is listening on the wrong port. worker=prod_se1 failed Error connecting to tomcat. Tomcat is probably not started or is listening on the wrong port. worker=prod_se2 failed You should be able to trace where your config is problematic. Kind regards / Met vriendelijke groet, Lawrence Lamprecht -Original Message- From: Matthew Laird [mailto:lai...@sfu.ca] Sent: Tuesday, June 02, 2009 8:53 PM To: Tomcat Users List Subject: Re: jk Status not showing errors Unfortunately I'm not seeing that. What I did was start both Tomcats in my LB pair, start Apache, then I take the second Tomcat down to see if it will detect it being failed. Unfortunately it never seems to, it just shows the second as OK/IDLE, and happily directs all requests to the first. This concerns me, because if the second were to fail, then later the first, everything would die and I'd have no advance warning. I can't seem to make it ping and detect a dead Tomcat. I am using the latest version of mod_jk, I upgraded that before I began playing with the load balancer settings. I'd appreciate any feedback on what I might be doing wrong. Thanks. workers.properties: worker.list=production,development,old,jkstatus worker.production.type=lb worker.production.balance_workers=production1,production2 worker.production.sticky_session=True worker.production.method=S worker.lbbasic.type=ajp13 worker.lbbasic.connect_timeout=1 worker.lbbasic.recovery_options=7 worker.lbbasic.socket_keepalive=1 worker.lbbasic.socket_timeout=60 worker.lbbasic.ping_mode=CI worker.production1.reference=worker.lbbasic worker.production1.port=8009 worker.production1.host=localhost worker.production2.reference=worker.lbbasic worker.production2.port=8012 worker.production2.host=localhost worker.development.port=8010 worker.development.host=localhost worker.development.type=ajp13 worker.old.port=8011 worker.old.host=localhost worker.old.type=ajp13 worker.jkstatus.type=status Lawrence Lamprecht wrote: I do not know if this is relevant or not, but I have just installed the latest version of mod_jk and the jkstatus is very much better than it used to be. I had the same issue with loadbalancers not showing when they are offline or broken. With the latest version, jksataus has the possibility to auto refresh itself. This now shouws when load balancers go down without a request being send to it. It is pretty dynamic as well. I ran several tests where I took one of the balancers down, and left jkstatus refreshing every 10 seconds and that told me that the worker was in error. It also shows you that the work is OK - IDLE when the worker is not being used but is good. As soon as it receives a request the status then changes to OK. Hope this helps. Kind regards / Met vriendelijke groet, Lawrence Lamprecht Application Content Manager QUADREM Netherlands B.V. Kabelweg 61, 1014 BA Amsterdam Post Office Box 20672, 1001 NR Amsterdam Office: +31 20 880 41 16 Mobile: +31 6 13 14 26 31 Fax: +31 20 880 41 02 Read our blog: Intelligent Supply Management - Your advantage -Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Saturday, May 30, 2009 2:46 PM To: Tomcat Users List Subject: Re: jk Status not showing errors On 29.05.2009 22:50, Matthew Laird wrote: Good afternoon, I've been trying to get the jkstatus component of mod_jk running, and I'm not quite sure what I'm doing wrong in trying to have it report dead Tomcat instances. I have two tomcat instances setup in a load balancer, as a test I've taken down one of them. However the jkstatus screen still shows both of them as OK. I'm not sure what I'm missing from my workers.properties file to make it test the Tomcat and report a failed instance, so I can set Nagios to monitor this page and report problems. My workers.properties is: worker.list=production,development,old,jkstatus worker.production.type=lb worker.production.balance_workers=production1,production2 worker.production.sticky_session=True worker.production.method=S worker.lbbasic.type=ajp13 worker.lbbasic.connect_timeout=1
RE: jk Status not showing errors
Below is the config that I have and this works. I have looked at your workers.properties file. There are few entries that I am not sure of. So I would suggest trying to simplify your config until you get a functional system. Once you reach that stage, then you can add more complication to it until you are happy with a final config. All the while tailing the mod_jk.log file to monitor the changes and see the effct of the system. Hop this helps. **Workers.properties file** worker.list= loadbalancer1, loadbalancer2, prod_se1, prod_se2, prod_sea, prod_seb worker.prod_se1.port=8009 worker.prod_se1.host=10.16.6.166 worker.prod_se1.type=ajp13 worker.prod_se1.lbfactor=1 worker.prod_se2.port=8009 worker.prod_se2.host=10.16.6.167 worker.prod_se2.type=ajp13 worker.prod_se2.lbfactor=1 worker.prod_sea.port=8210 worker.prod_sea.host=10.16.6.166 worker.prod_sea.type=ajp13 worker.prod_sea.lbfactor=1 worker.prod_seb.port=8210 worker.prod_seb.host=10.16.6.167 worker.prod_seb.type=ajp13 worker.prod_seb.lbfactor=1 worker.loadbalancer1.type=lb worker.loadbalancer1.balance_workers=prod_se1,prod_se2 worker.loadbalancer2.type=lb worker.loadbalancer2.balance_workers=prod_sea,prod_seb **jk.conf file** JkWorkersFile /etc/httpd/conf/workers.properties JkLogFile /etc/httpd/logs/mod_jk.log JkLogLevel debug ErrorLog /etc/httpd/logs/jk_error_log CustomLog /etc/httpd/logs/jk_access_log common JkMount /search-engine* loadbalancer1 JkMount /2-search-engine* loadbalancer2 Kind regards / Met vriendelijke groet, Lawrence Lamprecht -Original Message- From: Matthew Laird [mailto:lai...@sfu.ca] Sent: Tuesday, June 02, 2009 9:41 PM To: Tomcat Users List Subject: Re: jk Status not showing errors I'm not seeing anything like that. I just took both Tomcats down, I instantly get the 503 from Apache when I try to load the application. However tailing the mod_jk.log, I just see entries like this: [Tue Jun 02 12:36:23 2009] jkstatus www.innatedb.ca 0.000360 [Tue Jun 02 12:36:26 2009] jkstatus www.innatedb.ca 0.000263 [Tue Jun 02 12:36:39 2009] production www.innatedb.ca 0.498998 [Tue Jun 02 12:36:40 2009] jkstatus www.innatedb.ca 0.000282 mod_jk seems happy sending the requests to Tomcat, and doesn't seem to notice there's no actual Tomcat responding. Only after a few minutes does the JK Status screen go to ERR/REC for both. I would think this is the kind of thing mod_jk should notice instantly, when there's no Tomcat where there should be one. Or am I missing something? Thanks. Lawrence Lamprecht wrote: What you could do is tail -f mod_jk.log file. Then take down the tomcat, see if the errors appear. You should see something like the following. Good Entries to Track Attempting to map context URI '/search-engine*' ajp_unmarshal_response::jk_ajp_common.c (621): status = 302 Maintaining worker loadbalancer1 Maintaining worker prod_se1 Maintaining worker prod_se2 Maintaining worker prod_sea Maintaining worker prod_seb service::jk_lb_worker.c (612): service worker=prod_sea jvm_route=prod_sea service::jk_lb_worker.c (612): service worker=prod_seb jvm_route=prod_seb service::jk_lb_worker.c (612): service worker=prod_sea jvm_route=prod_se1 service::jk_lb_worker.c (612): service worker=prod_seb jvm_route=prod_se2 Possible Error Entries Error connecting to tomcat. Tomcat is probably not started or is listening on the wrong port. worker=prod_se1 failed Error connecting to tomcat. Tomcat is probably not started or is listening on the wrong port. worker=prod_se2 failed You should be able to trace where your config is problematic. Kind regards / Met vriendelijke groet, Lawrence Lamprecht -Original Message- From: Matthew Laird [mailto:lai...@sfu.ca] Sent: Tuesday, June 02, 2009 8:53 PM To: Tomcat Users List Subject: Re: jk Status not showing errors Unfortunately I'm not seeing that. What I did was start both Tomcats in my LB pair, start Apache, then I take the second Tomcat down to see if it will detect it being failed. Unfortunately it never seems to, it just shows the second as OK/IDLE, and happily directs all requests to the first. This concerns me, because if the second were to fail, then later the first, everything would die and I'd have no advance warning. I can't seem to make it ping and detect a dead Tomcat. I am using the latest version of mod_jk, I upgraded that before I began playing with the load balancer settings. I'd appreciate any feedback on what I might be doing wrong. Thanks. workers.properties: worker.list=production,development,old,jkstatus worker.production.type=lb worker.production.balance_workers=production1,production2 worker.production.sticky_session=True worker.production.method=S worker.lbbasic.type=ajp13 worker.lbbasic.connect_timeout=1 worker.lbbasic.recovery_options=7 worker.lbbasic.socket_keepalive=1 worker.lbbasic.socket_timeout=60 worker.lbbasic.ping_mode=CI
RE: jk Status not showing errors
I do not know if this is relevant or not, but I have just installed the latest version of mod_jk and the jkstatus is very much better than it used to be. I had the same issue with loadbalancers not showing when they are offline or broken. With the latest version, jksataus has the possibility to auto refresh itself. This now shouws when load balancers go down without a request being send to it. It is pretty dynamic as well. I ran several tests where I took one of the balancers down, and left jkstatus refreshing every 10 seconds and that told me that the worker was in error. It also shows you that the work is OK - IDLE when the worker is not being used but is good. As soon as it receives a request the status then changes to OK. Hope this helps. Kind regards / Met vriendelijke groet, Lawrence Lamprecht Application Content Manager QUADREM Netherlands B.V. Kabelweg 61, 1014 BA Amsterdam Post Office Box 20672, 1001 NR Amsterdam Office: +31 20 880 41 16 Mobile: +31 6 13 14 26 31 Fax: +31 20 880 41 02 Read our blog: Intelligent Supply Management - Your advantage -Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Saturday, May 30, 2009 2:46 PM To: Tomcat Users List Subject: Re: jk Status not showing errors On 29.05.2009 22:50, Matthew Laird wrote: Good afternoon, I've been trying to get the jkstatus component of mod_jk running, and I'm not quite sure what I'm doing wrong in trying to have it report dead Tomcat instances. I have two tomcat instances setup in a load balancer, as a test I've taken down one of them. However the jkstatus screen still shows both of them as OK. I'm not sure what I'm missing from my workers.properties file to make it test the Tomcat and report a failed instance, so I can set Nagios to monitor this page and report problems. My workers.properties is: worker.list=production,development,old,jkstatus worker.production.type=lb worker.production.balance_workers=production1,production2 worker.production.sticky_session=True worker.production.method=S worker.lbbasic.type=ajp13 worker.lbbasic.connect_timeout=1 worker.lbbasic.recovery_options=7 worker.lbbasic.socket_keepalive=1 worker.lbbasic.socket_timeout=60 worker.production1.reference=worker.lbbasic worker.production1.port=8009 worker.production1.host=localhost #worker.production1.redirect=production2 worker.production2.reference=worker.lbbasic worker.production2.port=8012 worker.production2.host=localhost #worker.production2.activation=disabled worker.development.port=8010 worker.development.host=localhost worker.development.type=ajp13 worker.old.port=8011 worker.old.host=localhost worker.old.type=ajp13 worker.jkstatus.type=status Any advice on extra options to make jkstatus check and report when one of the Tomcat instances isn't responding would be appreciated. I assume, that the actual error detection works and you are really only asking about display in status worker. I also assume your are using a recent mod_jk. Nevertheless do yourself a favor and look at the Timeouts documentation page to improve your configuration. Until recently, only workers used via a load balancing worker had good manageability with jkstatus. Very recently also pure AJP workers without any load balancer got more useful information in their display. So let's talk about your worker production. Whenever a request comes in the lb first checks whether it already carries a session for one of the nodes 1 or 2, or whether the request can be freely balanced. The status of a worker (node) in jkstatus can only change, if a request is been sent to the worker. So if all your requests belong say to node 2, you'll never notice anything is wrong with 1. But if 1 is broken, and a request for one comes in, or a request that is freely balanceable and the lb decides to send it to 1, then JK will detect the problem and display it. The display will switch from OK to ERR. If you want to parse the info, do not choose the html format, instead choose a different output format, like XML or the properties format (line oriented). Regards, Rainer - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
mod_jk status code and loadbalancer monitoring
I found a really old entry from way back in 2005. Reading through a trail of messages under the heading Adding working dynamically with mod_jk status What I would very much appreciate is two things. I would like to know where I can get some clear documents on the status codes for mod_jk. The status codes appear in the mod_jk.log file in the following format. [Wed May 20 06:27:47 2009] [21343:33088] [debug] jk_handler::mod_jk.c (1969): Service finished with status=200 for worker=loadbalancer1 I have also seen other entries for status=302 etc. I would like to understand what these status code mean or represent. The second thing that I would like to see is, do you have any information on monitoring the workers in the mod_jk config. I know about the jkstatus function, but that is not very dynamic. It only really reports errors if the wroker is called. I had an issue where I took down on of the two workers in the loadbalancer config, I saw the error that the first worker was no longer available, but as soon as the second worker took over the work, then the error was cleared and nothing more was reported. I have had an idea, but seeing as I am not a programmer, I have no idea how to achieve this, but I hope that someone else has also had the same bright spark. I was hoping that there was some application that did something similar to what webalizer does with the access.log file for the apache web server that could analyze the mod_jk.log file and dynamically report the status of the workers as things happen. Something like a dashboard to show what state the workers are in at any given time. When the log file is rolling in debug mode, there is very much information that could be used to report back on the worker status. I hope that someone might have some good ideas. Somehting that I have also thought about is possibly upgrading the mod_jk version that I am running at the mo. I have the following setup. Server version: Apache/2.0.52 Server built: Jun 29 2007 05:08:11 JK Version: 1.2.15 I get really confused with the different versions and compatibility, so could I keep the current Apache version and only upgrade mod_jk, Where would I find instauction on how to go about doing this. Thanks in advnce Lawrence Lamprecht
RE: mod_jk status code and loadbalancer monitoring
Rainer, Thanks for great advice, unfortunately I am not a programmer and have some issues when I try to compile the updated mod_jk. What I have tried is to compile mod_jk under Centos. Below are my proceedures. My config. [r...@localhost native]# head -n1 /etc/issue CentOS release 5.2 (Final) [r...@localhost native]# httpd -v Server version: Apache/2.2.3 Server built: Nov 12 2008 10:41:27 I started as instructed with ./buildconf.sh ./configure --with-apxs=/usr/sbin/apxs Make As far as I can see there were no error, I then copied the mod_jk.so to the apache modukes dir. The concern that I have was the location of the new file. /usr/src/tomcat-connectors-1.2.28-src/native/apache-2.0 My apache install is /etc/httpd/ The version of apache I am running is 2.2.3 and the one that was installed for is 2.0? Other than that I am completely lost. Thanks again for your help and advice. Kind regards Lawrence Lamprecht Read our blog: Intelligent Supply Management - Your advantage -Original Message- From: Rainer Jung [mailto:rainer.j...@kippdata.de] Sent: Friday, May 22, 2009 3:10 PM To: Tomcat Users List Subject: Re: mod_jk status code and loadbalancer monitoring Hi Lawrence, I will answer in a slightly different order below. On 22.05.2009 13:58, Lawrence Lamprecht wrote: I found a really old entry from way back in 2005. Reading through a trail of messages under the heading Adding working dynamically with mod_jk status What I would very much appreciate is two things. I would like to know where I can get some clear documents on the status codes for mod_jk. The status codes appear in the mod_jk.log file in the following format. [Wed May 20 06:27:47 2009] [21343:33088] [debug] jk_handler::mod_jk.c (1969): Service finished with status=200 for worker=loadbalancer1 I have also seen other entries for status=302 etc. I would like to understand what these status code mean or represent. The second thing that I would like to see is, do you have any information on monitoring the workers in the mod_jk config. I know about the jkstatus function, but that is not very dynamic. It only really reports errors if the wroker is called. I had an issue where I took down on of the two workers in the loadbalancer config, I saw the error that the first worker was no longer available, but as soon as the second worker took over the work, then the error was cleared and nothing more was reported. I have had an idea, but seeing as I am not a programmer, I have no idea how to achieve this, but I hope that someone else has also had the same bright spark. I was hoping that there was some application that did something similar to what webalizer does with the access.log file for the apache web server that could analyze the mod_jk.log file and dynamically report the status of the workers as things happen. Something like a dashboard to show what state the workers are in at any given time. When the log file is rolling in debug mode, there is very much information that could be used to report back on the worker status. I hope that someone might have some good ideas. Somehting that I have also thought about is possibly upgrading the mod_jk version that I am running at the mo. I have the following setup. Server version: Apache/2.0.52 Server built: Jun 29 2007 05:08:11 JK Version: 1.2.15 I get really confused with the different versions and compatibility, so could I keep the current Apache version and only upgrade mod_jk, Where would I find instauction on how to go about doing this. Thanks in advnce Lawrence Lamprecht The status worker was added to mod_jk around the time of your version 1.2.15. It improved *a lot* since then. Actually during the last releases it was the focus of most of the development. So do yourself a favour and update mod_jk. Recent is 1.2.28. Compatibility: mod_jk works for Apache httpd 1.3, 2.0 and 2.2. You need to either compile it against your web server (preferred way) or fetch a binary for your httpd version somewhere, e.g. in our download area. You cannot use a mod_jk for 2.2 to run in 2.0 or vice versa, but the minor digits behind usually do not matter. Compiling is easy (depending a bit on your OS), so you best go that way. We deprecated some configuration attributes long ago, so after updateing have a look at the mod_jk log during startup, if it contains info about not supported or deprecated attributes. The docs contain info about how to replace those. There are only few such attributes. We didn't deprecate anything during the last 10 releases, but chances are, that you should change a few configuration attributes you used with 1.2.15. There is one caveat: JkMount not isn't inherited from the global server to any virtual host or between virtual hosts. If you have virtual hosts, put your JkMount into those or use JkMountCopy. See docs. Now about status codes: The code in the log line you cited is the HTTP status code we
