changing location of conf/Catalina
Hi, I want to lock down the core Tomcat installation by making it read-only (and updateable only through a SCM). I've figured out how to relocate temp, work, logs, webapps directories, all of which get modified as part of Tomcat's standard operation. The last directory left inside the core that gets modified at runtime is conf/Catalina and I can't find a way to relocate it elsewhere. Is this even possible? Thanks Dmitry - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: jk-to-tomcat multiple retries
We were finally allowed to upgrade to Tomcat 5.5.27 and that seemed to have done away with the symptoms (I'm reluctant so say that upgrading fixed the problem, since I don't even know what it was in the first place ;-) Thanks for the help, everyone. d. The chunk length message seems pretty weird. Looks like a protocol corruption. Those indicate, that you should really try a TC update. Concerining your restriction can't update before any other options are exhausted: there will never be any other options exhausted. But after some options are taken, the rest get more and more expensive, risky and with a low chance of success. To me this look likes some weird error condition in Tomcat has hit an obscure bug in JK whereby it doesn't clear the response buffer between retries. Has anyone encountered this issue before or is just willing to land a helping hand in troubleshooting? Not encountered this before, and I think noone reported a similar observation. Concerning retries: Could you provide your full configuration (e.g. retries for an ajp13 worker is something very different from retries of a load balancer worker). Regards, Rainer - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
jk-to-tomcat multiple retries
Hi, We have the strangest problem started happening to us a few weeks ago (after several years of running pretty much the same configuration). 1. The problem is only happening in the production environment. We cannot reproduce it on staging, which as far as we can tell is configured identically to production. 2. The problem seems to be tied to the traffic volume and possible pattern (Hence probably why we cannot reproduce it on staging). 3. Our configuration: W2K server running IIS 6 and JK 1.2.27, Tomcat v. 5.5.12 running on the same box. The problem is as follows: some requests would result in multiple copies of the first buffer-full of expected data, ended either by a 503 error page data, a 502 error page data, or the rest of the proper page. The number of copies is directly related to the JK's number of retries setting. With the number of retries initially being set to 10, the maximum number of repeated copies in the response was 20. When we set the number of retries to 2, invalid replies contain only a single buffer-full of the page's proper data followed by an error page data. On the Tomcat side, these multiple copies show up as multiple request entries in the access log, while there is only one corresponding request entry in the IIS log. After a fresh restart of Tomcat, it takes a little while for this problem to start manifesting itself. With time, it is starting to affect increasingly more requests until finally Tomcat gets entirely locked up. At that point Tomcat needs to be restarted lather, rinse, repeat... Here's what a sample of error messages in JK's log looks like: [2760:2476] [error] jk_isapi_plugin.c (1199): WriteClient failed with 10053 (0x2745) [2760:4020] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:4020] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:1104] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:1104] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:1104] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:1104] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:1104] [error] jk_lb_worker.c (1432): All tomcat instances failed, no more workers left [2760:1104] [error] jk_isapi_plugin.c (2199): service() failed with http error 503 [2760:3876] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:3876] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:3232] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:3232] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:3232] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. [2760:3232] [error] jk_ajp_common.c (2426): (default_1) connecting to tomcat failed. [2760:3232] [error] jk_lb_worker.c (1432): All tomcat instances failed, no more workers left [2760:3232] [error] jk_isapi_plugin.c (2199): service() failed with http error 503 The Chunk length messages have been in our logs forever. Yesterday I temporarily changed JK Tomcat configuration matching the packet sizes. The chunk errors went away, but the problem seemed to persist, so I put everything back the way it was. To me this look likes some weird error condition in Tomcat has hit an obscure bug in JK whereby it doesn't clear the response buffer between retries. Has anyone encountered this issue before or is just willing to land a helping hand in troubleshooting? Thanks Dmitry - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: jk-to-tomcat multiple retries
Upgrade that Tomcat, it is *years* old - 5.5.27 is the latest. See if it still does it then. Unfortunately, the pesky reality is that I would not be permitted to do such an upgrade to our entire infrastructure until I can show that all other options have been exhausted. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: jk-to-tomcat multiple retries
If you suspect a bug in Tomcat, wouldn't upgrading to a version of Tomcat where that bug had potentially been fixed be a good strategy? True, but the problem is that I don't have enough data to make a decision. For all I know, there might be something really bad in our code that's triggering this weirdness, something that can be fixed with a 5 minute change. I would like figure this out before before just reinstalling Windows :) Hence, I'm asking here for help diagnosing the problem, but if everyone else is as stumped as I am then, sure, the next (reluctant) step would be to upgrade Tomcat in hope that it will go away (but as we all know, hope is not a strategy :-) Is there a way for you to re-play activity from your production logs against a staging server to generate the proper load and/or usage profile? We've tried exactly that with JMeter, but the problem didn't surface. D. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
jk's Chunk length too large message
Hi, Is this the right list for asking this question? We get a lot of errors in the jk log that look lilke the following: [Thu Apr 30 14:36:58.640 2009] [5184:5144] [error] jk_ajp_common.c (1726): Chunk length too large. Length of AJP message is 8188, chunk length is 8192. What do these mean and how can I fix this condition? Thanks Dmitry - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: mod_jk and Host matches server name
Just want to document the solution for folks facing the same problem in the future. Actually, it turned out to be quite nasty, in part because I wasn't paying much attention to the logs which were giving warnings that the AJP port was already bound. Being used to Tomcat failing to start when it hits a used port, I glazed over these warnings. Apparently, Tomcat will only fail to start when either the shutdown port or the http connector port are used, but will still continue with a warning if it can't bind a port for an AJP connector. The problem was caused by another application, VMWare's web admin console, using port 8009 for it's own needs. The problem was compounded by the fact that VMWare installed an AJP listener on that port. That's why it was responding to Apache's inquiries, but with an error message. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
mod_jk and Host matches server name
I've seen many references to this problem, but since the message is an umbrella to many possible problems, nothing of what I've read so far offers a solution for my particular symptoms. In the back I've got Tomcat 5.5.23 with default configuration: 1. default host is localhost 2. HTTP connector on 8080 3. AJP connector on 8009 After reading a message, I also added an alias to Host just to be sure. the application is deployed from a war into the default context with ROOT.xml descriptor On the front, I have Apache 2.2.4 with mod_jk 1.2.26 built from source (I needed a 64 bit version that would work with apache 2.2.4) Now, here are the symptoms: - Connecting to tomcat directly on the localhost interface: gets content - Connecting to tomcat directly on eth0:gets content - Connecting via mod_jk with worker.host=localhost: 400 No Host matches server name - Connecting via mod_jk with worker.host=10.10.1.12:400 No Host matches server name (The details of these tests are below) Any ideas what's going on here? Thanks Dmitry - Connecting directly to tomcat on the localhost interface: $ telnet localhost 8080 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'. GET / HTTP/1.1 Host: jira1 HTTP/1.1 302 Moved Temporarily Server: Apache-Coyote/1.1 Set-Cookie: JSESSIONID=365B38ABA34CBC8A1B3B9B456704517B; Path=/ Location: http://jira1/secure/Dashboard.jspa - - Connecting to tomcat on the eth0 interface: $ telnet jira1 8080 Trying 10.10.1.12... Connected to europe.travelsecure.local (10.10.1.12). Escape character is '^]'. GET / HTTP/1.1 Host: jira1 HTTP/1.1 302 Moved Temporarily Server: Apache-Coyote/1.1 Set-Cookie: JSESSIONID=BE2BE7AE2BFDF59EE393D6D76D8DF278; Path=/ Location: http://jira1/secure/Dashboard.jspa Content-Type: text/html;charset=UTF-8 Content-Length: 0 Date: Wed, 09 Jan 2008 17:42:09 GMT - - Connecting via mod_jk through jira1 host interface with the worker connecting on eth0 worker.worker_normal.type=ajp13 # worker.worker_normal.host=europe# worker.worker_normal.port=8009 # [EMAIL PROTECTED] Jira]$ telnet jira1 80 Trying 10.10.1.12... Connected to europe.travelsecure.local (10.10.1.12). Escape character is '^]'. GET / HTTP/1.1 Host: jira1 HTTP/1.1 400 No Host matches server name jira1 Date: Wed, 09 Jan 2008 17:48:08 GMT Server: Apache/2.2.3 (CentOS) Connection: close Transfer-Encoding: chunked Content-Type: text/plain; charset=UTF-8 from mod_jk.log: [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] init_ws_service::mod_jk.c (888): Service protocol=HTTP/1.1 method=GET host=(null) addr=10.10.1.12 name=jira1 port=80 auth=(null) user=(null) laddr=10.10.1.12 raddr=10.10.1.12 uri=/ [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_get_endpoint::jk_ajp_common.c (2579): acquired connection pool slot=0 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_marshal_into_msgb::jk_ajp_common.c (553): ajp marshaling done [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_service::jk_ajp_common.c (2050): processing worker_normal with 2 retries [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_send_request::jk_ajp_common.c (1352): (worker_normal) all endpoints are disconnected, detected by connect check (0), cping (0), send (0) [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] jk_open_socket::jk_connect.c (448): socket TCP_NODELAY set to On [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] jk_open_socket::jk_connect.c (548): trying to connect socket 23 to 10.10.1.12:8009 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] jk_open_socket::jk_connect.c (574): socket 23 connected to 10.10.1.12:8009 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_connect_to_endpoint::jk_ajp_common.c (878): Connected socket 23 to (10.10.1.12:8009) [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): sending to ajp13 pos=4 len=66 max=8192 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 12 34 00 3E 02 02 00 08 48 54 54 50 2F 31 2E 31 - .4.HTTP/1.1 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 001000 00 01 2F 00 00 0A 31 30 2E 31 30 2E 31 2E 31 - .../...10.10.1.1 [Wed Jan 09 09:48:12 2008] [14014:2934367456] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 002032 00 FF FF 00 05 6A 69 72 61 31 00 00 50 00 00 -
debugging connectors
Hi, I can't find any definitive documentation on this. Does the Connector element in Tomcat 5.5. support the debug property (I've seen examples of this in older Tomcats)? If so, where will the output go? Thanks Dmitry - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: clustering iis with JK
Hi Rainer, It's the item #3 on your Simple Setup list that I'm basically asking about (I already have #1 set up and working, and parts of #2). Here's where my problem with #3 lies: Let's say we have two load-balanced (clustered) web/IIS servers: W1 and W2, each configured with sticky forwarding by isapi_redirectors JK1 JK2 to Tomcat servers T1 T2. Let's follow this scenario 1. Request (R1) comes to the web cluster (C) and gets dispatched to W1. R1 is a new request, so it's gets arbitrarily sent to T2, where it's assigned a new session (S1) 2. Request R2 from session S1 comes to C. Let's say it gets dispatched to W1 again (C wouldn't know anything about S1). Since JK1 knows about S1, R2(S1) will be forwarded to T2, which started S1 --- everything is fine. 3. Request R3 from session S1 comes to C. And since C doesn't know anything about S1, R3(S1) is load-balanced to W2. JK2 at this point doesn't know anything about S1. To it, it's a new session, so it may forward it either to T1 or T2. If R3(S1) gets sent to T2, we are good. If not - our app is in trouble. I'm new to all this, so I might be missing something basic. I do realize that Tomcat session replication (your more complex setup) makes this problem moot. But, let's say, I do not take the session relication route, can I still make sure that #3 from above does not happen? Thanks Dmitry On Nov 28, 2007 3:02 AM, Rainer Jung [EMAIL PROTECTED] wrote: Dmitry Beransky wrote: Hi, Is it possible to implement the following setup with JK/isapi_redirect? 1. Two clustered IIS instances 2. Two load-balanced Tomcat instances 3. Each IIS uses JK to forward requests to two load-balanced Tomcat instances I know how to do each individual item in isolation, but I can't figure, once I put all three together, how to ensure that requests belonging to the same session are consistently served to the appropriate Tomcat instance. Any pointers? For IIS clustering I am considering NLB. Since want IIS clustering for mostly for reliability rather than scalability, an easy way out would be to do a fail over setup, but this is plan B. To isolate the layers: - Load-balancing the web servers (IIS) - Maybe stickyness already in the web layer, mainly in case you use SSL - Load-balancing between IIS layer and the Tomcat layer using the isapi redirector, including stickyness - Maybe session replication between the Tomcat instances to further increase transparency of nore failures Simple setup would be: - No session replication between Tomcat nodes (no Tomcat-Cluster). In case a node fails, the users with sessions on the nodes have to login again. OK, if sessions are cheap, i.e. not much work lost, not much information in the session, and failure rate is low (application, hardware, network relatively stable). - Combined with sticky forwarding by the isapi redirector (uses URL encoded sessions or standard Java session cookie JSESSIONID combined with the jvmRoute setting in server.xml of the Tomcat backends; TC adds the jvmRoute to the session id, and isapi redirector sees this tag in the URL or cookie and maps it to the correct backend). Works very robust. - Load-Balancing or high availability in the IIS layer would still be your job. Stickyness demand on the IIS layer itself depends on the fact, if the IIS layer is stateless (should be, apart from the SSL case, were you want to have a relatively good stickyness; don't need 100%, but the less sticky the LB to the IIS is, the more SSL handshakes you get). More complex setup: - Add session replication to the TC backends. Most likely nevertheles you want to keep stickyness with the isapi redirector, to reduce dependency on the rpelication during the time you actually didn't have a node failure. Regards, Rainer Thanks Dmitry - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: clustering iis with JK
Does that answer your question? Oh! I think it does. I was assuming that isapi_redirect maintained an internal map of sessions and tomcat nodes, but if the jvmRoute is in the session id, then it shouldn't matter which IIS/isapi is serving the request, it will still go to the correct instance of tomcat as long as it's available. - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
clustering iis with JK
Hi, Is it possible to implement the following setup with JK/isapi_redirect? 1. Two clustered IIS instances 2. Two load-balanced Tomcat instances 3. Each IIS uses JK to forward requests to two load-balanced Tomcat instances I know how to do each individual item in isolation, but I can't figure, once I put all three together, how to ensure that requests belonging to the same session are consistently served to the appropriate Tomcat instance. Any pointers? For IIS clustering I am considering NLB. Since want IIS clustering for mostly for reliability rather than scalability, an easy way out would be to do a fail over setup, but this is plan B. Thanks Dmitry - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Wr Rd in the JK status report
Hi, I'm having problems understanding the legend text for Wr Rd columns of the JK status report. What exactly do they mean? Here's a concrete example: Name Type Host Addr Act Stat D F M V Acc Err CE Wr Rd Busy Max Route RR Cd Rs static_1 ajp13 localhost:8012 127.0.0.1:8012 ACT OK 0 1 1 61 9613 0 29 8.3M 48M 0 8 static_1 0 So Wr=8.3M and Rd=48M. This worker handles all static resources of the website: images, scripts, styles, etc. If 48M is the amount of data sent to the client, what is 4.8M? Thanks Dmitry - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Incorrect function error w/isapi_redirect 1.2.25
Ay, caramba. I don't even have access to VS... do I need VS to compile or can I use a GNU compiler? On Nov 13, 2007 5:25 PM, Martin Gainty [EMAIL PROTECTED] wrote: This is the classic problem of Microsoft products not working with Microsoft Products If a dll isnt working you will need to get the source and build it for your specific OS http://tomcat.apache.org/connectors-doc/reference/iis.html - To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
alternative tag pool implementation
Hi, I've noticed that there are two implementations of the tag pool: TagHandlerPool and PerThreadTagHandlerPool. I haven't been able to figure out how to tell Jasper to use the per-thread implementation. Any pointers? Thanks Dmitry
