Re: Tomcat Connection Pool Problems with XtraBackup
On Sun, Oct 12, 2014 at 5:56 AM, Stefan Mayr ste...@mayr-stefan.de wrote: Hi John, Am 11.10.2014 23:30, schrieb John Smith: I'm trying to workout a managed backup scheme on a MySQL production database with XtraBackup. According to our DBA, XtraBackup doesn't lock the database, but issues a series of SHOW TABLE STATUS commands and then works on the file system level. Still, just as XtraBackup runs my logs start to blow up with connection pool errors: Error getting database connection:[http-nio-8080-exec-5] Timeout: Pool empty. Unable to fetch a connection in 10 seconds, none available[size:100; busy:100; idle:0; lastwait:1]. There's some suggestion that turning off 'innodb-stats-on-metadata' might help so we're trying that. ... Anything else seem like it could be tweaked in relation to XtraBackup? TIA, John XtraBackup can operate without locks if your database contains only InnoDB tables. For other storage engines like MyISAM it still uses locks. A problem we seen is high IO load during XtraBackup. Your DBA could check iostats while backup is in progress. If this makes your database server unresponsive he could try to throttle the backup: http://www.percona.com/doc/percona-xtrabackup/2.2/ innobackupex/throttling_ibk.html -Stefan - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Stefan, Thanks, that seems to be the problem. We have all InnoDB, and the backup runs without locks, but the it runs under fairly heavy IO load. It seems like turning 'innodb-stats-on-metadata' to OFF lessened the strain that the SHOW STATUS queries from XtraBackup caused. Bumping up maxActive allowed the pool to handle the slowdown of application queries during the backup. This has worked so far but if it repeats I'll look into the throttling. That was my first sense, that it wasn't locking, just pushing too hard on the DB. Very helpful. Thanks. -John
Re: Windows Tomcat install folder
I don't know about the recommended way, but personally I hate the SPACE in Program Files, so I definitely prefer the 2nd option. Same, I don't even bother with the second level directory. I just install to c:\apache-tomcat...
Tomcat Connection Pool Problems with XtraBackup
I'm trying to workout a managed backup scheme on a MySQL production database with XtraBackup. According to our DBA, XtraBackup doesn't lock the database, but issues a series of SHOW TABLE STATUS commands and then works on the file system level. Still, just as XtraBackup runs my logs start to blow up with connection pool errors: Error getting database connection:[http-nio-8080-exec-5] Timeout: Pool empty. Unable to fetch a connection in 10 seconds, none available[size:100; busy:100; idle:0; lastwait:1]. There's some suggestion that turning off 'innodb-stats-on-metadata' might help so we're trying that. At the same time I want to confirm my connection pool settings are good: Resource name=jdbc/mydatabase_master auth=Container type=javax.sql.DataSource factory=org.apache.tomcat.jdbc.pool.DataSourceFactory testWhileIdle=true testOnBorrow=true testOnReturn=false validationQuery=SELECT 1 validationInterval=3 timeBetweenEvictionRunsMillis=3 maxActive=100 minIdle=10 maxWait=1 initialSize=10 removeAbandonedTimeout=60 removeAbandoned=true minEvictableIdleTimeMillis=3 jdbcInterceptors=org.apache.tomcat.jdbc.pool.interceptor.ConnectionState; org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer username=db_user password=mypassword driverClassName=com.mysql.jdbc.Driver url=jdbc:mysql://my.ip:3306/my_database / For everything besides the backup, this configuration works very well. For most of our traffic, the connection pool size hovers around the initialSize. It seems like setting the maxActive higher will just spawn new connections in the pool that are hanging. Anything strike you as...too tight for lack of a better description, that would force the pool to continuously add connections when the backup runs? Anything else seem like it could be tweaked in relation to XtraBackup? TIA, John
Re: Daily catalina.out has thousands of NPEs
You haven't really provided any information, here. Please post full stack traces. You might want to do something like catch NPE around the area(s) where they are being thrown, and log information about the requests that are causing them. It may be something like a web search crawler requesting the XML with parameters you don't expect, etc. Chris, Sorry for the delayed reply. The thing is, there are no stack traces in the catalina logs. Just the report of the NPE's as cut and pasted above, over and over again. So I can't get to where the NPE is even being thrown. Again, the errors are not generated at the application level code, or at least they're not 500 errors or they would be written to the logback logs, instead of catalina.out. I have an a really high amount of traffic (500,000 visits a day) so detailed catalina logs become so massive they become untenable. I thinned out logging.properties to simply: ___ handlers = 1catalina.org.apache.juli.FileHandler 1catalina.org.apache.juli.FileHandler.level = SEVERE 1catalina.org.apache.juli.FileHandler.directory = ${catalina.base}/logs 1catalina.org.apache.juli.FileHandler.prefix = catalina. ___ Do you know if this configuration removing the full trace? Best, John
Daily catalina.out has thousands of NPEs
TC 7.0.54, RHEL 6, JDK 1.7.0_60. I have two RH servers with one instance of TC on each, set up for clustering. There's a HWLB routing 80 to 8080 with 5 min sticky sessions. My daily catalina logs are set up to just show SEVERE errors, every day they average about *30mb* per server with the same set of errors, over and over: Aug 21, 2014 12:00:04 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet [GetLevelServlet] in context with path [] threw exception java.lang.NullPointerException Aug 21, 2014 12:00:04 AM org.apache.coyote.http11.AbstractHttp11Processor process SEVERE: Error processing request java.lang.NullPointerException Aug 21, 2014 12:00:04 AM org.apache.coyote.http11.AbstractHttp11Processor endRequest SEVERE: Error finishing response java.lang.NullPointerException Aug 21, 2014 12:00:13 AM org.apache.catalina.core.StandardWrapperValve invoke SEVERE: Servlet.service() for servlet [GetLevelServlet] in context with path [] threw exception java.lang.NullPointerException Aug 21, 2014 12:00:13 AM org.apache.coyote.http11.AbstractHttp11Processor process SEVERE: Error processing request java.lang.NullPointerException Aug 21, 2014 12:00:13 AM org.apache.coyote.http11.AbstractHttp11Processor endRequest SEVERE: Error finishing response java.lang.NullPointerException GetLevelServlet is the workhorse of the site, providing XML to a SWF through a simple method: protected void writeXML(HttpServletResponse res, String xml) throws IOException { if (xml == null || xml.isEmpty()) return; res.setContentType(text/xml); PrintWriter out = res.getWriter(); out.write(xml); out.close(); } About 99% of the site usage is through that servlet. I'm catching all 500 errors and logging them with Logback to a different log file, and I'm not seeing the NPE's generated on the application level (i.e., com.mysite.*) I'm using the NIO connector. !-- Using the non blocking NIO connector -- Connector port=8080 protocol=org.apache.coyote.http11.Http11NioProtocol connectionTimeout=2 redirectPort=443 maxConnections=7500 maxThreads=400 / There's a sizable download (8mg) for the SWF (it's a game site). So useSendFile being on (the default) works incredibly well. The errors in catalina aren't presenting on the site as much as they are in the logs, that is, the site seems to be behaving reasonably well despite the enormous amount of errors in the logs. Anyone know what might be causing it? Best, John
Re: java.lang.IllegalArgumentException at java.nio.Buffer.limit
How would you use a proxy to do that? On Mon, Aug 18, 2014 at 3:23 PM, Yogesh Rao yog...@gmail.com wrote: How about tracing the http request and response for the call made? Tip : adding a proxy wud help here to log the entire request. Regards, -Yogesh On Monday, August 18, 2014, John Smith tomcat.ran...@gmail.com wrote: On Fri, Aug 8, 2014 at 9:10 AM, John Smith tomcat.ran...@gmail.com javascript:; wrote: On Thu, Aug 7, 2014 at 5:28 PM, Filip Hanik fi...@hanik.com javascript:; wrote: if you could capture the XML that you are trying to write, we can put it into a test case and reproduce. Filip I'll try that -- I should be able to catch the IllegalArgumentException there when it happens. Sorry for the delayed reply. I was out last week. I caught the error and wrote the XML to the logs. The thing is, 1) It's valid XML, and 2) the exact same XML doesn't always cause the exception to be thrown. I don't believe the issue is the XML String itself. For example the XML I caught was from memcached and its expiration time is very long. If it were just the XML String it would be happening each time, but it's intermittent.
Re: java.lang.IllegalArgumentException at java.nio.Buffer.limit
On Fri, Aug 8, 2014 at 9:10 AM, John Smith tomcat.ran...@gmail.com wrote: On Thu, Aug 7, 2014 at 5:28 PM, Filip Hanik fi...@hanik.com wrote: if you could capture the XML that you are trying to write, we can put it into a test case and reproduce. Filip I'll try that -- I should be able to catch the IllegalArgumentException there when it happens. Sorry for the delayed reply. I was out last week. I caught the error and wrote the XML to the logs. The thing is, 1) It's valid XML, and 2) the exact same XML doesn't always cause the exception to be thrown. I don't believe the issue is the XML String itself. For example the XML I caught was from memcached and its expiration time is very long. If it were just the XML String it would be happening each time, but it's intermittent.
Re: java.lang.IllegalArgumentException at java.nio.Buffer.limit
What's on line 182 of AbstractServlet.java? -Terence Bandoian It's the out.write(xml); line.
java.lang.IllegalArgumentException at java.nio.Buffer.limit
TC 7.0.54 / RHEL 6 / JDK 1.7.0_60 I'm getting a pretty consistent error in my logs that started showing up recently. I use logback and have a servlet catch all 500 errors and log them. The error seems to be associated with one servlet that writes XML output. Two changes I made recently were implement SSL for one subdirectory on the webapp, and removing redirects in IPTables to the HWLB. I can't really think of any other code level changes that might have caused the change in behavior. Please let me know if you need more information. Any thoughts? The writeXML method is: protected void writeXML(HttpServletResponse res, String xml) throws IOException { res.setContentType(text/xml); PrintWriter out = res.getWriter(); out.write(xml); out.close(); } The stacktrace almost always looks like this: _ 14:18:59.617 [http-nio-8080-exec-45] ERROR c.m.SiteExceptionHandlerServlet - Stacktrace was: java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:267) at org.apache.tomcat.util.buf.C2BConverter.convert(C2BConverter.java:85) at org.apache.catalina.connector.OutputBuffer.realWriteChars(OutputBuffer.java:481) at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:464) at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java:384) at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:554) at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java:174) at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java:184) at com.mysite.AbstractServlet.writeXML(AbstractServlet.java:182) at com.mysite.level.GetLevelServlet.getAllAsXML(GetLevelServlet.java:82) at com.mysite.level.GetLevelServlet.manageActions(GetLevelServlet.java:33) at com.mysite.AbstractServlet.doPost(AbstractServlet.java:52) at javax.servlet.http.HttpServlet.service(HttpServlet.java:646) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) ... __ Although there are variations like: Stacktrace was: java.lang.IllegalArgumentException at java.nio.Buffer.position(Buffer.java:236) at sun.nio.cs.ISO_8859_1$Encoder.encodeArrayLoop(ISO_8859_1.java:179) at sun.nio.cs.ISO_8859_1$Encoder.encodeLoop(ISO_8859_1.java:212) at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:561) at org.apache.tomcat.util.buf.C2BConverter.convert(C2BConverter.java:108) at org.apache.catalina.connector.OutputBuffer.realWriteChars(OutputBuffer.java:481) at org.apache.tomcat.util.buf.CharChunk.flushBuffer(CharChunk.java:464) at org.apache.tomcat.util.buf.CharChunk.append(CharChunk.java:384) at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:554) at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java:174) at org.apache.catalina.connector.CoyoteWriter.write(CoyoteWriter.java:184) at com.mysite.AbstractServlet.writeXML(AbstractServlet.java:182) ___
Re: Restricting SSL access within webapp
All, Thanks for the thoughtful advice and replies. To answer a few questions, belatedly, yes it would be an option to move the admin tools to another instance of TC, as Leo suggested -- in a way a better one, since it wouldn't need session replication, could exist on a single server since the traffic would be be trivial, and would be potentially more secure. I'll probably do this in the long term. If not that, then url-rewrites or a filter to bounce users out of https is another simpler option, as Chris suggested. Based on the information about SSL not being that expensive, I'll just leave it in for now, at the clients discretion, as Charles originally suggested. Our user base is probably not going to suddenly all jump on https, so I can watch and see if it affects performance. The area that mandatorily requires SSL is configured with a security constraint -- for the rest of the site, I'll leave it up to the user. Best, John
Re: JKS keystore password Encryption
You may find Wiki also useful: http://wiki.apache.org/tomcat/FAQ/Password -Ognjen Write your own datasource implementation which wraps your datasource and obscure your brains out (XOR http://en.wikipedia.org/wiki/XOR_cipher and ROT13 http://en.wikipedia.org/wiki/ROT13 are great candidates for this since their strength matches the protection you'll actually get) Haha. ROT13: Vg pna'g or penpxrq! Hmm, Interesting, ROT13 looks oddly like klingon.
SSL redirect problems
TC 7.0.54 / RHEL 6 I have two physical servers, each running an instance of TC. The servers are behind a hardware loadbalancer. IPTables is routing request on 80 to 8080. Tomcat runs under a non-root user. All good. I needed to protect an area of our webapp under SSL. Went ahead and installed the cert on each server. I can go directly to each server by IP under SSL and get the cert (with the expected IP doesn't match FQDN warning). But when I go through the loadbalancer I can't access anything under port 8443. I redirected 443 to 8443 on each TC server using IPTables, but still no luck. Is there anything I'm missing? I understand I can install the cert on the loadbalancer instead, or use httpd as a proxy, but I'd rather just leave it the way it is if there's any other option. TIA, John
Re: SSL redirect problems
TC 7.0.54 / RHEL 6 I have two physical servers, each running an instance of TC. The servers are behind a hardware loadbalancer. IPTables is routing request on 80 to 8080. This seems unnecessary. If you have a hardware load balancer in front of Tomcat, it is the only thing that would ever talk to Tomcat. Thus if you just configure it to go to port 8080 you don't need the iptables rule. I can't imagine it's hurting anything, but just thought I'd mention it. Not at all, it would seem like a better choice than an OS level redirect like iptables. Tomcat runs under a non-root user. All good. I needed to protect an area of our webapp under SSL. Went ahead and installed the cert on each server. I can go directly to each server by IP under SSL and get the cert (with the expected IP doesn't match FQDN warning). You probably want the SSL certificate installed on your hardware load balancer. End client's browsers are going to connect to the hardware load balancer, not Tomcat. Thus you'd want the certificate there so your end users can benefit from it. Ex: browser - HTTPS - load balancer - HTTP or HTTPS - Tomcat If you put an SSL certificate on your Tomcat servers, that would allow you to secure the connection between your load balancer and Tomcat. Depending on your network and security requirements this may or may not be necessary. I'd say most people don't do this because terminating SSL on the load balancer is sufficient. It just depends on your requirements though. Ok, that makes sense. I think just on the loadbalancer will work. In our configuration, unencrypted traffic between the LB and the servers is subject to minimal risk, and our security requirements aren't critical. But when I go through the loadbalancer I can't access anything under port 8443. I redirected 443 to 8443 on each TC server using IPTables, but still no luck. Is there anything I'm missing? The load balancer is almost certainly listening on port 80 and 443. To test, you'd want to connect to the load balancer on one of those ports. The load balancer would then connect to one of your backend nodes and proxy the request on your behalf. Your browser will not connect directly to the backend nodes (see my point above about not needing the iptables rule), unless you specifically point it to the ip address of one of the backend nodes. Sorry, I'm a bit unclear on this. What method of connecting would let me test? I think you'd want it on the load balancer. Possibly with additional certs on your backend nodes, if you want HTTPS communication between the load balancer and the Tomcat nodes. Dan Thanks so much for the detailed and quick reply. John
Re: SSL redirect problems
Not contradicting anything Daniel is saying, but maybe something to add, and maybe that's the missing part of the original puzzle : If Tomcat is expecting HTTPS requests on port 8443, then any re-direct or response that it is sending back is going to include that port number after the hostname. (even inside the pages, if you use absolute URL links there). So the browser who ultimately receives this, is going to try to talk to port 8443. But that will not work, if your front-end is expecting further requests on port 443, and blocks 8443. Unless in all your Tomcat responses, you arrange to replace any reference to port 8443, by 443, before they reach the browser again. Maybe using a browser plugin like HttpFox, LiveHttpHeaders or Fiddler2 would allow you to see more clearly what is going on there. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Well, that's the part that seems confusing. Left as default, I would have thought connecting through the LB on 8443 would have worked. Actually I'm still not clear on which part of the chain is having a problem. Originally, I had no iptable redirect - I just added it in the great tradition of programming - try everything and anything until it works. I don't care if the user has to have 8443 in the URL. Just to be clear, you are suggesting that then problem would be the iptables redirect?
Re: SSL redirect problems
On Fri, Aug 1, 2014 at 11:54 AM, Mark Thomas ma...@apache.org wrote: On 01/08/2014 16:30, Daniel Mikusa wrote: You probably want the SSL certificate installed on your hardware load balancer. End client's browsers are going to connect to the hardware load balancer, not Tomcat. Thus you'd want the certificate there so your end users can benefit from it. That depends on whether the load-balancer is operating at layer 4 or layer 7. Mark Mark, I have to check which layer it's operating at, but does that mean that, depending on the layer, the cert should *not* be on the LB?
Re: SSL redirect problems
No, I am not really going that far. I am suggesting that that may be the kind of thing that is happening, and that you may want to investigate with a browser plugin, that the requests/responses are really what you are expecting. Your initial explanation was a bit confusing and lacking in precise details, as to what the load balancer really does, where IPtables does what, and how your tomcats are configured (re Connectors, and possibly IPtables too). So we're all kind of guessing here, and just trying to give you some tips, to either simplify your setup, or to figure out better what is happening. Well, lets remove the IP tables. I know the certs work because as I said I can access them directly by going to either server on 8443 directly. The connectors are configured correctly. There's no security info in web.xml. The entire site should be available over SSL. Using Charles, with LB:8443 I get connection refused - without any other particularly useful info in the response.
Re: SSL redirect problems
As your testing keep this process in mind. If you encounter a problem just try to break down the flow from your browser to the server and back. If you look at the request at each hop through this process, you can often find where things went wrong. For example, did the request hit the LB? If not, maybe we have a firewall issue or ports are configured right. If so, did it hit one of the backend servers? If not, maybe there's a config issue in the lb. If it did, what response did it get? A 4xx / 5xx error, ok something went wrong on the backend, need to investigate the logs there for more details. Hope that helps to clarify. Dan Dan, It did. It was one of those cases where the simplest answer was assumed but not tested. The loadbalancer was not listening on 443 or 8443. I was able to have it redirect 443 to 8443 successfully. I also took your advice and redirected 80 to 8080 instead of using iptables. Thanks for your help. So many knowledgeable people on here. John
Re: SSL redirect problems
Is your LB configured to listen on 8443, or on 443? It won't pick up the port it's supposed to listen on from the TC instances; you have to specify it. Nailed it. Simplest solution, I didn't even consider it. Thanks, John
Re: SSL redirect problems
There is no response, since you are not even able to connect to that IP:port. If you are using the IP of the LB, then the LB is not accepting connections on port 8443. You won't get much further, unless you solve that first. But I thought that you wanted your users to access via port 443 ? Thanks, This was the problem. So simple I should have looked there first. Facepalms. I was able to redirect 443 to 8443 on the LB with success.
Restricting SSL access within webapp
In my webapp there's a directory '/admin' that's protected under SSL. Users are forced to use SSL via a security constraint in web.xml. It works great. As mentioned in the docs and other places, it would be good to prevent SSL everywhere else on the site, but I searched around and couldn't find anything that works.I tried adding another security constraint with transport guarantee set to NONE for url-pattern '/*' but it didn't prevent https access to the site as a whole. What's the correct way to selectively restrict https to only one area of a webapp? TIA, John
Re: SSL redirect problems
Thanks for letting us know what the issue was; many people never come back and tell us what fixed it. My pleasure. This list is awesome.
Re: SSL redirect problems
TLS is layer 5 so if the LB is operating at layer 4 it can't host the cert. Some LBs can operate at layer 5 so it will depend on your LB and/or its configuration. Mark I see. That's good to know. The LB is at 7.
Re: Restricting SSL access within webapp
On Fri, Aug 1, 2014 at 4:34 PM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: John Smith [mailto:tomcat.ran...@gmail.com] Subject: Restricting SSL access within webapp What's the correct way to selectively restrict https to only one area of a webapp? Why would you want to do that? Other than a few extra server CPU cycles, what's the harm in allowing SSL anywhere at the client's discretion? - Chuck From the docs: Also, while the SSL protocol was designed to be as efficient as securely possible, encryption/decryption is a computationally expensive process from a performance standpoint. It is not strictly necessary to run an entire web application over SSL, and indeed a developer can pick and choose which pages require a secure connection and which do not. For a reasonably busy site, it is customary to only run certain pages under SSL, namely those pages where sensitive information could possibly be exchanged. Unfortunately how to do this isn't explained. I might use a filter. Our site handles 500,000 visitors a day on two TC instances. Believe me, I need to consider performance costs.
Re: TC7 and SSL Questions
On Thu, Jul 24, 2014 at 6:24 PM, Ognjen Blagojevic ognjen.d.blagoje...@gmail.com wrote: John, On 24.7.2014 21:11, John Smith wrote: 1. Can I specify /admin/* as a security constraint url pattern so that only that directory runs under SSL? Yes, you can. 2. The NIO connector is accepted for JSSE, since I'm using it already, is there any point in not using it as my SSL connector? If /admin has low traffic, then I would say, there is no need to use anything else. For high traffic TLS/SSL applications you may want to do some performance measurements of different Tomcat connectors, simulating your traffic patterns. 3. Any known issues with routing 443 to 8443 in Iptables? I recommend using JSVC instead of iptables redirect. I had issues with redirect when used with virtual hosts. IPv6 (ip6tables) doesn't support redirect, either. 4. The admin tools share underlying classes with the rest of the web application, which is why it makes sense to have it just as a subdirectory in the same webapp. But would I be better off migrating the admin tools to their own webapp for the purposes of SSL? Yes, I think so. From the security standpoint, that is way better. It will be much easier to apply IP address filtering, move it to another port / server, to isolate admin and user privileges, and so on. -Ognjen - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Thanks for the info. Best, John
TC7 and SSL Questions
TC 7.0.54 / JDK 1.7.0_60 / RHEL 6 My webapp is the only one on my TC install. It's in webapps/ROOT. Iptables routes 80 to 8080 and I'm using the NIO connector. There are two physical servers with that same webapp, using session replication. Everything works great. There's a subdirectory /admin in the webapp that has some admin tools that we've been using behind our firewall and under BASIC authentication. I want to put just the /admin directory under SSL and have a user/hashed-pass in the database do the login and authentication instead of having them in tomcat-users.xml. Questions: 1. Can I specify /admin/* as a security constraint url pattern so that only that directory runs under SSL? 2. The NIO connector is accepted for JSSE, since I'm using it already, is there any point in not using it as my SSL connector? 3. Any known issues with routing 443 to 8443 in Iptables? 4. The admin tools share underlying classes with the rest of the web application, which is why it makes sense to have it just as a subdirectory in the same webapp. But would I be better off migrating the admin tools to their own webapp for the purposes of SSL? Apologies if I've missed any of this in the docs. Any additional info/advice appreciated. Thanks in Advance, John
Re: SSL on one subdirectory only.
On Tue, May 27, 2014 at 2:21 PM, Mark Thomas ma...@apache.org wrote: On 27/05/2014 17:31, John Smith wrote: Tomcat 7.0.42, RHEL6, JDK1.7.0_25, Standalone TC configuration. IPTABLES route port 80 to 8080 I've got a subdirectory like 'www.mysite.com/admin' that I want to put under FORM based authentication. That's clear enough, and I've got the java keytool cert working well enough on my dev box until I get one from a CA. Couple of questions: 1. Anyone familiar with any problems routing 443 to 8443 on *nix boxes for TC SSL certs? It's preferable to not have my end users needing port numbers. The cert doesn't care about the port, IIRC. Should be fine. 2. With the SSL connector enabled, https://* is globally respected on the entire webapp. Do I need to manually check the URL/protocol to deny or redirect https to http outside of '/admin'? Is there any built in TC mechanism or suggested best practice to handle this? or should I not care? Nothing to automatically handle https - http. Unless it causes an issue, I'd just leave it. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Mark, Thanks and appreciated, as always.
Re: SSL on one subdirectory only.
2. With the SSL connector enabled, https://* is globally respected on the entire webapp. Do I need to manually check the URL/protocol to deny or redirect https to http outside of '/admin'? Is there any built in TC mechanism or suggested best practice to handle this? or should I not care? We use two-factor authentification with SSL - but I think in your case this can be helpful too - not a big difference. Try look at this: http://wiki.metawerx.net/wiki/ForcingSSLForSectionsOfYourWebsite Arseny, thank you. I wasn't aware of the user-data-constraint and transport-guarantee elements. I'll give them a try.
SSL on one subdirectory only.
Tomcat 7.0.42, RHEL6, JDK1.7.0_25, Standalone TC configuration. IPTABLES route port 80 to 8080 I've got a subdirectory like 'www.mysite.com/admin' that I want to put under FORM based authentication. That's clear enough, and I've got the java keytool cert working well enough on my dev box until I get one from a CA. Couple of questions: 1. Anyone familiar with any problems routing 443 to 8443 on *nix boxes for TC SSL certs? It's preferable to not have my end users needing port numbers. The cert doesn't care about the port, IIRC. 2. With the SSL connector enabled, https://* is globally respected on the entire webapp. Do I need to manually check the URL/protocol to deny or redirect https to http outside of '/admin'? Is there any built in TC mechanism or suggested best practice to handle this? or should I not care? Best, John
Re: PooledConnection.getConnection - Tomcat JDBC Pool
If that is the case the tomcat jdbc pooling library handling the call incorrectly and its a bug. I'd be suspect of this. Are you actually using *org.apache.tomcat.jdbc.pool*? Since it's a Tomcat module it seems an odd choice to use outside of Tomcat. http://docs.oracle.com/javase/7/docs/api/javax/sql/PooledConnection.html *The connection pool manager, typically the application server, maintains a pool of PooledConnection objects. If there is a PooledConnection object available in the pool, the connection pool manager returns a Connection object that is a handle to that physical connection. If no PooledConnection object is available, the connection pool manager calls the ConnectionPoolDataSource method getPoolConnection to create a new physical connection. The JDBC driver implementing ConnectionPoolDataSource creates a new PooledConnection object and returns a handle to it. * My emphasis there. Just using the tomcat jdbc classes doesn't guarantee you have a pool. It does guarantee you'll get a connection, however it won't be pooled if there isn't one. Am I misunderstanding that outside of Tomcat means you're running Tomcat, but have some non-webapp application that's trying to use the JNDI datasource? Or are you not running Tomcat at all? Best, John
Re: Concurrency - Servlet created instances accessing static classes
If the method is thread-safe - no issue. If it isn't thread-safe then you have a problem. Mark Thanks Mark - Clearly and succinctly explained. Best, John
Re: Concurrency - Servlet created instances accessing static classes
First terminology problem: class X isn't instantiated here, an object of type class X is instantiated. As Matisse once said, exactitude is not truth. This sort of hair splitting isn't helpful. Say Class X is instantiated to a thousand programmers and they'll understand that it means an instance of Class X. Second terminology problem: classes don't make calls, threads do. Again, google class calls another class. Even the java docs use this terminology. The last sentence of your answer was helpful, but don't waste people's time being otherwise pedantic.
Re: Concurrency - Servlet created instances accessing static classes
You must have been fun to have as a student. Student is a vague term. Student of which grade, which subject? Student of a trade? Student of life? Unfortunately your lack of exactness means I can't understand your joke :) (I was an even worse employee) - John
Re: PooledConnection.getConnection - Tomcat JDBC Pool
On Tue, Mar 25, 2014 at 9:54 AM, Filip Hanik fi...@hanik.com wrote: Please open a bug, and we will get this taken care of. I do have one question,Aries library, on which call does it expect to return the connection to the pool? XAConnection.close() or XAConnection.getConnection().close(); ? Jonathan, Filip, If it is a bug then my answer is way off the mark. I'm sorry. If it's not a problem, could you explain, in this case, what acts as the connection pool manager when the Tomcat jdbc pool library is used outside of Tomcat? Best, John
Re: Can we increase the logging in localhost_access.log
We see 404 error in localhost_access so this the place from where we can dig into. Is there any way we can enhance the logging/information in localhost_access_log. Or then how can debug what happens between user requests and 404 response,? Outside of performance monitoring tools, you might try a custom 404 error that points to a servlet. Within that servlet you can capture information and log to a separate 404 log file (say, using LogBack). There are a few pages on the web that describe this process. Here's one: http://www.journaldev.com/1973/servlet-exception-and-error-handling-example-tutorial Best, John
Concurrency - Servlet created instances accessing static classes
I should know this, but I want to confirm with smarter people on the board. Assume the following: 1. Servlet receives an HTTP POST request. doPost(...) is called. 2. doPost(..) instantiates class X with each request 3. Class X calls a static method of class Y Assuming I have no synchronization in the method signature or body of Y, there is a danger of concurrency issues or thread safety issues. Many instances of X can call Y's static method at the same time, causing BAD THINGS to happen. True? I swear I used to know this but I've worked on so much stuff for this project in the last few months it simply fell out of my brain :). I'm also aware that the answer to this is often it depends but I'm looking for the general case answer. TIA John
Re: Effects of turning off sendFile in the NIO connector
MGwhen you enable sendfile support with request attr org.apache.tomcat.sendfile.support = true MGYou will need to set these 3 header attributes org.apache.tomcat.sendfile.filename: Canonical filename of the file which will be sent as a String org.apache.tomcat.sendfile.start: Start offset as a Long org.apache.tomcat.sendfile.end: End offset as a Long MGhtitps://tomcat.apache.org/tomcat-6.0-doc/aio.html MGCompression: MGset compression=on @ Connector MGhttps://tomcat.apache.org/tomcat-7.0-doc/config/http.html MGI did not read that TC cannot use sendfile with any compressed Stream? MGcan you show us the URL? MGThanks We also only really need compression on XML data, the site has minimal HTML, SWF's don't really benefit from gzip and some binary data we send back and forth is already compressed. I could manually implement compression on XML at the application level and within the SWF, if turning off sendFile will have negative consequences. Tomcat 7.0.42 RHEL6 ~4T outbound traffic/day Best, John Your first link refers to using sendFile for asynchronous writes from a servlet. Any servlet can instruct Tomcat to perform a sendfile call by setting the appropriate request attributes. Your answer is not accurate. I don't need to do anything explicitly with the headers. You should look at the documentation regarding the HTTP NIO Connector - which you already have as the second link in your reply. It discusses sendFile and compression, and how you cannot use both the NIO connector and compression if sendFile is on (on is the default).
Re: Effects of turning off sendFile in the NIO connector
John The consequences for disabling sendFile are extremely hard to quantify as there are so many variables. I would normally expect there to be more CPU load but how much more? No idea. It might be impossible to detect, it might leaver your CPUs pegged at 100%. The only way you will know for sure for your application is to test it with your application. Mark P.S. As you've probably figured out having been Martined, the best location for any response from Martin Gainty is /dev/null. I keep debating dropping him from the list as he causes far more harm than good. A topic for a separate thread I think. Mark, Thanks for the answer. As suggested, I'll test it and see what happens...kind of figured I'd have to do that :) Appreciate the heads-up on MG. I was wondering why the answer seemed off. Best, Alec
Effects of turning off sendFile in the NIO connector
What effect would setting useSendfile=false have on a web application using the NIO connector? I'm asking because I may want to use gzip compression in the connector. The docs state: *There is a tradeoff between using compression (saving your bandwidth) and using the sendfile feature (saving your CPU cycles). If the connector supports the sendfile feature, e.g. the NIO connector, using sendfile will take precedence over compression. The symptoms will be that static files greater that 48 Kb will be sent uncompressed.* It's trivial that adding compression uses CPU cycles, but does that imply that turning sendFile off even without enabling compression would increase CPU cycles? It's worth mentioning that the site serves a large (8mg) SWF file. I believe that was one of the pluses of NIO/sendFile, that it was good with sending large files under heavy traffic? We also only really need compression on XML data, the site has minimal HTML, SWF's don't really benefit from gzip and some binary data we send back and forth is already compressed. I could manually implement compression on XML at the application level and within the SWF, if turning off sendFile will have negative consequences. Tomcat 7.0.42 RHEL6 ~4T outbound traffic/day Best, John
Re: Site down for maintenance senario
Deploy a ROOT web application whose 404 page says Down for maintenance. You could even customize this kind of thing to only respond to certain URL-prefixes (like [ROOT]/mywebapp/*). What will you do while Tomcat is restarting, though, if you have to restart? Restarts take about a second or two, it's an acceptable downtime. Custom 404's are a good idea -- the only content I need in the directory is the 404 maintenance page itself. I can just swap the names of the actual webapp and 'maintenence' webapp directories (they're exploded). Thanks to everyone for the other good suggestions.
Hosting recommendations
We're getting killed by our hosting provider (RS) over bandwidth issues. I swear we scoped this out but somehow were over our agreement by an alarming amount. Our daily bandwidth looks like this: http://s17.postimg.org/btl0sj3jz/rs_traffic.png That's ~4T/day in bandwidth. I know it's a little outside the scope of this group, but there's a lot of smart people on here and I am using Tomcat on the two webservers. Can anyone suggest a managed hosting company they like. Preferential to iron, but if a cloud has worked for you please let me know. My boss is going to stab me. Thanks, John
Re: tomcat-native libraries
Installing the native library will make a difference. Whether the difference is large enough to notice depends very much on your application. If you want to improve your application's performance I suspect your time would be better spent with a profiler to see where the bottlenecks are in your application. Mark +1 I had the native APR installed and ended up removing it in favor of keeping things simple. The NIO connector often recommended by Chris S. and others works very well.
Re: Hosting recommendations
On Mon, Mar 17, 2014 at 9:55 AM, Mark Thomas ma...@apache.org wrote: On 17/03/2014 13:41, John Smith wrote: We're getting killed by our hosting provider (RS) over bandwidth issues. I swear we scoped this out but somehow were over our agreement by an alarming amount. Our daily bandwidth looks like this: http://s17.postimg.org/btl0sj3jz/rs_traffic.png That's ~4T/day in bandwidth. That looks odd. 1. Is that volume of traffic reasonable for your site? 2. Are those figures consistent with your access logs? 3. I'd expect incoming to be a lot less than outgoing for a typical site. I know it's a little outside the scope of this group, but there's a lot of smart people on here and I am using Tomcat on the two webservers. Can anyone suggest a managed hosting company they like. Preferential to iron, but if a cloud has worked for you please let me know. My boss is going to stab me. That sort of traffic level is going to cost $$$ pretty much anywhere. For example, the bandwidth alone is going to cost upwards of $70k/month with Amazon EC2. I know it isn't the question you asked but I'd be looking hard at the traffic to see if a) those numbers are correct and b) someone didn't do something REALLY silly that is causing excessive traffic. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Mark, 1. Yes, we have ~500,000 visitors per day, and the site is based around a very popular game that is very data intensive (users creating, browsing and loading levels and replays). 2. Yes 3. I agree. I'm not clear on the MRTG graphs. The one I posted is the firewall. The loadbalancer shows equal amounts of in/out, and the web servers show same incoming as the firewall, and slightly higher outgoing. It's seems unlikely, but it *may* reflect level saving. I would also think out would be higher than in though. All that said, I think the numbers are correct, especially since they look the same as the MRTG graphs from the old host. I'd at least like to get a quote from another hosting company. Any recs at all? Best, John
Re: Hosting recommendations
On Mon, Mar 17, 2014 at 10:25 AM, Mikolaj Rydzewski m...@ceti.pl wrote: On 17.03.2014 15:15, John Smith wrote: 1. Yes, we have ~500,000 visitors per day, and the site is based around a very popular game that is very data intensive (users creating, browsing and loading levels and replays). Just a rough idea: maybe differential level load/save will help? In other words: to download/upload only changed entities Like a diff? That's a good idea. It's not that there aren't optimizations for the data going back and forth, it's just there's got to be something comparable to RS that's not as expensive.
Site down for maintenance senario
Is there a straightforward way to toggle or add something in Tomcat, in the event a webapp is intentionally taken 'offline for maintenance? The user would receive the same single notification page saying as much, for any and all requests. Tomcat 7.0.42
Re: NIO connector - connections and threads
Thanks for your reply. So are the open HTTP connections that use my web application code waiting in line to be processed by the available threads specified in maxThreads? Best, John On Sun, Mar 9, 2014 at 12:44 PM, Konstantin Kolinko knst.koli...@gmail.comwrote: 2014-03-09 2:08 GMT+04:00 John Smith tomcat.ran...@gmail.com: Sorry, forgot: Tomcat 7.0.42 On Fri, Mar 7, 2014 at 3:59 PM, John Smith tomcat.ran...@gmail.com wrote: The NIO connector has two attributes from the standard HTTP Connector implementation, maxConnections and maxThreads with defaults of 1 and 200, respectively. Can anyone shine some light on how these work together? If I'm allowing up to 1 connections, would that mean I only have 200 threads to process through them? It would seem to be a disparity between the defaults. If I'm expecting maxConnection numbers in the area of ~2000 at any given time, wouldn't I want to bump up my maxThreads closer to match that? Production environment is: DELL PowerEdge R720 Single Socket Six Core Intel Xeon E5-2640 2.5GHz 32 GB RAM RHEL 6 Roughly speaking, The new APIs in java NIO and in Apache APR (and ultimately in underlying OS) allow to test whether there are incoming data on a network socket without actually reading it. A thread is needed when Tomcat calls your code in a web application to process a request. When request processing ends and control is returned to Tomcat, the request processing thread is decoupled from connection and is used to process other connections. With keep-alive feature in HTTP/1.1 protocol there may be several HTTP requests on the same HTTP connection, maxConnections = how many open HTTP connection can be hold by Tomcat maxThreads = how many requests are being actively processed at the same time. Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Executor thread pool
How dumb am I being by not using an Executor with a named thread pool? Currently I just have a Connector in server.xml: Connector port=8080 protocol=org.apache.coyote.http11.Http11NioProtocol connectionTimeout=2 redirectPort=8443 / Assuming ~2000 simultaneous connections. Tomcat 7.0.42. RHEL6. Best, John
Re: NIO connector - connections and threads
On Mon, Mar 10, 2014 at 11:48 AM, Caldarale, Charles R chuck.caldar...@unisys.com wrote: From: John Smith [mailto:tomcat.ran...@gmail.com] Subject: Re: NIO connector - connections and threads Don't top post. So are the open HTTP connections that use my web application code waiting in line to be processed by the available threads specified in maxThreads? The connections won't be waiting, but requests arriving over those connections may wait if all the threads are busy. Do you really have more than 200 simultaneous _requests_ active? - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org Don't top post. Sorry, just getting this list through gmail, normal replies are top posted. Do you really have more than 200 simultaneous _requests_ active? If you're implying are 200 people simultaneously, hitting the same page at the same time, or making the same HTTP POST at the same time, the answer is, yes, probably. -John
Re: NIO connector - connections and threads
Collecting some peak usage data might be interesting. You definitely want your max thread limit to be a bit above the number of concurrent requests you're handling. Of course, that has to be balanced against limits on other resources, such as memory and data base connections. - Chuck Thanks, I'm in the process of implementing MoSKito, so I'm hoping to get some good reporting there.
Re: NIO connector - connections and threads
Sorry, forgot: Tomcat 7.0.42 On Fri, Mar 7, 2014 at 3:59 PM, John Smith tomcat.ran...@gmail.com wrote: The NIO connector has two attributes from the standard HTTP Connector implementation, maxConnections and maxThreads with defaults of 1 and 200, respectively. Can anyone shine some light on how these work together? If I'm allowing up to 1 connections, would that mean I only have 200 threads to process through them? It would seem to be a disparity between the defaults. If I'm expecting maxConnection numbers in the area of ~2000 at any given time, wouldn't I want to bump up my maxThreads closer to match that? Production environment is: DELL PowerEdge R720 Single Socket Six Core Intel Xeon E5-2640 2.5GHz 32 GB RAM RHEL 6 Best, John
NIO connector - connections and threads
The NIO connector has two attributes from the standard HTTP Connector implementation, maxConnections and maxThreads with defaults of 1 and 200, respectively. Can anyone shine some light on how these work together? If I'm allowing up to 1 connections, would that mean I only have 200 threads to process through them? It would seem to be a disparity between the defaults. If I'm expecting maxConnection numbers in the area of ~2000 at any given time, wouldn't I want to bump up my maxThreads closer to match that? Production environment is: DELL PowerEdge R720 Single Socket Six Core Intel Xeon E5-2640 2.5GHz 32 GB RAM RHEL 6 Best, John
Re: Optimization on simple requests
Chris, Thanks! Very helpful advice. Best, John On Tue, Mar 4, 2014 at 1:54 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 John, On 3/4/14, 1:17 PM, John Smith wrote: Tomcat 7.0.42 on RHEL6. Assume that Tomcat is serving only one jsp page. Say it just rewrites a parameter value from the querystring to the html within the jsp. Also assume that there are ~200,000 users attempting to access that page - say almost simultaneously. What are the most relevant optimizations I can make to a single instance of tomcat for this scenario? So you want the highest-performance solution to the above scenario? As for Tomcat configuration, I would use the NIO connector with a large number of max connections (you'll have to see what practical size to give it) and a large number of threads in your thread pool (i.e. executor). NIO gets you the benefit of not blocking waiting for a second (or third, etc.) keepalive request to arrive over a connection before that thread can be used to do some real work. If all connections are Connection:close then this is less of an issue. If you have a big, beefy CPU relative to your Internet connection's bandwidth, you should probably enable compression on the connector: that will help you push bytes back to the client faster. You'll have to test whether or not this actually helps you in your particular situation, because you are trading CPU time for I/O time. Define only one Host element in your server.xml, and name it whatever your public hostname is: there is a slight optimization in the mapper that works slightly faster if you have exactly one Host element, and if that name matches the Host header from the request. (There is an even faster case for where there are no elements in the host list, then the default is used, but I'm not sure how to get a zero-element host list and yet still have a default host). Don't add any Valves or Filters that you don't absolutely need. I would remove any intermediate proxies that don't absolutely need to be there (like Apache httpd, Microsoft IIS, nginx, etc.). Tomcat itself comes fairly well-configured for performance out of the box (except for the use of the BIO connector, which gets the job done and it very stable and reliable, but certainly does not win any speed contests). If you want to optimize the hell out of the experience, you'll want to dump JSP: there's a lot of setup that goes into creating the environment in which a JSP page runs, and you don't mention that you need any of it above. If you just need to write HEADER + some value from query string + FOOTER, then try to do that all in 3 I/O writes, like this would be in a servlet: static final String HEADER = htmlheadtitleMy Fast Page/title/headbodyh1My Fast Page/h1pYour parameter value is i; static final String FOOTER = /i/p/body/html; void doGet(request, response) { ServletOutputStream out = response.getOutputStream(); out.print(HEADER); out.print(request.getParameter(key)); out.print(FOOTER); } To save network bandwidth, remove any non-essential whitespace from your text as I have done above. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTFiFPAAoJEBzwKT+lPKRYR5wP/iiaEcMIFxKBE9Rr9EP6ZhA+ +fxznQ1QED232LlhvAAcAiAjnOOv/dzLxmC62dai9EZoV0/24WcMpYaEjaRo2jZu jIyeGb4Dn4ommJj7aPG+yesPRRTBY6j23SIauWbnRNBCggn/YCpOnjERuUHPtjMO G4kDeZaHGGjfirwTuPYCKxiKlYow6C4H8HUzLH84BvuktPPCgO16qbtCSCI0st+b av4pza4lzKSO3YsjS3PBNa7eI9q7zvLYqTeB7TziyLq7Jf5OOWPL73qUVJUgb54A M6GzvsdIYWHCigGZff0iHT3oNbDEteSVK7TPLP8+XzI8x8F+xsn5G8yv5wXhStDH 44g2E2hZLwLhaaSiJqtxKGb2kTwoJA+CX33MnbngOkMGUO7SmRMlkx77d08GiYoA uvOKep8zz7R4Is8EZu5sdzUQSxPx2Y59uzQNMiBeER47d+hfu4aOl241QUrN2osO NsddzzXB6i9auvdhDdGUkNwbT2Iy8NtMKPBUvM+LWz2GC+8+/WyVeRjhQ5N3BUwc 5YHCKrHVEgZR/NO7j6HvsqXBdUnbt8JNFp0O6XtkCUtlilDabki50wIqVXn/jEmc rG9YJKYDFDQdxJSEnpeZEw5+iDmORkSyIOEMw5htqVCCgeBRp2jeATVWKpdcM76G EJD/P6bdni3Vj7kthhjs =ADJI -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Optimization on simple requests
Tomcat 7.0.42 on RHEL6. Assume that Tomcat is serving only one jsp page. Say it just rewrites a parameter value from the querystring to the html within the jsp. Also assume that there are ~200,000 users attempting to access that page - say almost simultaneously. What are the most relevant optimizations I can make to a single instance of tomcat for this scenario? TIA, Alec
tomcat log server
Hi Guys, I have tomcat 6.0.29 running different instances on same and different hardware, I want to create log server on one system (nfs mount the file system), so every instance must create log files in that place like SERVER.catalina.2011-07-30.log, SERVER1.catalina.2011-07-30.log, SERVER.catalina.out etc. rather than ${catalina.base}/logs Can some body guide me , really appreciated Regards John
tomcat clustering
Hi , I am trying to cluster 3 nodes on one machine, follow the instructions http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html At Deployer className, i have changed Deployer className=org.apache.catalina.ha.deploy.FarmWarDeployer tempDir=/usr/local/tomcat/web-temp/ deployDir=/usr/local/tomcat/webapps watchDir=/usr/local/tomcat/web-listen/ watchEnabled=false/ deployed my war file deployDir=/usr/local/tomcat/webapps after restarting the tomcat my war file is not exploded and in log i am getting SEVERE: FarmWarDeployer can only work as host cluster subelement! I checked Deployer className=org.apache.catalina.ha.deploy.FarmWarDeployer on tomcat site http://tomcat.apache.org/tomcat-6.0-doc/config/cluster-deployer.html This goober is currently pretty broken, but we are working hard to fix it search on google but couldn't find required info. My question, is where I can deploy my war file so that all three nodes can see and work in cluster. Any help will be appreciated Regards John
tomcat 6.0.20 different instances with diff ips but same port no 80
Hi Guys, Is it possible by using tomcat 6.0.20 with different instances with diff ips but same port no 80 on one system ??? I have three instances running on one machine named *a,b and c with each has different IPS (192.168.205.10/11/12). * I ran instance a on port 80 (it ran), but when I tried to run another instance b on port 80 ( I got LifecycleException: Protocol handler initialization failed: java.net.BindException: Address already in uselt;nullgt;:80)..*.because port 80 is already used * My question is I need to run three applications on port 80 on same machine but different IPs, what is the best way and how ? I really appreciate some one give me some hint (as I can do same think in ISS server) Regards John