-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 SP,
pichels wrote: > Christopher Schultz-2 wrote: >> Is this a browser error message, or something coming from the Tomcat side? > > Browser message - "Page cannot be displayed" - in Firefox or IE6/7 - even > using the IP address bypassing DNS of the Web server. Hmm. I wonder if you are actually having trouble contacting the server - -- that is, not an application error but possibly run out of available queued sockets on the server or some odd network error. Maybe check using a packet capture like Wireshark to see if the server is even picking up the phone. >> Are you using a connection pool? If so, what is the configuration? If >> not, how are you connecting to the database (and is there a limit on the >> number of db connections you allow)? > > No - Pooling. JDBC connector. No limit. That's bad. When you get a storm of activity, you can choke both your app server /and/ your database server if the number of connections is not limited. MySQL is pretty fast to establish a new connection, but many databases take significant amount of time to create new connections and so pooling is also a performance optimization. > Yes, for Mediawiki we are setup to use index.php - understood. > With Tomcat we don't use index.jsp as a front-controller for servlet > traffic. Okay. Does index.jsp do anything significant, then? It sounded like you were saying that you observe many problems with that specific page. Maybe you are just having generic app server slowness and since the home page is popular, you see a lot of failures there. Hmm. >> Would you care to post some of those? What are the most popular >> exceptions? What seem to be the worst-sounding? > > Most Popular below: > > WARNING: Error sending end packet > java.net.SocketException: Broken pipe > at java.net.SocketOutputStream.socketWrite0(Native Method) This is happening because the client hung up the phone before reading the entire response (or any of it). This will happen if a client hits the STOP button or clicks on another link or hits RELOAD while an existing request is still pending. Users will do this if the site feels too slow, so it's a vicious cycle: slow site generates additional requests from anxious users which ... slows down the site. :( > Mostly coming from Tomcat/webapp. Yes, one we fixed but we are still > having issues w/ slow queries. > Ok, we'll look into running MySQL Explain. You might want to get the O'Reilly book on optimizing MySQL. It's thin (but cheap!) and gives you some good information about what to look for. Unfortunately, MySQL isn't really that tunable. On the other hand, MySQL requires very little in the way of tuning! >>>> We have also tried using WinXP host files and using an IP address to >>>> bypass >>>> DNS altogether - no luck. > Are you actually performing DNS lookups? > > Yes, with all the other users/clients. > We have a pilot "test" group of users only using the IP address in the URL > box of their browsers. That shouldn't matter: the client's computer will do DNS resolution before your server is even contacted. If there are server-related DNS issues, it will be because the server is trying to do reverse-lookups on the clients, not because of what the client's URL bar shows. >>>> Problem around 4:15PM - Jan 9, 2009: >>>> http://10.97.24.23/Sales/IndentedATPServlet?partNum=086EXUCCCXMA&qty=1&warehouse=M > What was the problem? > > User was viewing a Servlet page that was querying the AS400 DB2 DB on the > backend for a part number. > Tomcat may have been waiting for a long time and then teh user recieved > "page cannot be displayed" or "Server cannot be found" in IE7 browser. Hmm. MySQL or DB2? How many databases are you using? > http://www.nabble.com/file/p21485970/dns.jpeg OIh, that could happen for lots of reasons, including "the server just isn't running at all". I hate those "helpful" error messages. Wild goosechases all around. > NOTE: The connection is thru Tomcat JDBC connector to AS400 DB2 DB. Technically, the JDBC connector is not Tomcat-specific. > We have Apache & Tomcat Debugging set very high. And have MySQL slow > queries and error logs. Note that debugging logging can slow down your system a lot. File IO is much worse than, say, object creations and traversals. > Can't find any relevant to error. > Can we set Tomcat debugging with date/time stamps or will Log4j help more > with debugging? Heh. See this week's archives for log4j issues people have been having. I wouldn't worry about that just yet. >> This is why I was asking about connection pooling: it looks like Tomcat >> is murdering your DB. This could be due to too many connections to >> MySQL, overly high query volume, or sloppy code in your application >> leaking database resources. > > Java Developer is looking into code constantly and cleaning up code. > He agreed that there is old sloppy code perhaps and possibly too many > connections. > Can we control the connections or help diagnose code errors in an easier > way? The very best thing to do is to use a database connection pool. Tomcat supplies one, which is convenient. What is inconvenient is that you probably will have to change the way you get new database connections in your code. Hopefully, your application uses a single method to get a connection, and that method can simply be re-implemented to use a connection pool. Otherwise, you have a lot of work ahead of you. Tomcat's connection pool can also be put into a debugging mode where connections improperly closed (or not at all) can be logged, including a stack trace for where those connections were requested. It's a great help when attempting to locate and fix these kinds of issues. I suspect that switching to use a connection pool will solve around 50% of your problems, and then give you a lot of information about how to further improve your application. You may find that your application leaks connections all over the place which can cause both Tomcat and MySQL to thrash, which makes the user experience suck for the user. As you mentioned, bouncing Tomcat makes everything better, so it sounds like your application is a bit sloppy when it comes to resource management. At least you're not running out of memory ;) It's also good that you have an engineer who is willing to go after old code and fix it rather than just saying "I didn't write it, don't blame me". If you're ready to make the jump to a connection pool (do it!), let me know and I'll give you info on configuration and how to do the code (actually, it's all here in the Tomcat site: http://tomcat.apache.org/tomcat-5.5-doc/jndi-datasource-examples-howto.html). A few caveats about the above page: - - The instructions tell you to put the <Resource> element into the server.xml file inside your <Context>. Assuming you are using a META-INF/context.xml file (you should be!) then put this configuration into your context.xml file, not into server.xml. - - The instructions tell you to use the autoReconnect=true parameter in your JDBC connection URL. You should not (MySQL has asked users not to use this for, literally, years). Instead, set the validationQuery attribute of the <Resource> element to something like "/* ping */ SELECT 1". This will issue a simply query to the server before the connection is handed over to your application, so you'll always get a fresh one. - - The MySQL example really contains some MySQL-specific stuff, then some generic configuration (using the MySQL driver as an example) and then stops. The Oracle example actually contains the code to get the DataSource (and then the JDBC connection) from the server. This is the same no matter what backend database you have. Finally, note that you can have multiple DataSources configured. Feel free to configure one for MySQL and another for your DB2 instance. > I've sent the logs to RHEL(RedHat Linux) support and they have been > reviewing and think we have a network issue - router or VMware problems > since this is a VMware guest. Well, vmware guests can certainly have issues. You aren't using one for production, are you? I'm not sure I would recommend that. > I was also told by another tech at RHEL support that RHEL5 runs fine on > WMWaare as is if it were a physical box - confusing. It should run fine. I use vmware guests all the time for testing and I've never had any problems. But, I don't use them for load testing or anything like that -- just easy ways to lug a linux server around on my win32 laptop. > I see there are many ways to load test" MySQL - do you have any > suggestions on where I can start? > A free/Open source tool would be preferred. <smile> MySQL is probably running very well. As I said earlier, MySQL has very few performance-tuning settings -- mostly changing the sizes of various caches. You'll get much more of a performance boost our of properly sizing resource pools (such as db connections) in your application and making sure that you properly clean up after yourself. Let us know what you decide, and how things turn out. Feel free to post back with more info and/or questions. Hope that helps, - -chris -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAklvsVUACgkQ9CaO5/Lv0PArfwCgqnEZgWX3qrCAlbMyYJ2kNZVG 6jwAoKAAyITQV62uh2hvopKlZR8Mc5rQ =B62W -----END PGP SIGNATURE----- --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
