Hi Chris or ?, I appreciate the response - here goes with my answers and and more questions!
Christopher Schultz-2 wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > SP, > > pichels wrote: >> We are experiencing users having issues connecting to our RH server and >> getting disconnected with errors in their browsers that state there is a >> DNS >> error or page cannot be displayed. > > Is this a browser error message, or something coming from the Tomcat side? > > Browser message - "Page cannot be displayed" - in Firefox or IE6/7 - even > using the IP address bypassing DNS of the Web server. > > >> Users connect to Apache then Tomcat serves up our JSP's that sometimes >> accesses specific MySQL server DB's. > > Are you using a connection pool? If so, what is the configuration? If > not, how are you connecting to the database (and is there a limit on the > number of db connections you allow)? > > > No - Pooling. JDBC connector. No limit. > > >> Most of the pages seems to be index.jsp or index.php lately for both >> websites(These are websites internal - our company web servers). > > MediaWiki uses index.php as it's front-controller, so it's no surprise > that you'd be getting many errors with that page in particular (because > requests always go through that page). Is index.jsp similar to index.php > in that it handles a great percentage of your servlet-server traffic? > > > Yes, for Mediawiki we are setup to use index.php - understood. > With Tomcat we don't use index.jsp as a front-controller for servlet > traffic. > > > >> In Tomcat: it is a mess of exceptions that our developers try and keep >> cleaned up. ( We do not use Log4j) > > Would you care to post some of those? What are the most popular > exceptions? What seem to be the worst-sounding? > > > Most Popular below: > > WARNING: Error sending end packet > java.net.SocketException: Broken pipe > at java.net.SocketOutputStream.socketWrite0(Native Method) > at > java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92) > at java.net.SocketOutputStream.write(SocketOutputStream.java:136) > at org.apache.jk.common.ChannelSocket.send(ChannelSocket.java:537) > at > org.apache.jk.common.JkInputStream.endMessage(JkInputStream.java:127) > at org.apache.jk.core.MsgContext.action(MsgContext.java:305) > at org.apache.coyote.Response.action(Response.java:183) > at org.apache.coyote.Response.finish(Response.java:305) > at > org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:205) > at > org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:283) > at > org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:773) > at > org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:703) > at > org.apache.jk.common.ChannelSocket$SocketConnection.runIt(ChannelSocket.java:895) > > at > org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:689) > > at java.lang.Thread.run(Thread.java:595) > > > >> In Mysql: there are lots of slow queries I've been logging - > 1 sec. > > Are these mostly coming from your webapp or MediaWiki? Have you > investigated these? Often, slow queries arise from inadequate or stale > indexes. Try running REPAIR TABLE [tablename] on a table that sees a lot > of slow queries. That should update the indexes in case you recently > bulk-loaded a lot of data. If that's not the problem (and it's unlikely > that it /is/ the problem), you should start running EXPLAIN on some of > those queries to see if you have appropriate indexes on your tables. > > > > Mostly coming from Tomcat/webapp. Yes, one we fixed but we are still > having issues w/ slow queries. > Ok, we'll look into running MySQL Explain. > > > > >> The errors that occur are random browser disconnects - urls that end or >> start with *.jsp/ or *.php/ pages. > > Something like SocketException: Connection closed? If that's the case, > the exceptions themselves are not a problem, but could be indicative of > a larger problem: poor performance causes people to hit STOP on their > browsers and re-try requests. > > > Yes, see above for SocketExceoptions. > Users are probably not hitting the Stop button - usually they refresh and > most of the time the page reloads OK. > We are looking at MySQL Performance. > > > >> We have also tried using WinXP host files and using an IP address to >> bypass >> DNS altogether - no luck. > > Are you actually performing DNS lookups? > > Yes, with all the other users/clients. > We have a pilot "test" group of users only using the IP address in the URL > box of their browsers. > > Two things might be happening if DNS is really your problem: > > 1. Your DNS server is slow. Mitigations are a) get a > faster/closer/extra DNS server or b) disable DNS lookups > > Maybe - perhaps our Active Directory DNS serves up a alot of requests from > our users and servers. > > > 2. Your DNS settings are misconfigured (e.g. wrong ip address, > etc.). Check your DNS configuration. > > > Definitely not misconfiguration. > > >> Problem around 4:15PM - Jan 9, 2009: >> http://10.97.24.23/Sales/IndentedATPServlet?partNum=086EXUCCCXMA&qty=1&warehouse=M > > What was the problem? > > User was viewing a Servlet page that was querying the AS400 DB2 DB on the > backend for a part number. > Tomcat may have been waiting for a long time and then teh user recieved > "page cannot be displayed" or "Server cannot be found" in IE7 browser. > > > http://www.nabble.com/file/p21485970/dns.jpeg > > > NOTE: The connection is thru Tomcat JDBC connector to AS400 DB2 DB. > > >> Problem around 5:05PM - Jan 9, 2009: >> http://wsi/Qoe/index.jsp >> (same thing....Page Cannot Be Displayed) > > Again... what was the problem? Look in catalina.out or some other log > file in your Tomcat's logs directory. > > We have Apache & Tomcat Debugging set very high. And have MySQL slow > queries and error logs. > Can't find any relevant to error. > Can we set Tomcat debugging with date/time stamps or will Log4j help more > with debugging? > > >> Our MysqL server has a tendency to ramp up or the load avg increases >> using >> massive CPU cycles - I restart Tomcat and the system usually calms down - >> Mysql included. > > This is why I was asking about connection pooling: it looks like Tomcat > is murdering your DB. This could be due to too many connections to > MySQL, overly high query volume, or sloppy code in your application > leaking database resources. > > > Java Developer is looking into code constantly and cleaning up code. > He agreed that there is old sloppy code perhaps and possibly too many > connections. > Can we control the connections or help diagnose code errors in an easier > way? > > >> We are currently using tcpdump on the server & Wireshark on the affected >> clients to look at the packet level of my connections to the webserver. > > Have you observed anything interesting using these tools? > > > I've sent the logs to RHEL(RedHat Linux) support and they have been > reviewing and think we have a network issue - router or VMware problems > since this is a > > VMware guest. > However, our clients are on a flat subnet with the server. > Clients = 10.97.25.X > Server = 10.97.24.23 > > I was also told by another tech at RHEL support that RHEL5 runs fine on > WMWaare as is if it were a physical box - confusing. > > > >> I know more details are probably needed - this is a complex problem. >> And only seems to happen in the am right when users login to their >> machines >> and start up their browsers and then happens at the end of the day as >> well. > > Maybe you are just getting more peak traffic than your setup can handle. > Have you performed any formal load testing? > > > Perhaps we have too much traffic or our applications need to be further > tuned or reinstalled from source code. > I am using the RHEL5 RPM's from RH Network. > > I see there are many ways to load test" MySQL - do you have any > suggestions on where I can start? > A free/Open source tool would be preferred. <smile> > > > Thanks for all your questions and insight! > > -SP > > > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org > > iEYEARECAAYFAklvYmUACgkQ9CaO5/Lv0PClcQCeOid/7wFe7/JvwpQHUE96xYft > bagAn1MC7Rz+At5JitXUA8R4GHO/8BMf > =Lxrm > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > > -- View this message in context: http://www.nabble.com/Tomcat%3A-client-disconnects--tp21424139p21485970.html Sent from the Tomcat - User mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
