Re: tomcat pausing and process killed
2012/5/9 Miguel Gonzalez miguel_3_gonza...@yahoo.es: I have tomcat 5.5.27 and jdk 1.5.0_17 (..) May 8, 2012 12:03:48 PM org.apache.tomcat.util.http.Parameters processParameters WARNING: Parameters: Character decoding failed. Parameter skipped. (...) I will check it, however I don't see in the access logs something relevant. Any string I can check or status code? 1. Requests that are rejected early (that is before they reach a Valve) would not be logged by AccessLogValve in any Tomcat 5.5.x version. This issue was fixed ~1,5 years ago in 6.0 and 7.0 only. Thus to see all received requests you need to have an up-to-date version of Tomcat 6 or 7. 2. When invalid chunk error is detected during parameter parsing, the bad parameter is just ignored as if it never existed in the query string. It does not cause request to fail, because Servlet API methods such as request.getParameter() do not support error reporting. They cannot throw exceptions and they cannot change the response. What your code does when a parameter is missing is up to your code. Your stacktrace mentions FastCommonAccessLogValve so this request should be mentioned in the access log. memory use increases just before the system crashed. Maybe some bot is walking around your server, and that causes creation of new session for each request? Maybe your web site is popular in certain hours and cannot withstand the load? Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
Miguel, just a couple of general remarks. Tomcat is an open-source project, developed and supported by volunteers. These volunteers do not have all the time in the world, so they tend to focus first on the current Tomcat version (7.x), a bit less on the previous version (6.x), and much less on the previous version (5.5). You are running Tomcat 5.5, and you are having a problem with it. And your problem, as you describe it, does not seem to be a very common problem experienced by a lot of users. The first recommendation would be to upgrade your installation to Tomcat 7, or at least Tomcat 6. That is because it may be that the problem that you are seeing, was due to some feature or bug in Tomcat 5.5, that has been resolved in the meantime. It also because, if the problem still happens in Tomcat 7, you are likely to get a lot more attention to it on the part of the developers and other volunteers on this list. Next, about your issue : It sounds very strange that you would have an error message in the error log about an invalid encoding of a request parameter, but that you cannot find that request in the access log. It could also be a misconfiguration on your part. Can you copy and paste your Tomcat server.xml here (removing all the comments and any confidential information please) ? (repeat : copy and paste in the message, not in a separate attachment; the list strips most attachments) As Konstantin mentions separately, there are (a few) cases where Tomcat will not log an invalid request. A solution in that case would be to set up an Apache httpd server in front of your Tomcat, listening on the port on which Tomcat is listening now (presumably 80), and proxying all request to Tomcat (changing the Tomcat port to something else, e.g. 8080). This is usually a rather simple setup, but if you have trouble doing that, ask. Then the access log of Apache httpd may show what Tomcat does not show. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
It makes sense. This tomcat 5.5 is a tarball from Apache. Do you suggest using a rpm (I'm running Centos 5.2) which probably will be Tomcat 6.x or download Tomcat 7 from Apache? I usually prefer to have software packaged because I can get software security updates but I'm from HTTP Apache and MySQL world and normally updates are quite normal. However I see that I would have to use non-official repos to install at least Tomcat 6. Regards, Miguel De: André Warnier a...@ice-sa.com Para: Tomcat Users List users@tomcat.apache.org Enviado: Jueves 10 de Mayo de 2012 14:02 Asunto: Re: tomcat pausing and process killed Miguel, just a couple of general remarks. Tomcat is an open-source project, developed and supported by volunteers. These volunteers do not have all the time in the world, so they tend to focus first on the current Tomcat version (7.x), a bit less on the previous version (6.x), and much less on the previous version (5.5). You are running Tomcat 5.5, and you are having a problem with it. And your problem, as you describe it, does not seem to be a very common problem experienced by a lot of users. The first recommendation would be to upgrade your installation to Tomcat 7, or at least Tomcat 6. That is because it may be that the problem that you are seeing, was due to some feature or bug in Tomcat 5.5, that has been resolved in the meantime. It also because, if the problem still happens in Tomcat 7, you are likely to get a lot more attention to it on the part of the developers and other volunteers on this list. Next, about your issue : It sounds very strange that you would have an error message in the error log about an invalid encoding of a request parameter, but that you cannot find that request in the access log. It could also be a misconfiguration on your part. Can you copy and paste your Tomcat server.xml here (removing all the comments and any confidential information please) ? (repeat : copy and paste in the message, not in a separate attachment; the list strips most attachments) As Konstantin mentions separately, there are (a few) cases where Tomcat will not log an invalid request. A solution in that case would be to set up an Apache httpd server in front of your Tomcat, listening on the port on which Tomcat is listening now (presumably 80), and proxying all request to Tomcat (changing the Tomcat port to something else, e.g. 8080). This is usually a rather simple setup, but if you have trouble doing that, ask. Then the access log of Apache httpd may show what Tomcat does not show. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: tomcat pausing and process killed
From: Miguel Gonzalez [mailto:miguel_3_gonza...@yahoo.es] Subject: Re: tomcat pausing and process killed Do you suggest using a rpm (I'm running Centos 5.2) Definitely not. If you use a 3rd-party repackaged version of Tomcat, you'll need to get support from that 3rd party, since they often mangle things so badly as to be unrecognizable to standard Tomcat users and developers. download Tomcat 7 from Apache? Get it from tomcat.apache.org - always. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
On 10/05/2012 13:55, Miguel Gonzalez wrote: I usually prefer to have software packaged because I can get software security updates I'm not sure that this is a good strategy for Tomcat installations. Some distro's are very slow about updating their packages. p -- [key:62590808] signature.asc Description: OpenPGP digital signature
Re: tomcat pausing and process killed
Hi. Miguel Gonzalez wrote: Dear all, I have tomcat 5.5.27 and jdk 1.5.0_17 That's all quite old,and you should probably upgrade. As someone suggested to the list I enabled HeapDumpOnOutOfMemoryError. I have javamelody running to monitor tomcat. The only thing that I see is that memory use increases just before the system crashed. The system has crashed only showing this in the catalina.log: May 8, 2012 12:03:48 PM org.apache.tomcat.util.http.Parameters processParameters WARNING: Parameters: Character decoding failed. Parameter skipped. java.io.CharConversionException: isHexDigit ... Many entries like: May 8, 2012 12:03:48 PM org.apache.tomcat.util.http.Parameters processParameters WARNING: Parameters: Invalid chunk ignored. Ther above seems to indicate that your server receives requests with URLs that are not properly URL-encoded. In a URL, roughly-speaking, any character (in fact, any byte) that is not a printable US-ASCII character must be encoded as a %xy hex sequence, where xy is the hexadecimal value of the character. The4 above error seems to indicate that you are receiving request URLs which contain sequences like %xy where xy are not valid hexadecimal digits. In consequence, the URL-decoding part of Tomcat cannot decode these URLs properly, and complains. until I get : May 8, 2012 1:50:41 PM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-80 May 8, 2012 1:50:41 PM org.apache.coyote.http11.Http11BaseProtocol pause INFO: Pausing Coyote HTTP/1.1 on http-443 ... Now this seems to be a normal Tomcat shutdown, following a received shutdown command. But it could be the result of running out of memory for example. I do not know if the two are related or not. But in my suspicious mind, I would try to examine which request URLs are the badly-encoded ones, and try to find out where these requests come from. That is because they could very well be attempts to break your server (if it is connected to the Internet e.g.). And maybe your unexplained shutdowns are due to a succesful break-in, or a succesful DOS attack ? Not sure if the isHexDigit exception is causing this. but it happens 2 hours before the system breaks. Maybe an attacker is trying a series of URLs in a row, and finally they manage one that finds a weakness in your very very old Tomcat. I really don't know what to enable to get to know what is causing Tomcat to crash Enabling the AccessLog Valve may enable you to see the bad request URLs. If they do not show up in the Access log (unlikely, since Tomcat apparently continues to run), you may want to set up an Apache httpd front-end to your Tomcat, and log the accesses there. Searching Google for tomcat isHexDigit attack provides some links you might want to look at. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
Maybe an attacker is trying a series of URLs in a row, and finally they manage one that finds a weakness in your very very old Tomcat. I really don't know what to enable to get to know what is causing Tomcat to crash Enabling the AccessLog Valve may enable you to see the bad request URLs. If they do not show up in the Access log (unlikely, since Tomcat apparently continues to run), you may want to set up an Apache httpd front-end to your Tomcat, and log the accesses there. Searching Google for tomcat isHexDigit attack provides some links you might want to look at. I will check it, however I don't see in the access logs something relevant. Any string I can check or status code? Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 5/9/12 11:29 AM, Miguel Gonzalez wrote: I will check it, however I don't see in the access logs something relevant. Any string I can check Anything with %-codes that are invalid. You'll have to write your own regular parser or regex to find them. or status code? Uh, 400 or 500? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk+qyeMACgkQ9CaO5/Lv0PD9QQCguRk0Z8nEd313mohONnIzohtu wlcAnibOJhZ3uesL2Qk5KAYqQ8BeO1Zu =jiH/ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: tomcat pausing and process killed
On 09/05/2012 21:47, Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miguel, On 5/9/12 11:29 AM, Miguel Gonzalez wrote: I will check it, however I don't see in the access logs something relevant. Any string I can check Anything with %-codes that are invalid. You'll have to write your own regular parser or regex to find them. or status code? Uh, 400 or 500? I don't find anything weird in the access log, I can't relate any of the Invalid chunk ignored to any GET or POST. In fact checking at a particular time I don't find any weird entry Parsing 400 or 500 errors don't show anything either. Miguel - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
