Re: Does maxPostSize has an effect on file upload?
* Mark Thomas ma...@apache.org: Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I asked because I could not find a hint in the docs or the INTERNET. What doc do you mean? I looked into http://tomcat.apache.org/tomcat-6.0-doc/config/http.html for maxPostSize. Thanks, Kai - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Tomcat 6.0 - http client
Hello Guys, I am facing response latency issue when i am invoking the web service call ( deployed on tomcat 6.x) from an http client. Can some body please share /let me know what are recommended configuration settings which i need to take care of at the http client at the tomcat level. I guess that is the issue. Secondly, which tomcat threads i need to monitor, as in threads dumps i am seeing different types of threads like http ,TP,scheduler etc. it will be great if you guys can suggest how to analyze these thread dumps Thanks Vicky - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0 - http client
On 14/12/2012 11:03, vicky007aggar...@yahoo.co.in wrote: Hello Guys, I am facing response latency issue when i am invoking the web service call ( deployed on tomcat 6.x) from an http client. Can some body please share /let me know what are recommended configuration settings which i need to take care of at the http client at the tomcat level. I guess that is the issue. Maybe your webservice is just really slow? p Secondly, which tomcat threads i need to monitor, as in threads dumps i am seeing different types of threads like http ,TP,scheduler etc. it will be great if you guys can suggest how to analyze these thread dumps Thanks Vicky - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org -- [key:62590808] signature.asc Description: OpenPGP digital signature
Re: Tomcat 6.0.35, PersistentManager with FileStore - Session file size increases endlessly
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Nico, On 12/13/12 4:29 AM, Nico Peters wrote: First, some information to our setup: Manager configuration? We have recognized an unusually high number of disk operations on one of our servers and investigated the origin. We found out that there was one tomcat session file that grew already to 235GB and was increasing quickly (all other sessions on our server are less than 10KB). We then removed that session file, but it was recreated (starting from 0 bytes) and was again growing quickly. We then did a backup of that file and removed it again. After the second removal the session file didn't appear again. The server returned to normal operation. Did the session file represent an actual session that Tomcat was still maintaining? Did you inspect the HttpSession object to see if it contained any large piece of data (like a String containing q~q~#...)? I've investigated the session file and the file contained 3 lines. I was able to recognize the data of the first two lines (the default session parameters like lastAccessedTime as well as some POJOs we have added to that session). But the third line was endlessly repeating the following string: q~q~#q~'q~( The same thing, over and over again, like q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(q~q~#q~'q~(...? And now my questions: Does anyone know what this string means? Not I. How is it possible that a session can increase to this size (larger than the heap size of tomcat)? Good question. Is it a known tomcat bug? Not that I know of. Is it a known type of attack? It seems like it might be an attack -- like someone trying to fill-up your session (and heap) with junk. It could also have been some component going absolutely crazy (JVM, filesystem, etc.). How can you prevent this problem? We don't know what caused it. If it happens again, please take a few thread dumps of the JVM that is creating the file. That will help significantly. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLU9QACgkQ9CaO5/Lv0PAIFgCfZoWB+DeAPWy4XWXbLiNuuys/ 6R0AoJzZdKKMUDQv5azyELTXwNSZZX9z =WUsT -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat uses only a single Core
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 To whom it may concern, On 12/13/12 2:44 PM, spr...@gmx.eu wrote: I have an application running on Tomcat 7 on a 8-Core Linux 64 bit System. Unfortunately it uses only a single core. I have googled alot about this fact but did not found a solution, even not a clear reason for it. How do you determine how many cores the JVM is using? What kind of load are you putting on it? Are there any other cirumstances you aren't describing (such as heavy load coming from anywhere else, processor affinity, virtualization, etc.)? What can be the problem here? Anything from instrumentation to configuration. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLV7kACgkQ9CaO5/Lv0PCtLACfYxQXn1HiE3no5FeWHRSewZ11 fYkAmwTX95Ucjg40Ev7SVZDZefqchzl9 =45s6 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Data sources definitions are lost in memory
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Robert, On 12/13/12 3:34 PM, Robert Anderson wrote: Caused by: java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory In my Oracle 1.7.0 JRE's rt.jar, I can find a few JNDI ContextFactory classes, but not this one. I looked in 1.6.0 but couldn't find that class, either. I looked in other JARs but couldn't find anything containing the fscontext package. Are you using an alternative JNDI provider? http://www.jarfinder.com/index.php/java/info/com.sun.jndi.fscontext.RefFSContextFactory Looks like you may have some old stuff sitting around in your JRE installation, or even in your webapp. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLWsoACgkQ9CaO5/Lv0PAXDACfXjUbGlPbzJJy5qhJDBxUfqUN ljgAn0RZCDKnYkt51GIUfvtYzeMbwxAo =sylS -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Mark, On 12/13/12 10:27 AM, Mark Thomas wrote: Kai Weber kai.we...@glorybox.de wrote: I see the following behaviour on Tomcat 6.0.24: The maxPostSize is not set, so uses the default of 2MB. I can upload files bigger than 2MB (5MB for example). I use commons-fileupload to read the file. As expected. Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. Here's what that attribute currently says (in its entirety): The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing. The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes). It says FORM URL parameter parsing. That may be some kind of code that means only when the type is application/x-www-form-urlencoded. We could probably explicitly state that multipart requests are excepted from this limit. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLW7UACgkQ9CaO5/Lv0PANRgCfUA69SHOZk5O87VlcoFz4LELt lYQAoMH4B2tFFDnARGkRoB5BDR2GxfV1 =itOY -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0 - http client
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Pid, On 12/14/12 11:10 AM, Pid wrote: On 14/12/2012 11:03, vicky007aggar...@yahoo.co.in wrote: Hello Guys, I am facing response latency issue when i am invoking the web service call ( deployed on tomcat 6.x) from an http client. Can some body please share /let me know what are recommended configuration settings which i need to take care of at the http client at the tomcat level. I guess that is the issue. Maybe your webservice is just really slow? +1 Usually, network latency can be separated from all other factors by just using 'ping'. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLYKgACgkQ9CaO5/Lv0PDQ8ACfZShWQtiFRLOhuPjY/Gokw+HZ xjcAn3H7K27XjqwTXs45iTZU2RTQWPVa =scvf -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Data sources definitions are lost in memory
Hi Christopher, I'm suspecting that some webapp is causing this problem, because nothing has changed neither in JVM nor Tomcat (I'm a sysadmin in this company). We will investigate each application looking for suspects JARs (Developers have access to webapp directory and can deploy applications without the intervention of sysadmins :( ). It's a powerfull and bad jar! It makes two tomcats (load balanced) lose all data sources definitions and we need restart them. The strange thing is that most of the time everything works normal. Thanks, Robert On Fri, Dec 14, 2012 at 1:58 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Robert, On 12/13/12 3:34 PM, Robert Anderson wrote: Caused by: java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory In my Oracle 1.7.0 JRE's rt.jar, I can find a few JNDI ContextFactory classes, but not this one. I looked in 1.6.0 but couldn't find that class, either. I looked in other JARs but couldn't find anything containing the fscontext package. Are you using an alternative JNDI provider? http://www.jarfinder.com/index.php/java/info/com.sun.jndi.fscontext.RefFSContextFactory Looks like you may have some old stuff sitting around in your JRE installation, or even in your webapp. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLWsoACgkQ9CaO5/Lv0PAXDACfXjUbGlPbzJJy5qhJDBxUfqUN ljgAn0RZCDKnYkt51GIUfvtYzeMbwxAo =sylS -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does maxPostSize has an effect on file upload?
From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Does maxPostSize has an effect on file upload? Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. I think you have to read the relevant RFC: http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the doc. - Chuck THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0 - http client
On Fri, Dec 14, 2012 at 9:23 AM, Christopher Schultz ch...@christopherschultz.net wrote: I am facing response latency issue when i am invoking the web service call ( deployed on tomcat 6.x) from an http client. Maybe your webservice is just really slow? Usually, network latency can be separated from all other factors by just using 'ping'. But response latency is not necessarily (or even usually) related to network latency :-) May I suggest NewRelic (http://newrelic.com/) for analyzing your application to find out where the time is being taken up? Disclaimer: just a satisfied customer. -- Hassan Schroeder hassan.schroe...@gmail.com http://about.me/hassanschroeder twitter: @hassan - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Data sources definitions are lost in memory
Hi, We've found a suspect application: [root@tomcat3 webapps]# locate fscontext.jar /usr/java/tomcat/webapps/ebookapp/WEB-INF/lib/fscontext.jar [root@tomcat3 lib]# jar -tf /usr/java/tomcat/webapps/ebookapp/WEB-INF/lib/fscontext.jar META-INF/ META-INF/MANIFEST.MF com/sun/jndi/fscontext/ com/sun/jndi/fscontext/RefFSContextFactory.class com/sun/jndi/fscontext/FSContext.class com/sun/jndi/fscontext/FSContext$FSFile.class com/sun/jndi/fscontext/FSContext$FileBindingEnumeration.class com/sun/jndi/fscontext/FSNameParser.class com/sun/jndi/fscontext/FileNameClassEnumeration.class com/sun/jndi/fscontext/FSContextFactory.class com/sun/jndi/fscontext/RefFSContext.class com/sun/jndi/fscontext/RefFSContext$ObjectBindingEnumeration.class com/sun/jndi/fscontext/ObjectNameClassEnumeration.class com/sun/jndi/fscontext/AggregateNamingEnumeration.class com/sun/jndi/url/file/ com/sun/jndi/url/file/fileURLContext.class com/sun/jndi/url/file/fileURLContextFactory.class It's a third party application. Now we need to get in touch with suppliers. Best regards, Robert On Fri, Dec 14, 2012 at 2:33 PM, Robert Anderson ranom...@gmail.com wrote: Hi Christopher, I'm suspecting that some webapp is causing this problem, because nothing has changed neither in JVM nor Tomcat (I'm a sysadmin in this company). We will investigate each application looking for suspects JARs (Developers have access to webapp directory and can deploy applications without the intervention of sysadmins :( ). It's a powerfull and bad jar! It makes two tomcats (load balanced) lose all data sources definitions and we need restart them. The strange thing is that most of the time everything works normal. Thanks, Robert On Fri, Dec 14, 2012 at 1:58 PM, Christopher Schultz ch...@christopherschultz.net wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Robert, On 12/13/12 3:34 PM, Robert Anderson wrote: Caused by: java.lang.ClassNotFoundException: com.sun.jndi.fscontext.RefFSContextFactory In my Oracle 1.7.0 JRE's rt.jar, I can find a few JNDI ContextFactory classes, but not this one. I looked in 1.6.0 but couldn't find that class, either. I looked in other JARs but couldn't find anything containing the fscontext package. Are you using an alternative JNDI provider? http://www.jarfinder.com/index.php/java/info/com.sun.jndi.fscontext.RefFSContextFactory Looks like you may have some old stuff sitting around in your JRE installation, or even in your webapp. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLWsoACgkQ9CaO5/Lv0PAXDACfXjUbGlPbzJJy5qhJDBxUfqUN ljgAn0RZCDKnYkt51GIUfvtYzeMbwxAo =sylS -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 12/14/12 12:38 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Does maxPostSize has an effect on file upload? Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. I think you have to read the relevant RFC: http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the doc. I'm not sure what part of that spec would apply. Given the (Tomcat) documentation, one might reasonably believe that the maxPostSize would refer to either the total Content-Length for an application/x-www-form-urlencoded POST message or an individual Content-Length of a single part of a multipart/form-data POST message. The latter is not the case, and I think it's worth pointing that out. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.17 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEAREIAAYFAlDLdMcACgkQ9CaO5/Lv0PCkPwCghEQ2KYNPSCV+buGj/VtDvmme e/cAoJ3ci461Q4xD992azyLQi8FxRAym =+GMt -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Tomcat 6.0 - http client
Thanks guys for responding, from network side there are no issues infact as confirmed by our Network team I am accessing my application over http port, i seek your advise on tunning my http client with respect to my tomcat 1We're using MultiThreadedHttpConnectionManager, In this do you guys have any recommendation/best practices for setting up the parameters like maxHostConnections, maxTotalConnections,connection timeout w.r.t tomcat To narrow down response time issue , i took couple of thread dumps ,related to it i have following concerns I am getting hard time in making sense out of thread dumps , please shed some light on my below queries , Any weblink will be a great help :- 2 There were many http threads were in waiting state, is it bad ?? 3 In thread dumps there are many types of thread like http ,TP,scheduler etc a) when request is received by tomcat which type of thread will handle the request. ?? b) does there is a different thread type in case we're forwarding a request from apache to Tomcat over AJP port ?? c) In case if slow response respone from an application , what ideally we should look in thread dumps??.Does we should focus on the no. of HTTP idle threads or scheduler threads or something else. ?? Thanks for your help Time Vicky From: Hassan Schroeder hassan.schroe...@gmail.com To: Tomcat Users List users@tomcat.apache.org Sent: Friday, 14 December 2012 11:12 PM Subject: Re: Tomcat 6.0 - http client On Fri, Dec 14, 2012 at 9:23 AM, Christopher Schultz ch...@christopherschultz.net wrote: I am facing response latency issue when i am invoking the web service call ( deployed on tomcat 6.x) from an http client. Maybe your webservice is just really slow? Usually, network latency can be separated from all other factors by just using 'ping'. But response latency is not necessarily (or even usually) related to network latency :-) May I suggest NewRelic (http://newrelic.com/) for analyzing your application to find out where the time is being taken up? Disclaimer: just a satisfied customer. -- Hassan Schroeder hassan.schroe...@gmail.com http://about.me/hassanschroeder twitter: @hassan - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 12/14/12 12:38 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Does maxPostSize has an effect on file upload? Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. I think you have to read the relevant RFC: http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the doc. I'm not sure what part of that spec would apply. Given the (Tomcat) documentation, one might reasonably believe that the maxPostSize would refer to either the total Content-Length for an application/x-www-form-urlencoded POST message or an individual Content-Length of a single part of a multipart/form-data POST message. The latter is not the case, and I think it's worth pointing that out. Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain to what the maxPostSize attribute of the Connector really does relate ? What /is/ being measured there ? Is it, like, just the request's Content-length header, or is there a counter which really counts the bytes being read e.g. (and yes, obviously, if this is limited to an application/x-www-form-urlencoded POST, then it wouldn't be just the Content-length header) (and, if it covers a multipart/form-data POST, then is it the total body size, before of after decoding the Base-64 encoded bits ?) After all, this is a Tomcat users list, not a Tomcat dev list; so we can't really expect the public here to go delve into the Java code to find out. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does maxPostSize has an effect on file upload?
So I read some code in org.apache.catalina.connector.Request.java. When parseParameters() is called, it checks whether the Content-Type is multipart/form-data or application/x-www-form-urlencoded. If it's application/x-www-form-urlencoded, it checks to make sure that the Content-Length is not greater than maxPostSize. If it's multipart/form-data, it delegates to another method, parseParts(). In parseParts(), the code begins looping over the parts and placing them into the list of parts for the request. As it processes each part, it adds the size (in bytes) of that part (including the part contents or file contents if this is a file upload) to an aggregate post size variable. If at any point the aggregate post size exceeds the maxPostSize variable, it fails. Why the Content-Length is not checked, I am unsure. It seems it would be less expensive to throw the exception before ever trying to parse the parts. However, this APPEARS to be the functional equivalent of checking the Content-Length. So, it would seem that maxPostSize does, indeed, affect file uploads, and that if the total size of your request, including file parts, exceeds maxPostSize, that the request will fail. Whether this behavior is consistent with that specified in the spec, I am not an expert on that. Not that it matters, but this behavior is consistent with the land of PHP, where a maxUploadFileSize value of 50MB does you no good if you forget to increas your maxPostSize from the default of 5MB. (Note: It's entirely possible that I'm reading the code wrong. I would suggest that an experiment might solve this conclusively, but perhaps that has been tried and I missed that somewhere in the chain.) Nick -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Friday, December 14, 2012 1:15 PM To: Tomcat Users List Subject: Re: Does maxPostSize has an effect on file upload? Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 12/14/12 12:38 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Does maxPostSize has an effect on file upload? Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. I think you have to read the relevant RFC: http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the doc. I'm not sure what part of that spec would apply. Given the (Tomcat) documentation, one might reasonably believe that the maxPostSize would refer to either the total Content-Length for an application/x-www-form-urlencoded POST message or an individual Content-Length of a single part of a multipart/form-data POST message. The latter is not the case, and I think it's worth pointing that out. Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain to what the maxPostSize attribute of the Connector really does relate ? What /is/ being measured there ? Is it, like, just the request's Content-length header, or is there a counter which really counts the bytes being read e.g. (and yes, obviously, if this is limited to an application/x-www-form-urlencoded POST, then it wouldn't be just the Content-length header) (and, if it covers a multipart/form-data POST, then is it the total body size, before of after decoding the Base-64 encoded bits ?) After all, this is a Tomcat users list, not a Tomcat dev list; so we can't really expect the public here to go delve into the Java code to find out. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this message and its attachment(s). Underwriters Laboratories Inc. and its affiliates disclaim all liability for any errors, omissions, corruption or virus in this message or any attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
Williams, Nick wrote: So I read some code in org.apache.catalina.connector.Request.java. When parseParameters() is called, it checks whether the Content-Type is multipart/form-data or application/x-www-form-urlencoded. If it's application/x-www-form-urlencoded, it checks to make sure that the Content-Length is not greater than maxPostSize. That is probably because in that case, the body is not encoded, and the Content-length matches the length of the posted data. If it's multipart/form-data, it delegates to another method, parseParts(). In parseParts(), the code begins looping over the parts and placing them into the list of parts for the request. As it processes each part, it adds the size (in bytes) of that part (including the part contents or file contents if this is a file upload) to an aggregate post size variable. If at any point the aggregate post size exceeds the maxPostSize variable, it fails. This thus looks like it applies to the total size of the parts, after decoding the ones that are Base-64 encoded (such as uploaded files contents). Why the Content-Length is not checked, I am unsure. It seems it would be less expensive to throw the exception before ever trying to parse the parts. However, this APPEARS to be the functional equivalent of checking the Content-Length. If it was using the global Content-length header, it would count not only the encoded data bytes, but also the parts separators, headers etc.. So that's nice. It counts only the net data bytes, which is easier to compare to the size on disk of a file that you would upload. So, it would seem that maxPostSize does, indeed, affect file uploads, and that if the total size of your request, including file parts, exceeds maxPostSize, that the request will fail. Whether this behavior is consistent with that specified in the spec, I am not an expert on that. You've done a pretty good job so far, I believe. What maybe is the difference here, is that you probably looked at the Tomcat 7 code, in which I believe the FileUpload code has been integrated/merged. Maybe in Tomcat 6 it is different, and when FileUpload is used, it doesn't count the bytes, or doesn't pass the result back to Tomcat to match the MaxPostSize ? Not that it matters, but this behavior is consistent with the land of PHP, where a maxUploadFileSize value of 50MB does you no good if you forget to increas your maxPostSize from the default of 5MB. (Note: It's entirely possible that I'm reading the code wrong. I would suggest that an experiment might solve this conclusively, but perhaps that has been tried and I missed that somewhere in the chain.) Nick -Original Message- From: André Warnier [mailto:a...@ice-sa.com] Sent: Friday, December 14, 2012 1:15 PM To: Tomcat Users List Subject: Re: Does maxPostSize has an effect on file upload? Christopher Schultz wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Chuck, On 12/14/12 12:38 PM, Caldarale, Charles R wrote: From: Christopher Schultz [mailto:ch...@christopherschultz.net] Subject: Re: Does maxPostSize has an effect on file upload? Does a file upload as multipart/form-data not count to the size of the POST? No, as the doc make clear. I'm not so sure the docs make it clear. I think you have to read the relevant RFC: http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the doc. I'm not sure what part of that spec would apply. Given the (Tomcat) documentation, one might reasonably believe that the maxPostSize would refer to either the total Content-Length for an application/x-www-form-urlencoded POST message or an individual Content-Length of a single part of a multipart/form-data POST message. The latter is not the case, and I think it's worth pointing that out. Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain to what the maxPostSize attribute of the Connector really does relate ? What /is/ being measured there ? Is it, like, just the request's Content-length header, or is there a counter which really counts the bytes being read e.g. (and yes, obviously, if this is limited to an application/x-www-form-urlencoded POST, then it wouldn't be just the Content-length header) (and, if it covers a multipart/form-data POST, then is it the total body size, before of after decoding the Base-64 encoded bits ?) After all, this is a Tomcat users list, not a Tomcat dev list; so we can't really expect the public here to go delve into the Java code to find out. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this
Re: Does maxPostSize has an effect on file upload?
On 14/12/2012 19:58, Williams, Nick wrote: (Note: It's entirely possible that I'm reading the code wrong. Yes you are. Not completely wrongly but there are errors. The short version is as follows: If Tomcat is responsible for reading the request body such as via a call to a method like getParameters(), getParts() etc. then Tomcat will apply the maxPostSize limit. If the application is responsible for reading the request body then the limit does not apply. Hence if an application uses a third-party file upload library or just calls getInputStream() or getReader() the limit will not apply. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does maxPostSize has an effect on file upload?
A valid point that I have not considered. Since parseParameters() is not called until you get the request parameters or parts, a developer could getInputStream() and parse all the parts/parameters themselves, thereby completely skipping the maxPostSize checks. Thanks for correcting me here, Mark. Nick -Original Message- From: Mark Thomas [mailto:ma...@apache.org] Sent: Friday, December 14, 2012 3:17 PM To: Tomcat Users List Subject: Re: Does maxPostSize has an effect on file upload? On 14/12/2012 19:58, Williams, Nick wrote: (Note: It's entirely possible that I'm reading the code wrong. Yes you are. Not completely wrongly but there are errors. The short version is as follows: If Tomcat is responsible for reading the request body such as via a call to a method like getParameters(), getParts() etc. then Tomcat will apply the maxPostSize limit. If the application is responsible for reading the request body then the limit does not apply. Hence if an application uses a third-party file upload library or just calls getInputStream() or getReader() the limit will not apply. Mark - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this message and its attachment(s). Underwriters Laboratories Inc. and its affiliates disclaim all liability for any errors, omissions, corruption or virus in this message or any attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does maxPostSize has an effect on file upload?
If it was using the global Content-length header, it would count not only the encoded data bytes, but also the parts separators, headers etc.. So that's nice. It counts only the net data bytes, which is easier to compare to the size on disk of a file that you would upload. Indeed. A great explanation for why this would be done, and a much more logical way to do it than just using Content-Length (the easy way out). Thanks! This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this message and its attachment(s). Underwriters Laboratories Inc. and its affiliates disclaim all liability for any errors, omissions, corruption or virus in this message or any attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
On 14/12/2012 21:13, André Warnier wrote: snip/ If it's multipart/form-data, it delegates to another method, parseParts(). snip/ Why the Content-Length is not checked, I am unsure. It seems it would be less expensive to throw the exception before ever trying to parse the parts. However, this APPEARS to be the functional equivalent of checking the Content-Length. I think that is simply an oversight in the commit [1] that added the checking. If a content-length is present then checking it would be simpler. The existing checks need to be kept for the chunked request body case. As always, patches welcome. Mark [1] http://svn.apache.org/viewvc?view=revisionrevision=1195968 - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does maxPostSize has an effect on file upload?
Mark Thomas wrote: On 14/12/2012 21:13, André Warnier wrote: snip/ If it's multipart/form-data, it delegates to another method, parseParts(). snip/ Why the Content-Length is not checked, I am unsure. It seems it would be less expensive to throw the exception before ever trying to parse the parts. However, this APPEARS to be the functional equivalent of checking the Content-Length. I think that is simply an oversight in the commit [1] that added the checking. If a content-length is present then checking it would be simpler. The existing checks need to be kept for the chunked request body case. As always, patches welcome. In the Apache httpd DAV implementation, the LimitXMLRequestBody is the only way to control the maximum size of an upload. It is a p-i-t-a, because that size does not relate easily to the size of the file one is really uploading, making it difficult to set a sensible limit. The way Tomcat is apparently doing it now is much more sensible, in my humble opinion, because it does allow a direct and easy comparison with the files being uploaded. And since as per above it needs to be kept in some cases anyway, my vote - if I had one - would be to not change it. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does maxPostSize has an effect on file upload?
The way Tomcat is apparently doing it now is much more sensible, in my humble opinion, because it does allow a direct and easy comparison with the files being uploaded. And since as per above it needs to be kept in some cases anyway, my vote - if I had one - would be to not change it. I must agree with André. The process of base64 encoding a file increases the number of bytes it takes to transmit it. But since that is not the actual size of the file, the extra length should not be counted towards the post size. The process by which the part lengths are added up DECODED is a much more accurate way to do it, in my opinion. How confusing would it be to a user who uploads a file that is 1,989,956 bytes to get notified that the file exceeded the 2 MB limit? The user certainly wouldn't understand that his file base64 encoded was larger than 2 MB. He would think the site was broken. N This e-mail may contain privileged or confidential information. If you are not the intended recipient: (1) you may not disclose, use, distribute, copy or rely upon this message or attachment(s); and (2) please notify the sender by reply e-mail, and then delete this message and its attachment(s). Underwriters Laboratories Inc. and its affiliates disclaim all liability for any errors, omissions, corruption or virus in this message or any attachments.
