RE: [Bug][Tomcat 5.5.x] Tomcat loses request parameters
Chris, Thank you for your answer. You are right about tomcat itself not being involved. I was fooled by the class ByteChunk belonging to tomcat, and didn't see that the MultipartStream belongs in fact to commons-fileupload.jar. The jbossweb-tomcat55.sar module in jboss 4.0.4.GA does come with commons-fileupload-1.2.jar. I found out that the bug I see was fixed in version 1.2.1 (issue FILEUPLOAD-144 related to issue FILEUPLOAD-135) and verified it solves my problem. Thanks again, Didier DUQUENNOY Didier, On 6/4/14, 11:30 AM, DDU DUQUENNOY Didier wrote: I think I found a problem in the way Tomcat parses the POST Http Request. It might be related to issues n°40860, 31523 and 42347 submitted to bugzilla https://issues.apache.org/bugzilla. I am running on a JBoss 4.0.4 GA server on a windows platform. I know it uses a 5.5.x tomcat, but I don't know exactly which version. Tomcat 5.5.x is no longer supported. We don't support JBoss, here, either. You'll have to get support from Red Hat at this point, or upgrade to a version of Tomcat that is supported. Symptoms : When I submit a POST request, sometimes one parameter get lost. I can tell using Eclipse's TCP/IP proxy that the value was submitted in the request. Without knowing what version of 5.5.x you are using, nobody is going to be able to tell you if a) there is a bug and b) if it was fixed in a version of Tomcat later than the one you are running. I can tell you pretty confidently that Tomcat does not lose request parameters, even old unsupported versions. ;) Analysis: The POST request looks like that and is 15kB long: -7de35e1c190e9e Content-Disposition: form-data; name=autoScroll 0,0 -7de35e1c190e9e Content-Disposition: form-data; name=principal:arbre::focused false -7de35e1c190e9e Content-Disposition: form-data; name=principal:arbre::expandedNodes It's worth pointing-out that Tomcat did not directly handle multipart requests until version 7.0.x. If you are having problems with multipart requests, the problem is likely with either /your/ (of JBoss's) multipart library or your own code. (Oddly, Tomcat does have the package-renamed classes from commons-http-upload available in the Tomcat 6 source...) Using remote debugging, I found that a MultipartStream object analyses the request using a 4KB buffer; this buffer is fed by a 8kB buffer used by a ByteChunk object. The pattern -7de35e1c190e9e is called 'boundary'. and is repeated for each field. I noted that the hex part of the boundary may not be the same length from one POST to another, I think that is why the problem does not always occur. when MultipartStream arrives at the end of its buffer and there is not enough bytes left to read the next boundary, it moves the few unread bytes to the beginning of the buffer, then loads the next chunk of data from the ByteChunk object (see MultipartStream$ItemInputStream.makeAvailable()) That class does not exist in Tomcat prior to Tomcat 6, and in Tomcat 6 does not have the makeAvailable method. In Tomcat 7, makeAvailable appears. I suspect you are using commons-http-upload, and probably a fairly old version. If this ever was a problem with commons-http-upload, I'm sure it's been fixed. In any case, you are going to have to upgrade at least commons-http-upload. I would recommend upgrading just about every component you currently have, as you are likely to be very out of date. Security patches, performance improvements, bug fixes, etc. are all available between your version and the current. - -chris Consultez nos sites internet : http://www.sofaxis.com http://www.sofcap-sofcah.com Sofaxis disclaimer : http://www.sofaxis.com/disclaimer-1.html - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Bug][Tomcat 5.5.x] Tomcat loses request parameters
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Didier, On 6/5/14, 4:10 AM, DDU DUQUENNOY Didier wrote: Chris, Thank you for your answer. You are right about tomcat itself not being involved. I was fooled by the class ByteChunk belonging to tomcat, and didn't see that the MultipartStream belongs in fact to commons-fileupload.jar. The jbossweb-tomcat55.sar module in jboss 4.0.4.GA does come with commons-fileupload-1.2.jar. I found out that the bug I see was fixed in version 1.2.1 (issue FILEUPLOAD-144 related to issue FILEUPLOAD-135) and verified it solves my problem. Glad to see it was only a point-version that you needed. Kind of a no-brainer :) I'm surprised that JBoss shipped with that broken version of commons-fileupload. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTkIr4AAoJEBzwKT+lPKRY1MgP/iiuu/72Ym/XD4j2K7n38tsK Ykum444vZkgp+XnyTMz3LDK6Y7UyHoxmJW4Yfbc0c6xI42ltrlekDjrnU7l7G+tc IXal7oSGhW8Y1jMg79TVzs/Uta9cz1U+sdSu2X/XuEWkcpJcmEH35ua+D/U4ihdL b1z7npiIHSfRrHPYvpdNxL9u5ohk98Pi1BrfIj96ADvgP4P92XPO090OBoVjTm2O q97Pz7Eer2KOApC7oDP65lnkPNzkmVVSRaminvKcSPQIVD39X7ZMoZs8pXVckm2/ qPlEb57EztI6Y4FL1rW5iQa2QqndyVGlrmKVgAyUGelQI04bDFidDKnZ8ZKXQ9tC 8rC41cr+0Iw58EGzBavwhLv6qQ+nBzviTWUUgyekUCMVhE5+SlryK+s3yZH3T6X7 R6vJSA+wrsFzwZyU3DK4plz0NCAeA5x7e5amuNHpKMwDpyaCwvxgnVxr0KjljEie i/TexZhQno7W5cWH5JhlBvGi8LYCR9aTer255U5bKmGv2EeCyqIu69sY5Ytuyf1k GaIEyTjXjHaUzw8lsA9uji0ge86JiQ3oX0ezwDiJx8donR3U/NGQEpf2ah16KcAw jXEhp2N6IH6u733FbDMS6aQdo1bBa/sg7yMndvBWnwdrTcZr462I8KTR9COTdcZE M8XmQbATZuRnzJplaSc7 =zIfA -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
[Bug][Tomcat 5.5.x] Tomcat loses request parameters
Hi,
I think I found a problem in the way Tomcat parses the POST Http Request. It
might be related to issues n°40860, 31523 and 42347 submitted to bugzilla
https://issues.apache.org/bugzilla.
I am running on a JBoss 4.0.4 GA server on a windows platform. I know it uses a
5.5.x tomcat, but I don't know exactly which version.
Symptoms :
When I submit a POST request, sometimes one parameter get lost. I can tell
using Eclipse's TCP/IP proxy that the value was submitted in the request.
Analysis:
The POST request looks like that and is 15kB long:
-7de35e1c190e9e
Content-Disposition: form-data; name=autoScroll
0,0
-7de35e1c190e9e
Content-Disposition: form-data; name=principal:arbre::focused
false
-7de35e1c190e9e
Content-Disposition: form-data; name=principal:arbre::expandedNodes
Using remote debugging, I found that a MultipartStream object analyses the
request using a 4KB buffer; this buffer is fed by a 8kB buffer used by a
ByteChunk object.
The pattern -7de35e1c190e9e is called 'boundary'.
and is repeated for each field. I noted that the hex part of the boundary may
not be the same length from one POST to another, I think that is why the
problem does not always occur.
when MultipartStream arrives at the end of its buffer and there is not enough
bytes left to read the next boundary, it moves the few unread bytes to the
beginning of the buffer, then loads the next chunk of data from the ByteChunk
object (see MultipartStream$ItemInputStream.makeAvailable())
ex : the buffer ends with false\n\r. These bytes are moved to the start
of the buffer, so that the buffer now starts with
false\n\r-7de35e1c190e9e[...] and the code can
now read the boundary.
In this example, after parsing the first 4096 bytes, we now parse the
(4096-11)=4085 next bytes (because we still have the 11 last bytes from the
first chunk.
At the end of the second chunk : the buffer now ends with something like
0,0\n\r-- (7 bytes) and we need the next chunk of data to read the next
boundary. Tomcats asks the ByteChunk object for the next (4096-7) bytes, but
there are only 11 available (8kB-4kB-4085 bytes). So MultipartStream only sees
a 18 bytes-long data, which is not enough to read the boundary. As a
consequence, the field is skipped and is value is lost.
So the problem, I think, relies in the ByteChunk.substract method, when the
available bytes in the buffer are not enough to fill the MultipartStream need.
This method should attempt to read the next chunk of data if available.
Solution :
Here is the modification I suggest in the code:
public int substract( byte src[], int off, int len )
throws IOException {
if ((end - start) == 0) {
if (in == null)
return -1;
int n = in.realReadBytes( buff, 0, buff.length );
if (n 0)
return -1;
}
int n = len;
if (len getLength()) {
n = getLength();
}
System.arraycopy(buff, start, src, off, n);
start += n;
// Start of modification - add
if (nlen) {
// not enough data in the buffer. We want more!
int n2 = substract(src, off+n, len-n);
if (n20) {
return n+n2;
}
}
// End of modification
return n;
}
below is a part of the stack trace to the point where I located the problem:
Daemon Thread [http-0.0.0.0-8080-5] (Suspended (breakpoint at line 404 in
ByteChunk))
ByteChunk.substract(byte[], int, int) line: 404
InputBuffer.read(byte[], int, int) line: 299
CoyoteInputStream.read(byte[], int, int) line: 192
MultipartStream$ItemInputStream.makeAvailable() line: 959
MultipartStream$ItemInputStream.read(byte[], int, int) line: 887
MultipartStream$ItemInputStream(InputStream).read(byte[]) line: 89
Streams.copy(InputStream, OutputStream, boolean, byte[]) line: 94
Streams.copy(InputStream, OutputStream, boolean) line: 64
MultipartStream.readBodyData(OutputStream) line: 593
MultipartStream.discardBodyData() line: 619
MultipartStream.skipPreamble() line: 638
FileUploadBase$FileItemIteratorImpl.findNextItem() line: 844
FileUploadBase$FileItemIteratorImpl.init(FileUploadBase,
RequestContext) line: 825
DiskFileUpload(FileUploadBase).getItemIterator(RequestContext) line:
323
DiskFileUpload(FileUploadBase).parseRequest(RequestContext) line: 341
DiskFileUpload(FileUploadBase).parseRequest(HttpServletRequest) line:
302
MultipartRequestWrapper.parseRequest() line: 85
MultipartRequestWrapper.getParameter(String) line: 181
Re: [Bug][Tomcat 5.5.x] Tomcat loses request parameters
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Didier, On 6/4/14, 11:30 AM, DDU DUQUENNOY Didier wrote: I think I found a problem in the way Tomcat parses the POST Http Request. It might be related to issues n°40860, 31523 and 42347 submitted to bugzilla https://issues.apache.org/bugzilla. I am running on a JBoss 4.0.4 GA server on a windows platform. I know it uses a 5.5.x tomcat, but I don't know exactly which version. Tomcat 5.5.x is no longer supported. We don't support JBoss, here, either. You'll have to get support from Red Hat at this point, or upgrade to a version of Tomcat that is supported. Symptoms : When I submit a POST request, sometimes one parameter get lost. I can tell using Eclipse's TCP/IP proxy that the value was submitted in the request. Without knowing what version of 5.5.x you are using, nobody is going to be able to tell you if a) there is a bug and b) if it was fixed in a version of Tomcat later than the one you are running. I can tell you pretty confidently that Tomcat does not lose request parameters, even old unsupported versions. ;) Analysis: The POST request looks like that and is 15kB long: -7de35e1c190e9e Content-Disposition: form-data; name=autoScroll 0,0 -7de35e1c190e9e Content-Disposition: form-data; name=principal:arbre::focused false -7de35e1c190e9e Content-Disposition: form-data; name=principal:arbre::expandedNodes It's worth pointing-out that Tomcat did not directly handle multipart requests until version 7.0.x. If you are having problems with multipart requests, the problem is likely with either /your/ (of JBoss's) multipart library or your own code. (Oddly, Tomcat does have the package-renamed classes from commons-http-upload available in the Tomcat 6 source...) Using remote debugging, I found that a MultipartStream object analyses the request using a 4KB buffer; this buffer is fed by a 8kB buffer used by a ByteChunk object. The pattern -7de35e1c190e9e is called 'boundary'. and is repeated for each field. I noted that the hex part of the boundary may not be the same length from one POST to another, I think that is why the problem does not always occur. when MultipartStream arrives at the end of its buffer and there is not enough bytes left to read the next boundary, it moves the few unread bytes to the beginning of the buffer, then loads the next chunk of data from the ByteChunk object (see MultipartStream$ItemInputStream.makeAvailable()) That class does not exist in Tomcat prior to Tomcat 6, and in Tomcat 6 does not have the makeAvailable method. In Tomcat 7, makeAvailable appears. I suspect you are using commons-http-upload, and probably a fairly old version. If this ever was a problem with commons-http-upload, I'm sure it's been fixed. In any case, you are going to have to upgrade at least commons-http-upload. I would recommend upgrading just about every component you currently have, as you are likely to be very out of date. Security patches, performance improvements, bug fixes, etc. are all available between your version and the current. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBCAAGBQJTj7StAAoJEBzwKT+lPKRYrmQP/iDgbuW2j0UxCn6IM2Skra4z PUIj1O4tBMz4Q668sjxboJHpMHYDPsW8uUtChIFH2UbyxQ8yonqjhugybPE3jBf3 jwmw+tk+x8FbW+tiGeQl/KxPdvgo5l7i51Hb7/SA8Wzhu82gGm9J3zjo5g7FhMje 8H8T9qMpSijUJO1QLfnNiGuXeLlDVOAL35lTihW+rTiuFjn+hy/rqwM01lGMyw0n hrh8d9ZHFZGorDBoIFnPc+Jbo+qhI+wZB/tgPgYFQrk1MTBgmvbiJ0Zd4wmV7mGj fOMCQF3Xbf8iJvkQBVq8C+17Yr/AHFH96t6zDCWfPLdwWIUtRVOA86EYvmxqd4cb lWBdPvf5o3+MCYHp4nX/JSMgD7oAKwZYsi0rTSj9MadWUHSgkZT31zD8VCQET5WQ M7BuAI8P3rsMvopCEF+eLLgjc5wOvqoW/egrldkenxeNA1EN7g8mqiFWIkDXcoh4 +mNciaag3fk0XmkmLfl9MGrGc+/81aQBoa2R1ge0QIZFGEhUnrLrVESCPwyZGfvB Wwpdy86KUTIJD1nwW5v4ghmDhVdYs3r3NYbcBsGv927UXSnsaFsoWpAzz7xYJNn7 JUG23HcPgG7Ff8jO1osE9AjLyua9hJKgxVM9vzwj1ctCS1TlZ4EAlEoufnJY5OOF 7AEjGq+cRV57F04cdGw2 =vxm6 -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
