subject:"file upload"

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

I've filed this in Bugzilla:

https://bz.apache.org/bugzilla/show_bug.cgi?id=64384

- -chris


On 4/27/20 12:10, Christopher Schultz wrote:
> Mark,
>
> On 4/27/20 12:02, Christopher Schultz wrote:
>> Mark,
>
>> On 4/27/20 11:30, Christopher Schultz wrote:
>>> Mark,
>
>>> On 3/30/20 16:51, Mark Thomas wrote:
>>>> On 30/03/2020 21:45, Christopher Schultz wrote:
>>>>> All,
>>>>>
>>>>> In my application under Tomcat 8.5.51, I have configured a
>>>>> servlet to allow multipart/form-data submissions and I
>>>>> have added this configuration as a part of the 
>>>>> config:
>>>>>
>>>>> 
>>>>> 1048576
>>>>> 1049600 
>>>>>
>>>>> Without the  section, the upload does
>>>>> not work at all, so I know I have added this in the right
>>>>> place.
>>>>>
>>>>> But I am able to upload files larger than 1MiB, and the
>>>>> data is being given to the servlet. I was expecting an
>>>>> error to be sent to the client (unlikely) or the data to be
>>>>> suppressed from the servlet, or some kind of indication to
>>>>> the servlet code that the upload was too big.
>>>>>
>>>>> The file I'm uploading as a test is 13658819 bytes, which
>>>>> is greater than both 1048576 and 1049600.
>>>>>
>>>>> What am I missing, here?
>
>>>> Are you reading the request body directly? That will bypass
>>>> the size checks.
>
>>>> If that doesn't explain it, I'd fire up a remote debugger,
>>>> debug through an upload and see why the size checks are
>>>> skipped.
>
>>> I finally had an opportunity to debug this.
>
>>> First of all, part of the problem was that Struts was
>>> intercepting the call which made debugging a little confusing.
>>> Tomcat was parsing the request, but it looked like Struts was
>>> *also* trying to parse it, which ended up with a deeper tree
>>> of wrapped request objects than necessary.
>
>>> Once I got Struts out of the way, I was able to determine that
>>> every multipart part was being written to the disk,
>>> temporarily, even the one-byte request parameters and stuff
>>> like that. Yuck, and oops.
>
>>> That was happening because I had set no 
>>> and so it defaulted to 0 bytes.
>
>>> Setting a  to something reasonable (I
>>> chose 1024 bytes) ended up immediately having Tomcat reject my
>>> known-too-large requests with HTTP 413 "Payload Too Large".
>
>>> So this is good: Tomcat is indeed complaining about the size
>>> of the request. However, it didn't do it until I set a
>>> non-zero . This is my current
>>> configuration in web.xml:
>
>>>  1048576 1049600
>>> 1024
>>> 
>
>>> With the  removed, Tomcat will happily
>>> process a 30MiB file upload, which I didn't expect.
>
>>> I'm going to try to recreate this with a trivial web-app and
>>> file a bug, because I don't think this is how it's expected to
>>> behave.
>
>> Interestingly, specifying:
>
>> 0
>
>> Causes things to work as expected as well. So it's not *merely*
>> a zero file-size-threshold. It's specifically a missing one
>> which causes the default to apply.
>
> - From the debugger, going up the tree from my servlet, I find the
> multipartConfigElement that ends up being set on my servlet
> wrapper has none of my configuration:
>
> multipartConfigElementMultipartConfigElement  (id=545)
> fileSizeThreshold 0 location  "" (id=550) maxFileSize -1
> maxRequestSize-1
>
> The code in WebRuleSet looks okay at first glance.
>
> -chris
>
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6nDFkACgkQHPApP6U8
pFg3OxAAhvRIg8M3Db/1CvgjWhX+qD7O4uHJokH+DB8dN81m2IOg93xe7xASVO2v
A/SLJYa0cynDyAAfzrEqutu4CmaPy+CpcQH9hgekk3+4vSpljdWKbBLFoXsuUrJ6
ns0U+JPvaHWpvjcAFopUrGeLNzBxeRl3fIuWDG3LKl+9vwE0l00esLNHQgsooQ/q
H7nWfopYgiWEm8p0CJ9jxw7WnobxJzrEz49C43M1dPIrx2nK9WAWWVkevsM+yWIj
2jLW+FnxtKOQCEpEgkvJTrq3BYsHaUih85WH3vIUFr+uOAi2jCmstVENhEGo1GDW
NswcHCkiD0bjejUOXA9Gx/erZ+8+w3nMzVej8ziRxYMFI4wroPwZzkpAyvNuUEWq
8FfTalKgKWOGgx5eMDG5NW3ndEFVzMdDxmZJpw8SBcav/k4ZlOQSIVU/J/8C9huC
cPBkmsoWx0h2bdK08GUAP3a7SCGjXsVDV4nro60GL12++upDR+wDsEjAMWwn7x2H
baJDSowjUb5kbJvcx3BsfhJgWTg/Zo8O234Xi+kiph64xFL+lZCfnQ3XqrK2j93c
OcrHMZcbG1c77AN+RrQCItmYjScIPhGXApvqBBfgKzappGrX7AZDKrm3bxX8BxSN
QgxMyy4Ya9JyHyQSeL1D7BexxY1sF8DW9+MqX9tO06kSFbuWSNo=
=qh4+
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Limiting multipart file upload sizes

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 4/27/20 12:02, Christopher Schultz wrote:
> Mark,
>
> On 4/27/20 11:30, Christopher Schultz wrote:
>> Mark,
>
>> On 3/30/20 16:51, Mark Thomas wrote:
>>> On 30/03/2020 21:45, Christopher Schultz wrote:
>>>> All,
>>>>
>>>> In my application under Tomcat 8.5.51, I have configured a
>>>> servlet to allow multipart/form-data submissions and I have
>>>> added this configuration as a part of the  config:
>>>>
>>>> 
>>>> 1048576
>>>> 1049600 
>>>>
>>>> Without the  section, the upload does not
>>>> work at all, so I know I have added this in the right place.
>>>>
>>>> But I am able to upload files larger than 1MiB, and the data
>>>> is being given to the servlet. I was expecting an error to
>>>> be sent to the client (unlikely) or the data to be suppressed
>>>> from the servlet, or some kind of indication to the servlet
>>>> code that the upload was too big.
>>>>
>>>> The file I'm uploading as a test is 13658819 bytes, which is
>>>> greater than both 1048576 and 1049600.
>>>>
>>>> What am I missing, here?
>
>>> Are you reading the request body directly? That will bypass
>>> the size checks.
>
>>> If that doesn't explain it, I'd fire up a remote debugger,
>>> debug through an upload and see why the size checks are
>>> skipped.
>
>> I finally had an opportunity to debug this.
>
>> First of all, part of the problem was that Struts was
>> intercepting the call which made debugging a little confusing.
>> Tomcat was parsing the request, but it looked like Struts was
>> *also* trying to parse it, which ended up with a deeper tree of
>> wrapped request objects than necessary.
>
>> Once I got Struts out of the way, I was able to determine that
>> every multipart part was being written to the disk, temporarily,
>> even the one-byte request parameters and stuff like that. Yuck,
>> and oops.
>
>> That was happening because I had set no 
>> and so it defaulted to 0 bytes.
>
>> Setting a  to something reasonable (I chose
>> 1024 bytes) ended up immediately having Tomcat reject my
>> known-too-large requests with HTTP 413 "Payload Too Large".
>
>> So this is good: Tomcat is indeed complaining about the size of
>> the request. However, it didn't do it until I set a non-zero
>> . This is my current configuration in
>> web.xml:
>
>>  1048576 1049600 1024 
>
>> With the  removed, Tomcat will happily
>> process a 30MiB file upload, which I didn't expect.
>
>> I'm going to try to recreate this with a trivial web-app and
>> file a bug, because I don't think this is how it's expected to
>> behave.
>
> Interestingly, specifying:
>
> 0
>
> Causes things to work as expected as well. So it's not *merely* a
> zero file-size-threshold. It's specifically a missing one which
> causes the default to apply.

- From the debugger, going up the tree from my servlet, I find the
multipartConfigElement that ends up being set on my servlet wrapper
has none of my configuration:

multipartConfigElement  MultipartConfigElement  (id=545)
fileSizeThreshold   0   
location"" (id=550) 
maxFileSize -1  
maxRequestSize  -1  

The code in WebRuleSet looks okay at first glance.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6nA9sACgkQHPApP6U8
pFhX+A/8Dae5N3J+1lqeM2RgfQsj/Tp/oOzSNIoJO2420qwbcNe7XY896k1MnFMW
fEA6QE6M2GLmAwcpiZYwsw3xm2xzF3TRr8JYEwV2FDnETUylJ0kdDZQz7kiaxlMZ
7s/5/r69DnplUtHlx9/JBTOgICmxMbR2QMIKzGI2iaQpYHFkqoAR/h54IzTvlprF
XzUvPk0np2ZHADYn99JL7uLKr0UrBWqaxvY62LAg+GA0TrRQJKZs8ijjKkd0l1HA
D5gqZcZBqaANSv7q7ogCQGjqWK57gAyrAW2px1ySsH8XvdZCmn0Qbl10Wj9EhiZ3
a0lNpEBeMfizo2rz/+IOS8nxHmb+krjxK25AWXjwbyrG8zMR7X3L27MsqNVYFnCc
m0kmEAwk+Ly0o06MitZ9kcL5AvY8AkQQ9PoudjQRdx9QEqQw6irY6WT6wP6qaYVL
3nzX1dGc+yWdinngLe3AumP0uHYRqTI50JQ5RJs1w/F218jzskQGt8waswM1wNeB
kOsLNkxr3GErreuq3TiqMe0/f+Lof2bV2W63Gu+1lzy+hKtSsvsI2ZNGzMvB06TA
Le4GaqJHZkjGH64QJTprL94hSz96xtAhZfTipI+Y+5tNA48gYM5lOZH+e6PJHu9l
g4MDZpCUqsdPoZ8j7n7QaLM/b1ey2NQYDQt+SiPzN71XqbINrH4=
=cYyi
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Limiting multipart file upload sizes

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 4/27/20 11:30, Christopher Schultz wrote:
> Mark,
>
> On 3/30/20 16:51, Mark Thomas wrote:
>> On 30/03/2020 21:45, Christopher Schultz wrote:
>>> All,
>>>
>>> In my application under Tomcat 8.5.51, I have configured a
>>> servlet to allow multipart/form-data submissions and I have
>>> added this configuration as a part of the  config:
>>>
>>>  1048576 1049600 
>>>
>>> Without the  section, the upload does not
>>> work at all, so I know I have added this in the right place.
>>>
>>> But I am able to upload files larger than 1MiB, and the data
>>> is being given to the servlet. I was expecting an error to be
>>> sent to the client (unlikely) or the data to be suppressed from
>>> the servlet, or some kind of indication to the servlet code
>>> that the upload was too big.
>>>
>>> The file I'm uploading as a test is 13658819 bytes, which is
>>> greater than both 1048576 and 1049600.
>>>
>>> What am I missing, here?
>
>> Are you reading the request body directly? That will bypass the
>> size checks.
>
>> If that doesn't explain it, I'd fire up a remote debugger, debug
>> through an upload and see why the size checks are skipped.
>
> I finally had an opportunity to debug this.
>
> First of all, part of the problem was that Struts was intercepting
> the call which made debugging a little confusing. Tomcat was
> parsing the request, but it looked like Struts was *also* trying to
> parse it, which ended up with a deeper tree of wrapped request
> objects than necessary.
>
> Once I got Struts out of the way, I was able to determine that
> every multipart part was being written to the disk, temporarily,
> even the one-byte request parameters and stuff like that. Yuck, and
> oops.
>
> That was happening because I had set no  and
> so it defaulted to 0 bytes.
>
> Setting a  to something reasonable (I chose
> 1024 bytes) ended up immediately having Tomcat reject my
> known-too-large requests with HTTP 413 "Payload Too Large".
>
> So this is good: Tomcat is indeed complaining about the size of
> the request. However, it didn't do it until I set a non-zero
> . This is my current configuration in
> web.xml:
>
>  1048576 1049600 1024
> 
>
> With the  removed, Tomcat will happily process
> a 30MiB file upload, which I didn't expect.
>
> I'm going to try to recreate this with a trivial web-app and file
> a bug, because I don't think this is how it's expected to behave.

Interestingly, specifying:

  0

Causes things to work as expected as well. So it's not *merely* a zero
file-size-threshold. It's specifically a missing one which causes the
default to apply.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6nAiIACgkQHPApP6U8
pFhmsBAAwavmRuXPmHO2at/cJTs4x21x+qMNT+qaJPA/O/KGnYr9x1zLSyCGq5vo
JYMwp9zrS2mrgOTIXQCUdby0+DwLe/HovNaSDeyU8M7scMAJuOg4Td6DwtxcOtO+
AAbqIY1EE6hPBr3NkqVvhaAQCP4ITOojUE4RZhtTXIYjy16ekreFuJpUf/wJZAjY
ptSwHKxE+KmLUU847jfts3qNaDnhTDnJ0BgeVfQSB3kmk8ZOfOW6FSFBMymrPrHv
SEGNTOXqnuj7ki6ej98MIOpNQHoRle0cQmkOlKlYRwlvcVkSFWBKfwoeUeTKSvvL
FrKk0ulbDVlVLCYQN2CL4V76iQenTYn3ngoc49yaoN1R8NrAmsJOuc+4vnkz/CUC
bmAawKztStmYP/qE+WxFSreBwLiFBg7h7pQFQ5nIzBan8Js/ooyyWZlu9fCqbdmV
7o1qOw17NpRZETIli4v7doXJOqshrMY/bZfpM/p3Y/pTrOd+W1UxUdY0y2ggfmRJ
+RU0foupVpk1BEN9VhSpSfJlfmasOdV84fq/M1xKLBmFspn5a5uLBwRtZvMntx1h
qwsVjaMzmKFa77F6aGS4O1T9r4fDSdxcLAt66SXP1O/Pdds3qjvlMBps0pxOI7JJ
0gs4IdbRdT/3ApfjrvhQ6ZIfVVpoYioxW+zpaoF4s92rcIC2378=
=fX7U
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Limiting multipart file upload sizes

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 3/30/20 16:51, Mark Thomas wrote:
> On 30/03/2020 21:45, Christopher Schultz wrote:
>> All,
>>
>> In my application under Tomcat 8.5.51, I have configured a
>> servlet to allow multipart/form-data submissions and I have added
>> this configuration as a part of the  config:
>>
>>  1048576 1049600 
>>
>> Without the  section, the upload does not work
>> at all, so I know I have added this in the right place.
>>
>> But I am able to upload files larger than 1MiB, and the data is
>> being given to the servlet. I was expecting an error to be sent
>> to the client (unlikely) or the data to be suppressed from the
>> servlet, or some kind of indication to the servlet code that the
>> upload was too big.
>>
>> The file I'm uploading as a test is 13658819 bytes, which is
>> greater than both 1048576 and 1049600.
>>
>> What am I missing, here?
>
> Are you reading the request body directly? That will bypass the
> size checks.
>
> If that doesn't explain it, I'd fire up a remote debugger, debug
> through an upload and see why the size checks are skipped.

I finally had an opportunity to debug this.

First of all, part of the problem was that Struts was intercepting the
call which made debugging a little confusing. Tomcat was parsing the
request, but it looked like Struts was *also* trying to parse it,
which ended up with a deeper tree of wrapped request objects than
necessary.

Once I got Struts out of the way, I was able to determine that every
multipart part was being written to the disk, temporarily, even the
one-byte request parameters and stuff like that. Yuck, and oops.

That was happening because I had set no  and so
it defaulted to 0 bytes.

Setting a  to something reasonable (I chose 1024
bytes) ended up immediately having Tomcat reject my known-too-large
requests with HTTP 413 "Payload Too Large".

So this is good: Tomcat is indeed complaining about the size of the
request. However, it didn't do it until I set a non-zero
. This is my current configuration in web.xml:

  1048576
  1049600
  1024

With the  removed, Tomcat will happily process a
30MiB file upload, which I didn't expect.

I'm going to try to recreate this with a trivial web-app and file a
bug, because I don't think this is how it's expected to behave.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6m+nkACgkQHPApP6U8
pFhjPw//Xo1veUY5zkczUQ01MjoTbR8awZxzTFHDZlWGFyJUVJutRMz+QbYxUkPO
sPovYR2uG3uygLv37cLXOSyQ5NxImmxWaKMhe+Vcjd779Tg1+f7AqknBrbUWMbRo
IkUsVUQemRsNg0ZDdyva+jyQUcB3hirFMy4JLwKbjSvSYLWtgCH+siql3tyOsDQx
noaTypqBK42C7i/nq1sBDT0vsyV+iHLJyP6bHKOWKt+dH+EmOPTg0mqlsHV3Hvsd
J9DTdqZU4fKh3Zxs+WA9mIJlcK5cPFvLEjP73WwnpGegGz8TDoF/bAz0zc3V/ibK
XEFFaO1i8cyohNd9dZtWLKa+6fQvIXpR/I7/TUoMSct6SM21JPBXBEfMVpyZ2EW8
fDOO4PO3IYB1tYUxwo6ovpx8kLfOMRQ3VgR71mvPirdVlsINakV6aAvN8uitCwDt
zF5zYl6Ef8+WpSKv7Y6ZS7K7xY3QCQMRf8fU5WDeooKp2+bKwtBMHLPsYRu035+E
0oTuNExN4qi+rv4VagOSwa685s51EIFlt26lC/5Jtsy7L30DqQkHLVeMKbLgz+pa
VFwnAwRm/uGbb7b9aLTNsl+bkjjmYGh9E9uC7wRdYIyejFwJPqpd3h0ByoR73ZeC
JVtqoZsGVY7cUSGQNJ4DoHSqEHlG5jt8oLvoLhZJ3ulP5AQ5bNU=
=uAP7
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Limiting multipart file upload sizes

2020-03-30 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 3/30/20 16:51, Mark Thomas wrote:
> On 30/03/2020 21:45, Christopher Schultz wrote:
>> All,
>>
>> In my application under Tomcat 8.5.51, I have configured a
>> servlet to allow multipart/form-data submissions and I have added
>> this configuration as a part of the  config:
>>
>>  1048576 1049600 
>>
>> Without the  section, the upload does not work
>> at all, so I know I have added this in the right place.
>>
>> But I am able to upload files larger than 1MiB, and the data is
>> being given to the servlet. I was expecting an error to be sent
>> to the client (unlikely) or the data to be suppressed from the
>> servlet, or some kind of indication to the servlet code that the
>> upload was too big.
>>
>> The file I'm uploading as a test is 13658819 bytes, which is
>> greater than both 1048576 and 1049600.
>>
>> What am I missing, here?
>
> Are you reading the request body directly? That will bypass the
> size checks.

Nope. The order of calls in my servlet (actually a Struts action, but
there shouldn't be much in the way of interference, there) is:

getContentType
getAttributeNames (for debugging; I was expecting to
getAttribute   see an attribute saying "too big" or something)
getSession
getParameter (a few times)
getCharacterEncoding
getParts

The file definitely has data, in it, too. I'm uploading a file much
larger than expected just as a test, so I don't care what it is. I'm
uploading a tarball as a CSV and my servlet says "umm, that ain't CSV"
and logs the first few bytes of the file (and they aren't null or
empty or whatever I might expect if Tomcat were rejecting the upload).

> If that doesn't explain it, I'd fire up a remote debugger, debug
> through an upload and see why the size checks are skipped.

Time to figure out how to attach a debugger :) Fortunately, I've got
everything running on my own laptop, so I don't have to instrument a
server somewhere else.

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6CbecACgkQHPApP6U8
pFitFA/8CjVZrWTbxnSTp7EiNyCYDJWEZyWr8O4PAqZCp9csIwKoKEhAG087UHnc
g+Br8UPaH8DejOSCDu5X9HuTkjhtsDfu9l5lsnXcp1dTy4DzWfAPgdKi23i6o94y
BpAiRESMxT3PJ6mMbKae0auiPAVZ1K5WJ2m3rKC7dPGbqNlqaiAw1ZMoTWVfJNix
wAKTGEAIkJSo687qyoqTNEBjS0AeqFln2HvHkErHb4rt79uy1R0bL4QbnptED/h6
UihnUSqckn9F2R67f7duQeXQBvtmIFDVBegY0iZF/pyBnR4ifogv7V2QwWJikc8W
SkmMF9ZC2LWa0pXjvS4nVQNFv1Bg9tWYUZJFRqTr2lqqUkkEKqiWrGRNgmX8P/mx
6sNnGR+dc+q6L3Xp5ujdo85Hc46xuv7kUrW9SSpxkDCmhIiefPf+6HY5qh+1kvNW
vMn2Bz1PIK6+TbkgFFcJlrTns2kL9mea64dwyKDuJxdt2o/TLGJpA/Z6Uocieh9+
6cP3J3oy9WuuxlIq3q6jwnJRua41gDIXiEyLug4iVBCBJzGO7kUsvOUEze9L1r3N
IZWwLDCnPwg6QdWFrP9K7Kp7TqLHjvBF2hOLJHmCAAJlx21FIA5OscpVevKlVOuS
S0DvdunN8uj9biv/pueMZmFUl1XCvgLw+D8JpGEgAUHbdhJwIQw=
=pwCh
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Limiting multipart file upload sizes

2020-03-30 Thread Mark Thomas

On 30/03/2020 21:45, Christopher Schultz wrote:
> All,
> 
> In my application under Tomcat 8.5.51, I have configured a servlet to
> allow multipart/form-data submissions and I have added this
> configuration as a part of the  config:
> 
> 
>   1048576
>   1049600
> 
> 
> Without the  section, the upload does not work at
> all, so I know I have added this in the right place.
> 
> But I am able to upload files larger than 1MiB, and the data is being
> given to the servlet. I was expecting an error to be sent to the
> client (unlikely) or the data to be suppressed from the servlet, or
> some kind of indication to the servlet code that the upload was too big.
> 
> The file I'm uploading as a test is 13658819 bytes, which is greater
> than both 1048576 and 1049600.
> 
> What am I missing, here?

Are you reading the request body directly? That will bypass the size checks.

If that doesn't explain it, I'd fire up a remote debugger, debug through
an upload and see why the size checks are skipped.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Limiting multipart file upload sizes

2020-03-30 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

All,

In my application under Tomcat 8.5.51, I have configured a servlet to
allow multipart/form-data submissions and I have added this
configuration as a part of the  config:


  1048576
  1049600


Without the  section, the upload does not work at
all, so I know I have added this in the right place.

But I am able to upload files larger than 1MiB, and the data is being
given to the servlet. I was expecting an error to be sent to the
client (unlikely) or the data to be suppressed from the servlet, or
some kind of indication to the servlet code that the upload was too big.

The file I'm uploading as a test is 13658819 bytes, which is greater
than both 1048576 and 1049600.

What am I missing, here?

- -chris
-BEGIN PGP SIGNATURE-
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl6CWnEACgkQHPApP6U8
pFi1tRAAvs2wnlIgmfTeBpWWWFXh+vP+hOdgqW+7u/WiSb5viXvnYfoa+ezpiy7S
XHzctsBoQhs4Q+ErC1Ikz41btnuGbppnFe97mVplGDyca7aknxxdiLBP11azrD++
Ojn37bHjVFg1YzDFNDIJgG+yQC6/P0iFiMAUUsU1LM9oAOupLfiDyn+H6KLs/yV8
Z/NteP9GDgLvfNdmN1c+2gtAn0diRYOSvIOStXyS2Bsa1kueAu9zjIoHLMZExkdi
TrbmOBZV6/dR4LDP5/ZtIfuKA9urTIfMIpXqufyZh4A0Dk9QcDZMeh2t1txaAmEU
cRCwJkOIlkjq0GYAp8PvaTOSguvaw0om4d2z4V/l0Htov4bvW7h6Tzfx2LGXm2e8
KoqtGkQ9DSUclH3b/ik/Urwxoher24TnN4RcZlTamIJj7JBI5+n+x0YVzqn501Zz
VDTiuHUdNqKX1eqDTRjish0OFUKU6rUipnMZNzVQnKtgHqm5qHvwJan1r/chpQ1f
t4Fyujvs6N2q/K9N6gbvi+MY+hqlEyquOffuC6SY0dZGJkLXBffDqrMZyheOz+5h
RRxGqnrs7ZE2pme5KUdaYFmdDk1gUUph/ezudj1zuQNWVgx/W5T7J/+VuP2SAgoB
oKh5dHtRqCSPQq/Gv79OxxtzD/b5OcimvX2gGvq/OhCmMdYuVv0=
=kWh7
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Async file upload server has 33% less throughput compared to the sync version

2019-12-04 Thread Mark Thomas

On 04/12/2019 02:23, Behrang Saeedzadeh wrote:
> I am not expecting it to take less time, but to provide better throughput.

Despite what you appear to think, you have written two synchronous,
blocking I/O upload servlets. Not one synchronous and one
non-synchronous. With synchronous I/O less time == more throughput.

> With 1000 concurrent users, the sync version was still performing better
> (mean req/second).

Entirely as expected.

An asynchronous approach is not magic pixie dust that automatically
improves performance. In some scenarios it will make performance
(throughput and time per request) worse.

Neither is writing an asynchronous upload servlet as simple as taking
all the code from your synchronous doPost() method and surrounding it with:
asyncContext.start(() -> {

});

All that does is start a separate thread to do the synchronous, blocking
I/O you had in the original method. i.e. more server side work to
achieve exactly the same result.

For an asynchronous file upload servlet to provide improved throughput:

1. It needs to use non-blocking I/O. Your doesn't.

2. The clients need to be sending data sufficiently slowly that a thread
reading the upload with blocking I/O spends a reasonable amount of time
blocking, waiting for more data to arrive. Your tests don't do that.

Mark

> 
> 
> Best regards,
> Behrang Saeedzadeh
> (Sent from my cellphone.)
> 
> On Mon, 2 Dec. 2019, 8:44 am Mark Thomas,  wrote:
> 
>> On 01/12/2019 02:17, Behrang Saeedzadeh wrote:
>>
>> 
>>
>>> Any ideas what am I missing here?
>>
>> Async provides scalability, not raw performance.
>>
>> You haven't written a async file upload servlet. That would require
>> non-blocking I/O and look more like this:
>>
>> https://github.com/apache/tomcat/blob/master/webapps/examples/WEB-INF/classes/nonblocking/ByteCounter.java
>>
>> Step back from your code for a second. The async version does exactly
>> the same thing as the sync version apart from it does a bunch of *extra*
>> stuff (creating the Runnable and dispatching it to a different thread).
>> Why would you expect the code that does extra stuff, to take less time
>> when it does more?
>>
>> Mark
>>
>> -
>> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
>> For additional commands, e-mail: users-h...@tomcat.apache.org
>>
>>
> 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Async file upload server has 33% less throughput compared to the sync version

2019-12-03 Thread Behrang Saeedzadeh

I am not expecting it to take less time, but to provide better throughput.
With 1000 concurrent users, the sync version was still performing better
(mean req/second).


Best regards,
Behrang Saeedzadeh
(Sent from my cellphone.)

On Mon, 2 Dec. 2019, 8:44 am Mark Thomas,  wrote:

> On 01/12/2019 02:17, Behrang Saeedzadeh wrote:
>
> 
>
> > Any ideas what am I missing here?
>
> Async provides scalability, not raw performance.
>
> You haven't written a async file upload servlet. That would require
> non-blocking I/O and look more like this:
>
> https://github.com/apache/tomcat/blob/master/webapps/examples/WEB-INF/classes/nonblocking/ByteCounter.java
>
> Step back from your code for a second. The async version does exactly
> the same thing as the sync version apart from it does a bunch of *extra*
> stuff (creating the Runnable and dispatching it to a different thread).
> Why would you expect the code that does extra stuff, to take less time
> when it does more?
>
> Mark
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

Re: Async file upload server has 33% less throughput compared to the sync version

2019-12-01 Thread Mark Thomas

On 01/12/2019 02:17, Behrang Saeedzadeh wrote:



> Any ideas what am I missing here?

Async provides scalability, not raw performance.

You haven't written a async file upload servlet. That would require
non-blocking I/O and look more like this:
https://github.com/apache/tomcat/blob/master/webapps/examples/WEB-INF/classes/nonblocking/ByteCounter.java

Step back from your code for a second. The async version does exactly
the same thing as the sync version apart from it does a bunch of *extra*
stuff (creating the Runnable and dispatching it to a different thread).
Why would you expect the code that does extra stuff, to take less time
when it does more?

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Async file upload server has 33% less throughput compared to the sync version

2019-11-30 Thread Behrang Saeedzadeh

Source code with Gatling tests here (WIP):
https://github.com/turingg/file-server

I wanted to compare the performance/throughput of an async file upload
servlet to a sync version. To do that, I intentionally configured Tomcat to:

* Use at most 2 HTTP connector threads
* Accept up to 1000 connections
* With a queue length of 0



However despite that, my async servlet (
https://github.com/turingg/file-server/blob/master/src/main/java/xyz/behrang/fileserver/b/AsyncUploadServlet.java)
has 33% less throughput compared to the sync version ((
https://github.com/turingg/file-server/blob/master/src/main/java/xyz/behrang/fileserver/a/SyncUploadServlet.java
):

# Async

 Global Information

> request count   5000 (OK=5000   KO=0
)
> min response time218 (OK=218KO=-
)
> max response time   2234 (OK=2234   KO=-
)
> mean response time   674 (OK=674KO=-
)
> std deviation486 (OK=486KO=-
)
> response time 50th percentile464 (OK=464KO=-
)
> response time 75th percentile773 (OK=772KO=-
)
> response time 95th percentile   1890 (OK=1891   KO=-
)
> response time 99th percentile   2142 (OK=2142   KO=-
)
*> mean requests/sec555.556 (OK=555.556
KO=- )*
 Response Time Distribution

> t < 800 ms  3780 ( 76%)
> 800 ms < t < 1200 ms 525 ( 11%)
> t > 1200 ms  695 ( 14%)
> failed 0 (  0%)


# Sync


 Global Information

> request count   5000 (OK=5000   KO=0
)
> min response time  1 (OK=1  KO=-
)
> max response time   1617 (OK=1617   KO=-
)
> mean response time   207 (OK=207KO=-
)
> std deviation333 (OK=333KO=-
)
> response time 50th percentile 36 (OK=36 KO=-
)
> response time 75th percentile150 (OK=150KO=-
)
> response time 95th percentile992 (OK=992KO=-
)
> response time 99th percentile   1312 (OK=1312   KO=-
)
*> mean requests/sec833.333 (OK=833.333
KO=- )*
 Response Time Distribution

> t < 800 ms  4458 ( 89%)
> 800 ms < t < 1200 ms 479 ( 10%)
> t > 1200 ms   63 (  1%)
> failed 0 (  0%)


Any ideas what am I missing here?

Thanks in advance,
Behrang

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-04 Thread Umesh Sehgal


 On Wed, Mar 4, 2015 at 3:41 PM, Umesh Sehgal umesh.seh...@gmail.com
 wrote:



 On Tue, Mar 3, 2015 at 8:31 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

 Am 03.03.2015 um 13:45 schrieb Umesh Sehgal:

  On Mon, Mar 2, 2015 at 6:20 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

   On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com
 

 wrote:


   On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de
 

 wrote:

   Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:


   Thanks for the quick reply, I tried using the maxSwallowSize with

 increased
 value but to no effect. The max size that I have been able to
 upload is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update
 55
 but
 the behavior I'm seeing is 50 update onwards, is there any other
 param
 too?
 is there any logging that can be turned on tomcat to help debug?


  Please do not top post. For the rest see below.

On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung 
 rainer.j...@kippdata.de

  wrote:

Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:


 Hi,


  We recently upgraded our application's tomcat from 7.0.30 to
 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to
 nail
 it
 down to the point that the problem manifests 7.0.50 onwards.
 Here is
 the
 exception that I see inside logs:

   Caused by: java.net.SocketException: Connection reset by
 peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file
 but
 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could
 explain
 this
 behavior. Can someone please provide some pointer as what could
 be
 causing
 this?


   https://bz.apache.org/bugzilla/show_bug.cgi?id=57617


 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in
 the
 changelog


Could it be If a request that includes an Expect:
 100-continue header

 receives anything other than a 2xx response, close the connection
 This
 protects against misbehaving clients that may not sent the request
 body
 in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the
 first
 version with this change.


 Regards,

 Rainer


  I did see this in changelog but in the captured traffic don't see
 any
 expect 100 header request. Any other way I can confirm this on the
 tomcat
 side?

 Thanks,
 Umesh


  Can you please point me to SVN change for :
 If a request that includes an Expect: 100-continue header receives
 anything other than a 2xx response, close the connection This protects
 against misbehaving clients that may not sent the request body in that
 case
 and send the next request instead. (markt) ?


 http://svn.apache.org/r1540689


 Hi Rainer,


 Thanks, I think the problem is indeed caused by this change. I
 downloaded
 the tomcat source, removed the above change from AbstractHttp11Processor
 and delpoyed the updated jar. The file upload didn't work right away
 but at
 least now maxSwallowSize is honored and I can upload the files per the
 size
 specified.

 I did the above work to confirm, but of course I don't want to ship it
 carrying modified code. Can you please suggest as what could be done in
 this case?


 OK, good to know. I'd say now it would be good to find out why your
 webapp sends a non-2xx response code and hwich it is.

 Since you already suceeded to build tomcat, simply add a custom log or
 System.out.println() statement printing response.getStatus() where the
 change in r1540689 was added and tell us what it is for the failing uploads.

 Regards,

 Rainer


 I see 401 response code. Not sure why it's 401 but may be because of the
 way authorization is done.

 Thanks,
 Umesh


 But shouldn't it by any means swallow the remaining bytes before closing
 the connection?

 Thanks,

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org


 Investigating further around the 401 response returned by tomcat during
 Expect: 100-continue processing, I interestingly notice that if I use a
 different client (Abdera client) to post the request, there isn't any 401
 response for the Expect: 100-continue. However  for my apache common
 httpClient(3.1 version) there is a 401 in tomcat logs while processing
 Expect: 100-continue.


Is there any known problem with this combination? Anyone?

-Umesh

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-04 Thread Umesh Sehgal

On Tue, Mar 3, 2015 at 8:31 PM, Rainer Jung rainer.j...@kippdata.de wrote:

 Am 03.03.2015 um 13:45 schrieb Umesh Sehgal:

  On Mon, Mar 2, 2015 at 6:20 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

   On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com

 wrote:


   On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de

 wrote:

   Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:


   Thanks for the quick reply, I tried using the maxSwallowSize with

 increased
 value but to no effect. The max size that I have been able to upload
 is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update 55
 but
 the behavior I'm seeing is 50 update onwards, is there any other
 param
 too?
 is there any logging that can be turned on tomcat to help debug?


  Please do not top post. For the rest see below.

On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung 
 rainer.j...@kippdata.de

  wrote:

Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:


 Hi,


  We recently upgraded our application's tomcat from 7.0.30 to
 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to
 nail
 it
 down to the point that the problem manifests 7.0.50 onwards. Here
 is
 the
 exception that I see inside logs:

   Caused by: java.net.SocketException: Connection reset by
 peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but
 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could
 explain
 this
 behavior. Can someone please provide some pointer as what could be
 causing
 this?


   https://bz.apache.org/bugzilla/show_bug.cgi?id=57617


 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in the
 changelog


Could it be If a request that includes an Expect: 100-continue
 header

 receives anything other than a 2xx response, close the connection This
 protects against misbehaving clients that may not sent the request
 body
 in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the first
 version with this change.


 Regards,

 Rainer


  I did see this in changelog but in the captured traffic don't see any
 expect 100 header request. Any other way I can confirm this on the
 tomcat
 side?

 Thanks,
 Umesh


  Can you please point me to SVN change for :
 If a request that includes an Expect: 100-continue header receives
 anything other than a 2xx response, close the connection This protects
 against misbehaving clients that may not sent the request body in that
 case
 and send the next request instead. (markt) ?


 http://svn.apache.org/r1540689


 Hi Rainer,


 Thanks, I think the problem is indeed caused by this change. I downloaded
 the tomcat source, removed the above change from AbstractHttp11Processor
 and delpoyed the updated jar. The file upload didn't work right away but
 at
 least now maxSwallowSize is honored and I can upload the files per the
 size
 specified.

 I did the above work to confirm, but of course I don't want to ship it
 carrying modified code. Can you please suggest as what could be done in
 this case?


 OK, good to know. I'd say now it would be good to find out why your webapp
 sends a non-2xx response code and hwich it is.

 Since you already suceeded to build tomcat, simply add a custom log or
 System.out.println() statement printing response.getStatus() where the
 change in r1540689 was added and tell us what it is for the failing uploads.

 Regards,

 Rainer


 I see 401 response code. Not sure why it's 401 but may be because of the
way authorization is done.

Thanks,
Umesh

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-04 Thread Umesh Sehgal

On Wed, Mar 4, 2015 at 3:41 PM, Umesh Sehgal umesh.seh...@gmail.com wrote:



 On Tue, Mar 3, 2015 at 8:31 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

 Am 03.03.2015 um 13:45 schrieb Umesh Sehgal:

  On Mon, Mar 2, 2015 at 6:20 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

   On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com

 wrote:


   On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de

 wrote:

   Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:


   Thanks for the quick reply, I tried using the maxSwallowSize with

 increased
 value but to no effect. The max size that I have been able to
 upload is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update 55
 but
 the behavior I'm seeing is 50 update onwards, is there any other
 param
 too?
 is there any logging that can be turned on tomcat to help debug?


  Please do not top post. For the rest see below.

On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung 
 rainer.j...@kippdata.de

  wrote:

Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:


 Hi,


  We recently upgraded our application's tomcat from 7.0.30 to
 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to
 nail
 it
 down to the point that the problem manifests 7.0.50 onwards. Here
 is
 the
 exception that I see inside logs:

   Caused by: java.net.SocketException: Connection reset by
 peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but
 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could
 explain
 this
 behavior. Can someone please provide some pointer as what could be
 causing
 this?


   https://bz.apache.org/bugzilla/show_bug.cgi?id=57617


 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in
 the
 changelog


Could it be If a request that includes an Expect: 100-continue
 header

 receives anything other than a 2xx response, close the connection
 This
 protects against misbehaving clients that may not sent the request
 body
 in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the
 first
 version with this change.


 Regards,

 Rainer


  I did see this in changelog but in the captured traffic don't see
 any
 expect 100 header request. Any other way I can confirm this on the
 tomcat
 side?

 Thanks,
 Umesh


  Can you please point me to SVN change for :
 If a request that includes an Expect: 100-continue header receives
 anything other than a 2xx response, close the connection This protects
 against misbehaving clients that may not sent the request body in that
 case
 and send the next request instead. (markt) ?


 http://svn.apache.org/r1540689


 Hi Rainer,


 Thanks, I think the problem is indeed caused by this change. I downloaded
 the tomcat source, removed the above change from AbstractHttp11Processor
 and delpoyed the updated jar. The file upload didn't work right away but
 at
 least now maxSwallowSize is honored and I can upload the files per the
 size
 specified.

 I did the above work to confirm, but of course I don't want to ship it
 carrying modified code. Can you please suggest as what could be done in
 this case?


 OK, good to know. I'd say now it would be good to find out why your
 webapp sends a non-2xx response code and hwich it is.

 Since you already suceeded to build tomcat, simply add a custom log or
 System.out.println() statement printing response.getStatus() where the
 change in r1540689 was added and tell us what it is for the failing uploads.

 Regards,

 Rainer


 I see 401 response code. Not sure why it's 401 but may be because of the
 way authorization is done.

 Thanks,
 Umesh


But shouldn't it by any means swallow the remaining bytes before closing
the connection?

Thanks,

 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-03 Thread Umesh Sehgal

On Mon, Mar 2, 2015 at 6:20 PM, Rainer Jung rainer.j...@kippdata.de wrote:

 Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

  On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com
 wrote:


  On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:

  Thanks for the quick reply, I tried using the maxSwallowSize with
 increased
 value but to no effect. The max size that I have been able to upload is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update 55
 but
 the behavior I'm seeing is 50 update onwards, is there any other param
 too?
 is there any logging that can be turned on tomcat to help debug?


 Please do not top post. For the rest see below.

   On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de

 wrote:

   Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:


Hi,


 We recently upgraded our application's tomcat from 7.0.30 to 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to nail
 it
 down to the point that the problem manifests 7.0.50 onwards. Here is
 the
 exception that I see inside logs:

  Caused by: java.net.SocketException: Connection reset by peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but
 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could
 explain
 this
 behavior. Can someone please provide some pointer as what could be
 causing
 this?


  https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in the
 changelog


  Could it be If a request that includes an Expect: 100-continue header
 receives anything other than a 2xx response, close the connection This
 protects against misbehaving clients that may not sent the request body
 in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the first
 version with this change.


 Regards,

 Rainer


 I did see this in changelog but in the captured traffic don't see any
 expect 100 header request. Any other way I can confirm this on the tomcat
 side?

 Thanks,
 Umesh


 Can you please point me to SVN change for :
 If a request that includes an Expect: 100-continue header receives
 anything other than a 2xx response, close the connection This protects
 against misbehaving clients that may not sent the request body in that
 case
 and send the next request instead. (markt) ?


 http://svn.apache.org/r1540689


 Hi Rainer,

Thanks, I think the problem is indeed caused by this change. I downloaded
the tomcat source, removed the above change from AbstractHttp11Processor
and delpoyed the updated jar. The file upload didn't work right away but at
least now maxSwallowSize is honored and I can upload the files per the size
specified.

I did the above work to confirm, but of course I don't want to ship it
carrying modified code. Can you please suggest as what could be done in
this case?

Thanks again.


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-03 Thread Rainer Jung


Am 03.03.2015 um 13:45 schrieb Umesh Sehgal:

On Mon, Mar 2, 2015 at 6:20 PM, Rainer Jung rainer.j...@kippdata.de wrote:


Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

  On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com

wrote:



  On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de

wrote:

  Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:


  Thanks for the quick reply, I tried using the maxSwallowSize with

increased
value but to no effect. The max size that I have been able to upload is
~16
KB. I also see that the maxSwallowSize got introduced with update 55
but
the behavior I'm seeing is 50 update onwards, is there any other param
too?
is there any logging that can be turned on tomcat to help debug?



Please do not top post. For the rest see below.

   On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de


wrote:

   Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:



Hi,



We recently upgraded our application's tomcat from 7.0.30 to 7.0.59.
After
upgrade the file upload feature has broken. I have been able to nail
it
down to the point that the problem manifests 7.0.50 onwards. Here is
the
exception that I see inside logs:

  Caused by: java.net.SocketException: Connection reset by peer:
socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
at sun.security.ssl.OutputRecord.write(Unknown Source)

Also, I notice that the problem doesn't happen with a 2KB file but
2MB. I
don't see anything obvious in the 7.0.50 changelog which could
explain
this
behavior. Can someone please provide some pointer as what could be
causing
this?


  https://bz.apache.org/bugzilla/show_bug.cgi?id=57617


Fixed for next 7.0.60 in

http://svn.apache.org/r1659295

The original change can be found looking for maxSwallowSize in the
changelog



  Could it be If a request that includes an Expect: 100-continue header

receives anything other than a 2xx response, close the connection This
protects against misbehaving clients that may not sent the request body
in
that case and send the next request instead. (markt) ?

It was changed in 7.0.49, but 49 was not released, so 50 was the first
version with this change.


Regards,

Rainer



I did see this in changelog but in the captured traffic don't see any
expect 100 header request. Any other way I can confirm this on the tomcat
side?

Thanks,
Umesh



Can you please point me to SVN change for :
If a request that includes an Expect: 100-continue header receives
anything other than a 2xx response, close the connection This protects
against misbehaving clients that may not sent the request body in that
case
and send the next request instead. (markt) ?



http://svn.apache.org/r1540689


Hi Rainer,


Thanks, I think the problem is indeed caused by this change. I downloaded
the tomcat source, removed the above change from AbstractHttp11Processor
and delpoyed the updated jar. The file upload didn't work right away but at
least now maxSwallowSize is honored and I can upload the files per the size
specified.

I did the above work to confirm, but of course I don't want to ship it
carrying modified code. Can you please suggest as what could be done in
this case?


OK, good to know. I'd say now it would be good to find out why your 
webapp sends a non-2xx response code and hwich it is.


Since you already suceeded to build tomcat, simply add a custom log or 
System.out.println() statement printing response.getStatus() where the 
change in r1540689 was added and tell us what it is for the failing uploads.


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-02 Thread Rainer Jung


Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:

Hi,

We recently upgraded our application's tomcat from 7.0.30 to 7.0.59. After
upgrade the file upload feature has broken. I have been able to nail it
down to the point that the problem manifests 7.0.50 onwards. Here is the
exception that I see inside logs:

   Caused by: java.net.SocketException: Connection reset by peer: socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
at sun.security.ssl.OutputRecord.write(Unknown Source)

Also, I notice that the problem doesn't happen with a 2KB file but 2MB. I
don't see anything obvious in the 7.0.50 changelog which could explain this
behavior. Can someone please provide some pointer as what could be causing
this?


https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

Fixed for next 7.0.60 in

http://svn.apache.org/r1659295

The original change can be found looking for maxSwallowSize in the 
changelog


Regards,

Rainer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-02 Thread Rainer Jung


Am 02.03.2015 um 13:34 schrieb Umesh Sehgal:

On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com wrote:



On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de
wrote:


Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:


Thanks for the quick reply, I tried using the maxSwallowSize with
increased
value but to no effect. The max size that I have been able to upload is
~16
KB. I also see that the maxSwallowSize got introduced with update 55 but
the behavior I'm seeing is 50 update onwards, is there any other param
too?
is there any logging that can be turned on tomcat to help debug?



Please do not top post. For the rest see below.

  On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de

wrote:

  Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:


   Hi,



We recently upgraded our application's tomcat from 7.0.30 to 7.0.59.
After
upgrade the file upload feature has broken. I have been able to nail it
down to the point that the problem manifests 7.0.50 onwards. Here is
the
exception that I see inside logs:

 Caused by: java.net.SocketException: Connection reset by peer:
socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
at sun.security.ssl.OutputRecord.write(Unknown Source)

Also, I notice that the problem doesn't happen with a 2KB file but
2MB. I
don't see anything obvious in the 7.0.50 changelog which could explain
this
behavior. Can someone please provide some pointer as what could be
causing
this?



https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

Fixed for next 7.0.60 in

http://svn.apache.org/r1659295

The original change can be found looking for maxSwallowSize in the
changelog




Could it be If a request that includes an Expect: 100-continue header
receives anything other than a 2xx response, close the connection This
protects against misbehaving clients that may not sent the request body in
that case and send the next request instead. (markt) ?

It was changed in 7.0.49, but 49 was not released, so 50 was the first
version with this change.


Regards,

Rainer



I did see this in changelog but in the captured traffic don't see any
expect 100 header request. Any other way I can confirm this on the tomcat
side?

Thanks,
Umesh



Can you please point me to SVN change for :
If a request that includes an Expect: 100-continue header receives
anything other than a 2xx response, close the connection This protects
against misbehaving clients that may not sent the request body in that case
and send the next request instead. (markt) ?


http://svn.apache.org/r1540689


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de wrote:

 Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:

 Thanks for the quick reply, I tried using the maxSwallowSize with
 increased
 value but to no effect. The max size that I have been able to upload is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update 55 but
 the behavior I'm seeing is 50 update onwards, is there any other param
 too?
 is there any logging that can be turned on tomcat to help debug?


 Please do not top post. For the rest see below.

  On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:

   Hi,


 We recently upgraded our application's tomcat from 7.0.30 to 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to nail it
 down to the point that the problem manifests 7.0.50 onwards. Here is the
 exception that I see inside logs:

 Caused by: java.net.SocketException: Connection reset by peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but 2MB.
 I
 don't see anything obvious in the 7.0.50 changelog which could explain
 this
 behavior. Can someone please provide some pointer as what could be
 causing
 this?


 https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in the
 changelog


 Could it be If a request that includes an Expect: 100-continue header
 receives anything other than a 2xx response, close the connection This
 protects against misbehaving clients that may not sent the request body in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the first
 version with this change.


 Regards,

 Rainer


I did see this in changelog but in the captured traffic don't see any
expect 100 header request. Any other way I can confirm this on the tomcat
side?

Thanks,
Umesh

Re: Fwd: File upload fails after upgrade to 7.0.59

On Mon, Mar 2, 2015 at 5:25 PM, Umesh Sehgal umesh.seh...@gmail.com wrote:




 On Mon, Mar 2, 2015 at 3:52 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

 Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:

 Thanks for the quick reply, I tried using the maxSwallowSize with
 increased
 value but to no effect. The max size that I have been able to upload is
 ~16
 KB. I also see that the maxSwallowSize got introduced with update 55 but
 the behavior I'm seeing is 50 update onwards, is there any other param
 too?
 is there any logging that can be turned on tomcat to help debug?


 Please do not top post. For the rest see below.

  On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de
 wrote:

  Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:

   Hi,


 We recently upgraded our application's tomcat from 7.0.30 to 7.0.59.
 After
 upgrade the file upload feature has broken. I have been able to nail it
 down to the point that the problem manifests 7.0.50 onwards. Here is
 the
 exception that I see inside logs:

 Caused by: java.net.SocketException: Connection reset by peer:
 socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but
 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could explain
 this
 behavior. Can someone please provide some pointer as what could be
 causing
 this?


 https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in the
 changelog


 Could it be If a request that includes an Expect: 100-continue header
 receives anything other than a 2xx response, close the connection This
 protects against misbehaving clients that may not sent the request body in
 that case and send the next request instead. (markt) ?

 It was changed in 7.0.49, but 49 was not released, so 50 was the first
 version with this change.


 Regards,

 Rainer


 I did see this in changelog but in the captured traffic don't see any
 expect 100 header request. Any other way I can confirm this on the tomcat
 side?

 Thanks,
 Umesh


Can you please point me to SVN change for :
If a request that includes an Expect: 100-continue header receives
anything other than a 2xx response, close the connection This protects
against misbehaving clients that may not sent the request body in that case
and send the next request instead. (markt) ?

Thanks,
Umesh

Fwd: File upload fails after upgrade to 7.0.59

Hi,

We recently upgraded our application's tomcat from 7.0.30 to 7.0.59. After
upgrade the file upload feature has broken. I have been able to nail it
down to the point that the problem manifests 7.0.50 onwards. Here is the
exception that I see inside logs:

  Caused by: java.net.SocketException: Connection reset by peer: socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
at sun.security.ssl.OutputRecord.write(Unknown Source)

Also, I notice that the problem doesn't happen with a 2KB file but 2MB. I
don't see anything obvious in the 7.0.50 changelog which could explain this
behavior. Can someone please provide some pointer as what could be causing
this?


Thanks,
Umesh

Re: Fwd: File upload fails after upgrade to 7.0.59

Thanks for the quick reply, I tried using the maxSwallowSize with increased
value but to no effect. The max size that I have been able to upload is ~16
KB. I also see that the maxSwallowSize got introduced with update 55 but
the behavior I'm seeing is 50 update onwards, is there any other param too?
is there any logging that can be turned on tomcat to help debug?

Thanks,
Umesh

Thanks,
Umesh Sehgal

On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de wrote:

 Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:

  Hi,

 We recently upgraded our application's tomcat from 7.0.30 to 7.0.59. After
 upgrade the file upload feature has broken. I have been able to nail it
 down to the point that the problem manifests 7.0.50 onwards. Here is the
 exception that I see inside logs:

Caused by: java.net.SocketException: Connection reset by peer: socket
 write error
 at java.net.SocketOutputStream.socketWrite0(Native Method)
 at java.net.SocketOutputStream.socketWrite(Unknown Source)
 at java.net.SocketOutputStream.write(Unknown Source)
 at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
 at sun.security.ssl.OutputRecord.write(Unknown Source)

 Also, I notice that the problem doesn't happen with a 2KB file but 2MB. I
 don't see anything obvious in the 7.0.50 changelog which could explain
 this
 behavior. Can someone please provide some pointer as what could be causing
 this?


 https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

 Fixed for next 7.0.60 in

 http://svn.apache.org/r1659295

 The original change can be found looking for maxSwallowSize in the
 changelog

 Regards,

 Rainer


 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Fwd: File upload fails after upgrade to 7.0.59

2015-03-02 Thread Rainer Jung


Am 02.03.2015 um 11:02 schrieb Umesh Sehgal:

Thanks for the quick reply, I tried using the maxSwallowSize with increased
value but to no effect. The max size that I have been able to upload is ~16
KB. I also see that the maxSwallowSize got introduced with update 55 but
the behavior I'm seeing is 50 update onwards, is there any other param too?
is there any logging that can be turned on tomcat to help debug?


Please do not top post. For the rest see below.


On Mon, Mar 2, 2015 at 2:32 PM, Rainer Jung rainer.j...@kippdata.de wrote:


Am 02.03.2015 um 09:34 schrieb Umesh Sehgal:

  Hi,


We recently upgraded our application's tomcat from 7.0.30 to 7.0.59. After
upgrade the file upload feature has broken. I have been able to nail it
down to the point that the problem manifests 7.0.50 onwards. Here is the
exception that I see inside logs:

Caused by: java.net.SocketException: Connection reset by peer: socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.net.SocketOutputStream.write(Unknown Source)
at sun.security.ssl.OutputRecord.writeBuffer(Unknown Source)
at sun.security.ssl.OutputRecord.write(Unknown Source)

Also, I notice that the problem doesn't happen with a 2KB file but 2MB. I
don't see anything obvious in the 7.0.50 changelog which could explain
this
behavior. Can someone please provide some pointer as what could be causing
this?



https://bz.apache.org/bugzilla/show_bug.cgi?id=57617

Fixed for next 7.0.60 in

http://svn.apache.org/r1659295

The original change can be found looking for maxSwallowSize in the
changelog


Could it be If a request that includes an Expect: 100-continue header 
receives anything other than a 2xx response, close the connection This 
protects against misbehaving clients that may not sent the request body 
in that case and send the next request instead. (markt) ?


It was changed in 7.0.49, but 49 was not released, so 50 was the first 
version with this change.


Regards,

Rainer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-18 Thread Mark Thomas

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 17/12/2012 19:11, Christopher Schultz wrote:
 Nick,
 
 On 12/14/12 4:28 PM, Williams, Nick wrote:
 If it was using the global Content-length header, it would
 count not only the encoded data bytes, but also the parts
 separators, headers etc..
 
 So that's nice. It counts only the net data bytes, which is 
 easier to compare to the size on disk of a file that you would 
 upload.
 
 Indeed. A great explanation for why this would be done, and a
 much more logical way to do it than just using Content-Length
 (the easy way out).
 
 Also, Tomcat has to fully drain the input stream, anyway, so
 bombing earlier by checking the Content-Length doesn't actually
 improve anything.

Not quite. There is a little more to it than that and the exact
behaviour varies depending on the circumstances. See [1] and look for
swallowAbortedUploads. It explains what happens quite nicely.

Mark


[1] http://tomcat.apache.org/tomcat-7.0-doc/config/context.html
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (Darwin)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iQIcBAEBAgAGBQJQ0EboAAoJEBDAHFovYFnnQ7QQANyzCqL1zpv5r8L1HVT4abaR
0kTy4D5Er4TDsmhA+TvWonGLrNj9nK58OOb5hxx2xz63xLBg/MxjGmo6txxaoxe7
ZYL6wmL2iCp2pthJyhpXA9xFfZvyGfhbmNpxKQ8RHbL3vru5FTPwt69TtpFjtjrL
UJmyZ3LYGjVaDlad6fyR/ZuxmlwxApd1vYDjmNbRUTTL4b3pcSyKYkH9UBs3ukih
toQGU9cCnSwwGRnQaZngvBB8uvedP35j4H2zTkRAhoZ3Il/rVrtDexh+ibgp+tBq
TAat4YSmIgBob7BQMMMFjdWezLn4Zuh99s+oSS1GAO8FkcCw+DfrlodGVFSDjhe4
HwomlIw+1ehLdmAd75D7kkL1arvv7l4Csk+HrC/TFSwyrsfHlOMn7rsam0sXicJ5
jyN17JJK0iY+sddU6E6WVafAKqET5g6bO3s0FtPmREuvQveSwDlz3d4JDK7S/4QN
O9B9cEE4WlroGboNM9CZgdi35JYkM1BfVjm6MIph2ILvnDj+h+pBbmg9lgLO4iH2
uBj3FEJ5Y+7WtLKwNPcJUMD/ErcUoWKQM6yf8fscrARNvu0wBnF8qamvqXGKFlnt
EvetAbSTe31cRbUrDDk8eSmuzpQAvcsuMaUA0/LhvlYkkPSquA9iiKiURk2ZIZo3
lUH9LwbGwVvns6T+bIjF
=ZQTL
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-17 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Nick,

On 12/14/12 4:28 PM, Williams, Nick wrote:
 If it was using the global Content-length header, it would count 
 not only the encoded data bytes, but also the parts separators, 
 headers etc..
 
 So that's nice. It counts only the net data bytes, which is
 easier to compare to the size on disk of a file that you would
 upload.
 
 Indeed. A great explanation for why this would be done, and a much 
 more logical way to do it than just using Content-Length (the
 easy way out).

Also, Tomcat has to fully drain the input stream, anyway, so bombing
earlier by checking the Content-Length doesn't actually improve
anything.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlDPbmsACgkQ9CaO5/Lv0PDEQgCfYIxtmmsCqWthDN5fciu4CYup
l80An3Bt3nlSB0zfe3YN6ax1VHnnx7o/
=Eny7
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [OT] Does maxPostSize has an effect on file upload?

2012-12-17 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Nick,

On 12/14/12 5:36 PM, Williams, Nick wrote:
 The way Tomcat is apparently doing it now is much more sensible,
 in my humble opinion, because it does allow a direct and easy 
 comparison with the files being uploaded. And since as per above
 it needs to be kept in some cases anyway, my vote - if I had one
 - would be to not change it.
 
 I must agree with André. The process of base64 encoding a file 
 increases the number of bytes it takes to transmit it. But since
 that is not the actual size of the file, the extra length should
 not be counted towards the post size. The process by which the part
 lengths are added up DECODED is a much more accurate way to do it,
 in my opinion.

Right.

It also protects against uploading a file using gzip encoding where
the actual file is larger that the upload limit.

It also means that uploading a zip bomb[1] can be detected and prevented.

Reasonable people can argue about the appropriateness of the first
point above (is the admin trying to cap the number of bytes uploaded
or the number of bytes effectively placed on the filesystem?) but the
second one is very important to the stability of a server.

 How confusing would it be to a user who uploads a file that is
 1,989,956 bytes to get notified that the file exceeded the 2 MB 
 limit? The user certainly wouldn't understand that his file base64 
 encoded was larger than 2 MB. He would think the site was broken.

Files are rarely uploaded using base64 over HTTP -- that kind of thing
is much more common when using binary-intolerant systems like SMTP.
I'm not sure any normal user-agents can upload using base64, anyway. I
think you basically have identity, compress, gzip, and deflate
content-encodings. Anything done with base64 would have to have a
custom client and the webapp would have to handle the decoding.

- -chris

[1] http://en.wikipedia.org/wiki/Zip_bomb

-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlDPcSUACgkQ9CaO5/Lv0PDmOgCePVBEtJBvjvZKA8Y6q0HJKxx6
v/gAoJwZePfcY+yiCO6A1YtciYH9cJZZ
=sG2l
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Kai Weber

* Mark Thomas ma...@apache.org:

 Does a file upload as multipart/form-data not count to the size of the
 POST?
 
 No, as the doc make clear.

I asked because I could not find a hint in the docs or the INTERNET. What doc
do you mean? I looked into 
http://tomcat.apache.org/tomcat-6.0-doc/config/http.html
for maxPostSize.

Thanks, Kai

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Mark,

On 12/13/12 10:27 AM, Mark Thomas wrote:
 
 
 Kai Weber kai.we...@glorybox.de wrote:
 
 I see the following behaviour on Tomcat 6.0.24:
 
 The maxPostSize is not set, so uses the default of 2MB. I can
 upload files bigger than 2MB (5MB for example). I use
 commons-fileupload to read the file.
 
 As expected.
 
 Does a file upload as multipart/form-data not count to the size
 of the POST?
 
 No, as the doc make clear.

I'm not so sure the docs make it clear. Here's what that attribute
currently says (in its entirety):


The maximum size in bytes of the POST which will be handled by the
container FORM URL parameter parsing. The limit can be disabled by
setting this attribute to a value less than or equal to 0. If not
specified, this attribute is set to 2097152 (2 megabytes).


It says FORM URL parameter parsing. That may be some kind of code
that means only when the type is application/x-www-form-urlencoded.
We could probably explicitly state that multipart requests are
excepted from this limit.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlDLW7UACgkQ9CaO5/Lv0PANRgCfUA69SHOZk5O87VlcoFz4LELt
lYQAoMH4B2tFFDnARGkRoB5BDR2GxfV1
=itOY
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Caldarale, Charles R

 From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
 Subject: Re: Does maxPostSize has an effect on file upload?

   Does a file upload as multipart/form-data not count to the size
   of the POST?

  No, as the doc make clear.

 I'm not so sure the docs make it clear.

I think you have to read the relevant RFC:
http://www.ietf.org/rfc/rfc1867.txt
which is implicitly part of the doc.

 - Chuck

THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Chuck,

On 12/14/12 12:38 PM, Caldarale, Charles R wrote:
 From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
 Subject: Re: Does maxPostSize has an effect on file upload?

 Does a file upload as multipart/form-data not count to the
 size of the POST?

 No, as the doc make clear.

 I'm not so sure the docs make it clear.

 I think you have to read the relevant RFC: 
 http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the
 doc.

I'm not sure what part of that spec would apply.

Given the (Tomcat) documentation, one might reasonably believe that
the maxPostSize would refer to either the total Content-Length for an
application/x-www-form-urlencoded POST message or an individual
Content-Length of a single part of a multipart/form-data POST message.

The latter is not the case, and I think it's worth pointing that out.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEAREIAAYFAlDLdMcACgkQ9CaO5/Lv0PCkPwCghEQ2KYNPSCV+buGj/VtDvmme
e/cAoJ3ci461Q4xD992azyLQi8FxRAym
=+GMt
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread André Warnier


Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Chuck,

On 12/14/12 12:38 PM, Caldarale, Charles R wrote:
From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
Subject: Re: Does maxPostSize has an effect on file upload?

Does a file upload as multipart/form-data not count to the
size of the POST?

No, as the doc make clear.

I'm not so sure the docs make it clear.
I think you have to read the relevant RFC: 
http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the

doc.


I'm not sure what part of that spec would apply.

Given the (Tomcat) documentation, one might reasonably believe that
the maxPostSize would refer to either the total Content-Length for an
application/x-www-form-urlencoded POST message or an individual
Content-Length of a single part of a multipart/form-data POST message.

The latter is not the case, and I think it's worth pointing that out.



Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain to what 
the maxPostSize attribute of the Connector really does relate ?

What /is/ being measured there ?
Is it, like, just the request's Content-length header, or is there a counter which 
really counts the bytes being read e.g.
(and yes, obviously, if this is limited to an application/x-www-form-urlencoded POST, 
then it wouldn't be just the Content-length header)
(and, if it covers a multipart/form-data POST, then is it the total body size, before of 
after decoding the Base-64 encoded bits ?)
After all, this is a Tomcat users list, not a Tomcat dev list; so we can't really 
expect the public here to go delve into the Java code to find out.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Does maxPostSize has an effect on file upload?

So I read some code in org.apache.catalina.connector.Request.java. When 
parseParameters() is called, it checks whether the Content-Type is 
multipart/form-data or application/x-www-form-urlencoded. If it's 
application/x-www-form-urlencoded, it checks to make sure that the 
Content-Length is not greater than maxPostSize. If it's multipart/form-data, 
it delegates to another method, parseParts().

In parseParts(), the code begins looping over the parts and placing them into 
the list of parts for the request. As it processes each part, it adds the size 
(in bytes) of that part (including the part contents or file contents if this 
is a file upload) to an aggregate post size variable. If at any point the 
aggregate post size exceeds the maxPostSize variable, it fails. Why the 
Content-Length is not checked, I am unsure. It seems it would be less expensive 
to throw the exception before ever trying to parse the parts. However, this 
APPEARS to be the functional equivalent of checking the Content-Length.

So, it would seem that maxPostSize does, indeed, affect file uploads, and that 
if the total size of your request, including file parts, exceeds maxPostSize, 
that the request will fail. Whether this behavior is consistent with that 
specified in the spec, I am not an expert on that.

Not that it matters, but this behavior is consistent with the land of PHP, 
where a maxUploadFileSize value of 50MB does you no good if you forget to 
increas your maxPostSize from the default of 5MB.

(Note: It's entirely possible that I'm reading the code wrong. I would suggest 
that an experiment might solve this conclusively, but perhaps that has been 
tried and I missed that somewhere in the chain.)

Nick

-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Friday, December 14, 2012 1:15 PM
To: Tomcat Users List
Subject: Re: Does maxPostSize has an effect on file upload?

Christopher Schultz wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA256

 Chuck,

 On 12/14/12 12:38 PM, Caldarale, Charles R wrote:
 From: Christopher Schultz [mailto:ch...@christopherschultz.net]
 Subject: Re: Does maxPostSize has an effect on file upload?
 Does a file upload as multipart/form-data not count to the size of
 the POST?
 No, as the doc make clear.
 I'm not so sure the docs make it clear.
 I think you have to read the relevant RFC:
 http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the
 doc.

 I'm not sure what part of that spec would apply.

 Given the (Tomcat) documentation, one might reasonably believe that
 the maxPostSize would refer to either the total Content-Length for an
 application/x-www-form-urlencoded POST message or an individual
 Content-Length of a single part of a multipart/form-data POST message.

 The latter is not the case, and I think it's worth pointing that out.


Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain 
to what the maxPostSize attribute of the Connector really does relate ?
What /is/ being measured there ?
Is it, like, just the request's Content-length header, or is there a counter 
which really counts the bytes being read e.g.
(and yes, obviously, if this is limited to an 
application/x-www-form-urlencoded POST, then it wouldn't be just the 
Content-length header) (and, if it covers a multipart/form-data POST, then is 
it the total body size, before of after decoding the Base-64 encoded bits ?) 
After all, this is a Tomcat users list, not a Tomcat dev list; so we can't 
really expect the public here to go delve into the Java code to find out.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



This e-mail may contain privileged or confidential information. If you are not 
the intended recipient: (1) you may not disclose, use, distribute, copy or rely 
upon this message or attachment(s); and (2) please notify the sender by reply 
e-mail, and then delete this message and its attachment(s). Underwriters 
Laboratories Inc. and its affiliates disclaim all liability for any errors, 
omissions, corruption or virus in this message or any attachments.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread André Warnier


Williams, Nick wrote:

So I read some code in org.apache.catalina.connector.Request.java. When parseParameters() is called, it 
checks whether the Content-Type is multipart/form-data or 
application/x-www-form-urlencoded. If it's application/x-www-form-urlencoded, it 
checks to make sure that the Content-Length is not greater than maxPostSize.


That is probably because in that case, the body is not encoded, and the Content-length 
matches the length of the posted data.


 If it's multipart/form-data, it delegates to another method, parseParts().


In parseParts(), the code begins looping over the parts and placing them into 
the list of parts for the request. As it processes each part, it adds the size 
(in bytes) of that part (including the part contents or file contents if this 
is a file upload) to an aggregate post size variable. If at any point the 
aggregate post size exceeds the maxPostSize variable, it fails.


This thus looks like it applies to the total size of the parts, after decoding the ones 
that are Base-64 encoded (such as uploaded files contents).


 Why the Content-Length is not checked, I am unsure. It seems it would be less expensive 
to throw the exception before ever trying to parse the parts. However, this APPEARS to be 
the functional equivalent of checking the Content-Length.


If it was using the global Content-length header, it would count not only the encoded data 
bytes, but also the parts separators, headers etc..


So that's nice. It counts only the net data bytes, which is easier to compare to the size 
on disk of a file that you would upload.




So, it would seem that maxPostSize does, indeed, affect file uploads, and that 
if the total size of your request, including file parts, exceeds maxPostSize, 
that the request will fail. Whether this behavior is consistent with that 
specified in the spec, I am not an expert on that.



You've done a pretty good job so far, I believe.

What maybe is the difference here, is that you probably looked at the Tomcat 7 code, in 
which I believe the FileUpload code has been integrated/merged.


Maybe in Tomcat 6 it is different, and when FileUpload is used, it doesn't count the 
bytes, or doesn't pass the result back to Tomcat to match the MaxPostSize ?




Not that it matters, but this behavior is consistent with the land of PHP, 
where a maxUploadFileSize value of 50MB does you no good if you forget to 
increas your maxPostSize from the default of 5MB.

(Note: It's entirely possible that I'm reading the code wrong. I would suggest 
that an experiment might solve this conclusively, but perhaps that has been 
tried and I missed that somewhere in the chain.)

Nick

-Original Message-
From: André Warnier [mailto:a...@ice-sa.com]
Sent: Friday, December 14, 2012 1:15 PM
To: Tomcat Users List
Subject: Re: Does maxPostSize has an effect on file upload?

Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

Chuck,

On 12/14/12 12:38 PM, Caldarale, Charles R wrote:

From: Christopher Schultz [mailto:ch...@christopherschultz.net]
Subject: Re: Does maxPostSize has an effect on file upload?

Does a file upload as multipart/form-data not count to the size of
the POST?

No, as the doc make clear.

I'm not so sure the docs make it clear.

I think you have to read the relevant RFC:
http://www.ietf.org/rfc/rfc1867.txt which is implicitly part of the
doc.

I'm not sure what part of that spec would apply.

Given the (Tomcat) documentation, one might reasonably believe that
the maxPostSize would refer to either the total Content-Length for an
application/x-www-form-urlencoded POST message or an individual
Content-Length of a single part of a multipart/form-data POST message.

The latter is not the case, and I think it's worth pointing that out.



Just a suggestion : maybe one of the Tomcat/Java gurus here could first explain 
to what the maxPostSize attribute of the Connector really does relate ?
What /is/ being measured there ?
Is it, like, just the request's Content-length header, or is there a counter 
which really counts the bytes being read e.g.
(and yes, obviously, if this is limited to an application/x-www-form-urlencoded POST, then it wouldn't be 
just the Content-length header) (and, if it covers a multipart/form-data POST, then is it the total body 
size, before of after decoding the Base-64 encoded bits ?) After all, this is a Tomcat users list, not a 
Tomcat dev list; so we can't really expect the public here to go delve into the Java code to find out.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



This e-mail may contain privileged or confidential information. If you are not 
the intended recipient: (1) you may not disclose, use, distribute, copy or rely 
upon this message or attachment(s); and (2) please notify the sender by reply 
e-mail, and then delete

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Mark Thomas

On 14/12/2012 19:58, Williams, Nick wrote:
 (Note: It's entirely possible that I'm reading the code wrong.

Yes you are. Not completely wrongly but there are errors.

The short version is as follows:

If Tomcat is responsible for reading the request body such as via a call
to a method like getParameters(), getParts() etc. then Tomcat will apply
the maxPostSize limit.

If the application is responsible for reading the request body then the
limit does not apply. Hence if an application uses a third-party file
upload library or just calls getInputStream() or getReader() the limit
will not apply.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Does maxPostSize has an effect on file upload?

A valid point that I have not considered. Since parseParameters() is not called 
until you get the request parameters or parts, a developer could 
getInputStream() and parse all the parts/parameters themselves, thereby 
completely skipping the maxPostSize checks. Thanks for correcting me here, Mark.

Nick

-Original Message-
From: Mark Thomas [mailto:ma...@apache.org]
Sent: Friday, December 14, 2012 3:17 PM
To: Tomcat Users List
Subject: Re: Does maxPostSize has an effect on file upload?

On 14/12/2012 19:58, Williams, Nick wrote:
 (Note: It's entirely possible that I'm reading the code wrong.

Yes you are. Not completely wrongly but there are errors.

The short version is as follows:

If Tomcat is responsible for reading the request body such as via a call to a 
method like getParameters(), getParts() etc. then Tomcat will apply the 
maxPostSize limit.

If the application is responsible for reading the request body then the limit 
does not apply. Hence if an application uses a third-party file upload library 
or just calls getInputStream() or getReader() the limit will not apply.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



This e-mail may contain privileged or confidential information. If you are not 
the intended recipient: (1) you may not disclose, use, distribute, copy or rely 
upon this message or attachment(s); and (2) please notify the sender by reply 
e-mail, and then delete this message and its attachment(s). Underwriters 
Laboratories Inc. and its affiliates disclaim all liability for any errors, 
omissions, corruption or virus in this message or any attachments.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Does maxPostSize has an effect on file upload?

 If it was using the global Content-length header, it would count not only the 
 encoded data bytes, but also the parts separators, headers etc..

 So that's nice. It counts only the net data bytes, which is easier to compare 
 to the size on disk of a file that you would upload.

Indeed. A great explanation for why this would be done, and a much more logical 
way to do it than just using Content-Length (the easy way out).

Thanks!


This e-mail may contain privileged or confidential information. If you are not 
the intended recipient: (1) you may not disclose, use, distribute, copy or rely 
upon this message or attachment(s); and (2) please notify the sender by reply 
e-mail, and then delete this message and its attachment(s). Underwriters 
Laboratories Inc. and its affiliates disclaim all liability for any errors, 
omissions, corruption or virus in this message or any attachments.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread Mark Thomas

On 14/12/2012 21:13, André Warnier wrote:

snip/

  If it's multipart/form-data, it delegates to another method,
 parseParts().

snip/

  Why the Content-Length is not checked, I am unsure. It seems it would
 be less expensive to throw the exception before ever trying to parse the
 parts. However, this APPEARS to be the functional equivalent of checking
 the Content-Length.

I think that is simply an oversight in the commit [1] that added the
checking. If a content-length is present then checking it would be
simpler. The existing checks need to be kept for the chunked request
body case.

As always, patches welcome.

Mark

[1] http://svn.apache.org/viewvc?view=revisionrevision=1195968


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-14 Thread André Warnier


Mark Thomas wrote:

On 14/12/2012 21:13, André Warnier wrote:

snip/


 If it's multipart/form-data, it delegates to another method,
parseParts().


snip/


 Why the Content-Length is not checked, I am unsure. It seems it would
be less expensive to throw the exception before ever trying to parse the
parts. However, this APPEARS to be the functional equivalent of checking
the Content-Length.


I think that is simply an oversight in the commit [1] that added the
checking. If a content-length is present then checking it would be
simpler. The existing checks need to be kept for the chunked request
body case.

As always, patches welcome.



In the Apache httpd DAV implementation, the LimitXMLRequestBody is the only way to control 
the maximum size of an upload.  It is a p-i-t-a, because that size does not relate easily 
to the size of the file one is really uploading, making it difficult to set a sensible limit.


The way Tomcat is apparently doing it now is much more sensible, in my humble opinion, 
because it does allow a direct and easy comparison with the files being uploaded.
And since as per above it needs to be kept in some cases anyway, my vote - if I had one - 
would be to not change it.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Does maxPostSize has an effect on file upload?

 The way Tomcat is apparently doing it now is much more sensible, in my humble 
 opinion, because it does allow a direct and easy comparison with the files 
 being uploaded.
 And since as per above it needs to be kept in some cases anyway, my vote - if 
 I had one - would be to not change it.

I must agree with André. The process of base64 encoding a file increases the 
number of bytes it takes to transmit it. But since that is not the actual size 
of the file, the extra length should not be counted towards the post size. The 
process by which the part lengths are added up DECODED is a much more accurate 
way to do it, in my opinion. How confusing would it be to a user who uploads a 
file that is 1,989,956 bytes to get notified that the file exceeded the 2 MB 
limit? The user certainly wouldn't understand that his file base64 encoded was 
larger than 2 MB. He would think the site was broken.

N

This e-mail may contain privileged or confidential information. If you are not 
the intended recipient: (1) you may not disclose, use, distribute, copy or rely 
upon this message or attachment(s); and (2) please notify the sender by reply 
e-mail, and then delete this message and its attachment(s). Underwriters 
Laboratories Inc. and its affiliates disclaim all liability for any errors, 
omissions, corruption or virus in this message or any attachments.

Does maxPostSize has an effect on file upload?

2012-12-13 Thread Kai Weber

I see the following behaviour on Tomcat 6.0.24:

The maxPostSize is not set, so uses the default of 2MB. I can upload files
bigger than 2MB (5MB for example). I use commons-fileupload to read the file.

Does a file upload as multipart/form-data not count to the size of the POST?

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Does maxPostSize has an effect on file upload?

2012-12-13 Thread Mark Thomas



Kai Weber kai.we...@glorybox.de wrote:

I see the following behaviour on Tomcat 6.0.24:

The maxPostSize is not set, so uses the default of 2MB. I can upload
files
bigger than 2MB (5MB for example). I use commons-fileupload to read the
file.

As expected.

Does a file upload as multipart/form-data not count to the size of the
POST?

No, as the doc make clear.

Mark

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [Servlet 3.0] Monitoring File Upload Progress

2011-09-06 Thread Ole Ersoy


Hi Andre,

I'm looking for something like this:

pfu.setProgressListener(new FileUploadProgressListener());

Described in this article:
http://www.ibm.com/developerworks/web/library/wa-aj-dwr/?ca=dgr-lnxw06AjaxDWR

I could just go back to commons file upload, but thought I'd look around to see 
if anything something similar could be done with servlet 3.0.

Thanks,
- Ole


On 09/05/2011 03:12 PM, André Warnier wrote:

Ole Ersoy wrote:

Hi,

Anyone know whether it's possible to monitor progress of a file upload?


What do you mean by monitoring ?

Is it a question of providing the user with some feedback, like a
progress bar ?

If so, then one of the easier ways would be to write your own java
applet, downloaded and run by the browser in your upload form, to do the
upload and display some progress bar to the user.
You may want to search for something already available to do it though,
because writing it from scratch is not really trivial.
Personally, I would only do that if it was /really/ worth the effort.
Like if many users get impatient and break off the upload before it
finishes. Or of course if the marketing guys insist on it, for the look.
But then tell them the cost.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-06 Thread Ole Ersoy


Thanks guys!

Ole

On 09/03/2011 10:51 AM, Konstantin Preißer wrote:

Hi,


-Original Message-
From: Jonathan Soons [mailto:jso...@juilliard.edu]
Sent: Saturday, September 03, 2011 2:24 PM
To: Ole Ersoy; Tomcat Users List
Subject: RE: Servlet 3.0 File Upload

You need to add a line in in your form:
input type=text name=filename /

Then in your servlet GetPost() method you put this filename in a
variable:
String filename;
filename = req.getParameter(filename);

Then instead of part.write(samplefile);
do:
part.write(filename);



Doesn't that mean that the user has to enter the filename by himself?

What I usually do to get the filename is:

Part uploadPart = request.getPart(uploadfield); // get the Part
String contDispoHeader = uploadPart.getHeader(Content-Disposition); // get 
Content-Disposition header
String uploadFilename = null;
if (contDispoHeader != null) {
try {
uploadFilename = new 
ContentDisposition(contDispoHeader).getParameter(filename);
} catch (ParseException e) { }
}

Note that ContentDisposition class is from JavaMail package 
(javax.mail.internet.ContentDisposition). Browser usually send filenames in the 
filename parameter of a Content-Disposition header.


Regards,

Konstantin Preißer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [Servlet 3.0] Monitoring File Upload Progress

2011-09-06 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ole,

On 9/6/2011 11:03 AM, Ole Ersoy wrote:
 Described in this article: 
 http://www.ibm.com/developerworks/web/library/wa-aj-dwr/?ca=dgr-lnxw06AjaxDWR

  I could just go back to commons file upload, but thought I'd look
 around to see if anything something similar could be done with
 servlet 3.0.

The servlet API does not have any such provisions: your code executes
after the file upload has completed.

If you want to interact with the stream, you'll have to either handle
the stream yourself or use a library (like commons-fileupload) that
lets you watch the stream.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5mbkMACgkQ9CaO5/Lv0PC/JQCfRV9lZuCJ+Zvc5IL7ooctJYVh
TcEAnjkOqfAOgJPzhaXUfh2FOWlT/gZD
=MXx4
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-06 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Konstantin,

On 9/3/2011 11:51 AM, Konstantin Preißer wrote:
 What I usually do to get the filename is:
 
 Part uploadPart = request.getPart(uploadfield); // get the Part 
 String contDispoHeader =
 uploadPart.getHeader(Content-Disposition); // get
 Content-Disposition header String uploadFilename = null; if
 (contDispoHeader != null) { try { uploadFilename = new
 ContentDisposition(contDispoHeader).getParameter(filename); }
 catch (ParseException e) { } }

It seems dangerous to allow the client to specify the file name. All
kinds of bad things can happen such as specifying special file names
(does PRN still work in win32? through Java?) or overwriting files
from other clients.

I would highly recommend that some portion of the temporary file name
be completely random, as well as using something keyed on the request
to disambiguate the file as well.

I usually just use File.createTempFile, though performance of that
method can be less than ideal.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk5mb14ACgkQ9CaO5/Lv0PAYTACgi6ldsMdMYH4v3XLdfv5J6+U4
zh8An17xhq5gBZ1FJ5ElFLzXd1XVLX0q
=groU
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-06 Thread verlag.preis...@t-online.de


Hi Chris,
 
 It seems dangerous to allow the client to specify the file name. All
 kinds of bad things can happen such as specifying special file names
 (does PRN still work in win32? through Java?) or overwriting files
 from other clients.
 
 I would highly recommend that some portion of the temporary file name
 be completely random, as well as using something keyed on the request
 to disambiguate the file as well.

did you read my other reply to that thread? ;-)

Of course, I don't use that filename to save that file on the server (I assumed 
it is completely clear that one wouldn't do this). But I want to use the 
filename for displaying purposes.
E.g., I have a web application where the user can upload pictures, combined to 
a picture gallery (http://bildergalerie.pleier-it.de/ , it is a German site 
however, using TC 7). On uploading, the server reads the submitted filename and 
stores it in a field in the corresponding DB entry (without the file ending). 
Then it generates a filename based on the DB Row-ID (not on the filename) to 
actually store that image.
When the user visits the site, it displays miniature icons, using the filename 
field of the DB entry as description. Or, if the user choses to download the 
file, I can append a Content-Disposition header 
(javax.mail.internet.ContentDisposition) and set a filename parameter, so the 
user's browser download dialog can display the original filename (or a new 
name, if he edited the entry), without the actual URL having to contain that 
filename.  :)


Regards,

Konstantin Preißer



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-05 Thread André Warnier


This must be about the worst advice I have ever seen.
What about someone typing e.g. /etc/passwd in that text box?

If you allow people to upload files to your server, you should create your own location 
and naming scheme for the uploaded files.  You should not even use the original filename, 
unless you are dying to experience all the silly things that people can think of in terms 
of filenames (with spaces in them, or characters that are valid on one platform but not 
another, or characters in various character sets and so on.)



Jonathan Soons wrote:

You need to add a line in in your form:
input type=text name=filename /

Then in your servlet GetPost() method you put this filename in a variable:
String filename;
filename = req.getParameter(filename);

Then instead of part.write(samplefile);
do:
part.write(filename);

Jonathan Soons

From: Ole Ersoy [ole.er...@gmail.com]
Sent: Friday, September 02, 2011 6:50 PM
To: Tomcat Users List
Subject: Servlet 3.0 File Upload

Hi,

I have a working file upload servlet, with the exception that it calls the uploaded file 
samplefile instead of using the name of the file.  So if I upload different 
files, they all overwrite each other.  Any ideas on how to fix this?  I used this 
tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html

TIA,
- Ole


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-05 Thread Ole Ersoy


Thank you for the advice.  I'll stick to hard coded file locations and names :).

Thanks again,
- Ole

On 09/05/2011 03:22 AM, André Warnier wrote:

This must be about the worst advice I have ever seen.
What about someone typing e.g. /etc/passwd in that text box?

If you allow people to upload files to your server, you should create
your own location and naming scheme for the uploaded files. You should
not even use the original filename, unless you are dying to experience
all the silly things that people can think of in terms of filenames
(with spaces in them, or characters that are valid on one platform but
not another, or characters in various character sets and so on.)


Jonathan Soons wrote:

You need to add a line in in your form:
input type=text name=filename /

Then in your servlet GetPost() method you put this filename in a
variable:
String filename;
filename = req.getParameter(filename);

Then instead of part.write(samplefile);
do:
part.write(filename);

Jonathan Soons

From: Ole Ersoy [ole.er...@gmail.com]
Sent: Friday, September 02, 2011 6:50 PM
To: Tomcat Users List
Subject: Servlet 3.0 File Upload

Hi,

I have a working file upload servlet, with the exception that it calls
the uploaded file samplefile instead of using the name of the file.
So if I upload different files, they all overwrite each other. Any
ideas on how to fix this? I used this tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html


TIA,
- Ole


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-05 Thread Ole Ersoy


Thank you for the advice.  I'll stick to hard coded file locations and names :).

Thanks again,
- Ole

On 09/05/2011 03:22 AM, André Warnier wrote:

This must be about the worst advice I have ever seen.
What about someone typing e.g. /etc/passwd in that text box?

If you allow people to upload files to your server, you should create
your own location and naming scheme for the uploaded files. You should
not even use the original filename, unless you are dying to experience
all the silly things that people can think of in terms of filenames
(with spaces in them, or characters that are valid on one platform but
not another, or characters in various character sets and so on.)


Jonathan Soons wrote:

You need to add a line in in your form:
input type=text name=filename /

Then in your servlet GetPost() method you put this filename in a
variable:
String filename;
filename = req.getParameter(filename);

Then instead of part.write(samplefile);
do:
part.write(filename);

Jonathan Soons

From: Ole Ersoy [ole.er...@gmail.com]
Sent: Friday, September 02, 2011 6:50 PM
To: Tomcat Users List
Subject: Servlet 3.0 File Upload

Hi,

I have a working file upload servlet, with the exception that it calls
the uploaded file samplefile instead of using the name of the file.
So if I upload different files, they all overwrite each other. Any
ideas on how to fix this? I used this tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html


TIA,
- Ole


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: [Servlet 3.0] Monitoring File Upload Progress

2011-09-05 Thread André Warnier


Ole Ersoy wrote:

Hi,

Anyone know whether it's possible to monitor progress of a file upload?


What do you mean by monitoring ?

Is it a question of providing the user with some feedback, like a progress bar ?

If so, then one of the easier ways would be to write your own java applet, downloaded and 
run by the browser in your upload form, to do the upload and display some progress bar to 
the user.
You may want to search for something already available to do it though, because writing it 
from scratch is not really trivial.
Personally, I would only do that if it was /really/ worth the effort. Like if many users 
get impatient and break off the upload before it finishes.  Or of course if the marketing 
guys insist on it, for the look.  But then tell them the cost.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Servlet 3.0 File Upload

2011-09-03 Thread Jonathan Soons

You need to add a line in in your form:
input type=text name=filename /

Then in your servlet GetPost() method you put this filename in a variable:
String filename;
filename = req.getParameter(filename);

Then instead of part.write(samplefile);
do:
part.write(filename);

Jonathan Soons

From: Ole Ersoy [ole.er...@gmail.com]
Sent: Friday, September 02, 2011 6:50 PM
To: Tomcat Users List
Subject: Servlet 3.0 File Upload

Hi,

I have a working file upload servlet, with the exception that it calls the 
uploaded file samplefile instead of using the name of the file.  So if I 
upload different files, they all overwrite each other.  Any ideas on how to fix 
this?  I used this tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html

TIA,
- Ole


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Servlet 3.0 File Upload

2011-09-03 Thread Konstantin Preißer

Hi,

 -Original Message-
 From: Jonathan Soons [mailto:jso...@juilliard.edu]
 Sent: Saturday, September 03, 2011 2:24 PM
 To: Ole Ersoy; Tomcat Users List
 Subject: RE: Servlet 3.0 File Upload
 
 You need to add a line in in your form:
 input type=text name=filename /
 
 Then in your servlet GetPost() method you put this filename in a
 variable:
 String filename;
 filename = req.getParameter(filename);
 
 Then instead of part.write(samplefile);
 do:
 part.write(filename);
 

Doesn't that mean that the user has to enter the filename by himself?

What I usually do to get the filename is:

Part uploadPart = request.getPart(uploadfield); // get the Part
String contDispoHeader = uploadPart.getHeader(Content-Disposition); // get 
Content-Disposition header
String uploadFilename = null;
if (contDispoHeader != null) {
try {
uploadFilename = new 
ContentDisposition(contDispoHeader).getParameter(filename);
} catch (ParseException e) { }
}

Note that ContentDisposition class is from JavaMail package 
(javax.mail.internet.ContentDisposition). Browser usually send filenames in the 
filename parameter of a Content-Disposition header.


Regards,

Konstantin Preißer


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-03 Thread chris derham

  You need to add a line in in your form:
  input type=text name=filename /
 
  Then in your servlet GetPost() method you put this filename in a
  variable:
  String filename;
  filename = req.getParameter(filename);
 
  Then instead of part.write(samplefile);
  do:
  part.write(filename);


Letting the remote user control the name of the file that is written to disk
exposes a potential security risk. Due to bad configuration, the posted name
may allow the caller to save arbitrary files anywhere they wish. The server
should generate the name that is used to save the file to remove this risk

Chris

RE: Servlet 3.0 File Upload

2011-09-03 Thread Konstantin Preißer

Hi,

 -Original Message-
 From: cjder...@gmail.com [mailto:cjder...@gmail.com] On Behalf Of chris
 derham
 Sent: Saturday, September 03, 2011 6:51 PM
 To: Tomcat Users List
 Subject: Re: Servlet 3.0 File Upload

 Letting the remote user control the name of the file that is written to
 disk
 exposes a potential security risk. Due to bad configuration, the posted
 name
 may allow the caller to save arbitrary files anywhere they wish. The
 server
 should generate the name that is used to save the file to remove this
 risk

 Chris

Yes, the user could supply a name like ../../badfile.exe (even in the 
Content-Disposition header) to inject a file anywhere in the file system. Of 
course, one shouldn't use the supplied filename to save the file on the server. 
I usually make a DB entry with the supplied filename stored in a text field, 
and use the ID of the entry to generate a filename where the actual file 
contents are stored. The supplied filename is then only for displaying purposes.

Regards,

Konstantin Preißer

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

[Servlet 3.0] Monitoring File Upload Progress

2011-09-03 Thread Ole Ersoy


Hi,

Anyone know whether it's possible to monitor progress of a file upload?

TIA,
- Ole

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Servlet 3.0 File Upload

2011-09-02 Thread Ole Ersoy


Hi,

I have a working file upload servlet, with the exception that it calls the uploaded file 
samplefile instead of using the name of the file.  So if I upload different 
files, they all overwrite each other.  Any ideas on how to fix this?  I used this 
tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html

TIA,
- Ole   


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Servlet 3.0 File Upload

2011-09-02 Thread Ole Ersoy


Never mind...I see the example hard codes the name of the file.  Sorry for the 
noise.

On 09/02/2011 05:50 PM, Ole Ersoy wrote:

Hi,

I have a working file upload servlet, with the exception that it calls
the uploaded file samplefile instead of using the name of the file. So
if I upload different files, they all overwrite each other. Any ideas on
how to fix this? I used this tutorial to get it working:

http://www.servletworld.com/servlet-tutorials/servlet3/multipartconfig-file-upload-example.html


TIA,
- Ole


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Setting max file upload size

2010-12-29 Thread Anjib Mulepati

I am trying to upload the larger file through my app developed using 
Struts 1.3.8. I did change the config in struts to upload file upto 3GB 
but that doesn't work.
So now I am trying to find the configuration in Tomcat where I can set 
the max size for file upload.

Is there any?

Anjib



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Setting max file upload size

2010-12-29 Thread Ronald Klop


Set Connector maxPostSize=very large number / in server.xml.
3 GB might be very large. I don't know if the value is a long or an int in the 
internals of Tomcat.

This is from my live server.

   Connector port=8080 maxHttpHeaderSize=8192
  maxThreads=300 minSpareThreads=25 maxSpareThreads=75
  enableLookups=false redirectPort=8443 acceptCount=1024
  connectionTimeout=2 disableUploadTimeout=true
  compression=on
  maxPostSize=104857600 /

Ronald.

Op woensdag, 29 december 2010 17:03 schreef Anjib Mulepati 
anji...@hotmail.com:


 
I am trying to upload the larger file through my app developed using Struts 1.3.8. I did change the config in struts to upload file upto 3GB but that doesn't work.

So now I am trying to find the configuration in Tomcat where I can set the max 
size for file upload.
Is there any?

Anjib



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Setting max file upload size

2010-12-29 Thread André Warnier

And if you are uploading this through a browser form, don't forget that the browser is 
probably going to encode that file in Base64 encoding, which will roughly inflate the size 
by 1/3 of the original.

I don't know if Tomcat's maxPostSize counts the size prior, or after decoding.

Ronald Klop wrote:

Set Connector maxPostSize=very large number / in server.xml.
3 GB might be very large. I don't know if the value is a long or an int 
in the internals of Tomcat.


This is from my live server.

   Connector port=8080 maxHttpHeaderSize=8192
  maxThreads=300 minSpareThreads=25 maxSpareThreads=75
  enableLookups=false redirectPort=8443 acceptCount=1024
  connectionTimeout=2 disableUploadTimeout=true
  compression=on
  maxPostSize=104857600 /

Ronald.

Op woensdag, 29 december 2010 17:03 schreef Anjib Mulepati 
anji...@hotmail.com:


 
I am trying to upload the larger file through my app developed using 
Struts 1.3.8. I did change the config in struts to upload file upto 
3GB but that doesn't work.
So now I am trying to find the configuration in Tomcat where I can set 
the max size for file upload.

Is there any?

Anjib



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org











-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Problem moving from file upload servlet to JSP

Using Tomcat 6.0.18 (to be 6.0.26) and Google App Engine, for parallel  
development (different db tech)


Short form:  I need to accept a file upload in a servlet, do some  
computations on the upload, and then transition to a JSP with some  
data resulting from the computations. I'm having some trouble with the  
transition at the end.  I need either to make the current code work or  
find an

alternate way to achieve the goal.

Long form.  Here's the html for file upload (vanilla):

form name=csvUploadForm action=csvfileupload method=post  
enctype=multipart/form-data

File:input type=file name=csvfile2uploadbr/br
	input type=submit name=Submit value=Upload CSV File  
onclick=uploadCSVFile();return false;

/form

Here's the servlet mapping for the receiving/processing servlet:

servlet-mapping
servlet-nameCSVFileUpload/servlet-name
url-pattern/csvfileupload/*/url-pattern
/servlet-mapping

The CSVFileUpload servlet doPost method uses
org.apache.commons.fileupload.servlet.ServletFileUpload;
to accept and process the upload, and puts the resulting data into the  
db.  If I don't care about duplicates or partial matches,

this works fine; at the end of the servlet processing, I execute
response.sendRedirect(/myStartPage.jsp);
-- where response is the doPost's HttpServletResponse.

Now I need to deal with partial matches between incoming data from the  
upload, and information already in the db. Specifically,
I need to interact with the user to determine how to resolve the  
ambiguities.  I /thought/ I could create a bean with the necessary  
computed
info, use setAttribute to attach the bean to request (the doPost's  
HttpServletRequest), and then forward to a JSP with code like this:


String nextJSP = /nextPage.jsp;
	RequestDispatcher dispatcher =  
getServletContext().getRequestDispatcher(nextJSP);

dispatcher.forward(request, response);

Unfortunately, I'm running into the following errors showing in the log:

WARNING: Nested in javax.servlet.ServletException:  
org.apache.commons.fileupload.FileUploadBase 
$InvalidContentTypeException: the request doesn't contain a multipart/ 
form-data or multipart/mixed stream, content type header is null:
org.apache.commons.fileupload.FileUploadBase 
$InvalidContentTypeException: the request doesn't contain a multipart/ 
form-data or multipart/mixed stream, content type header is null
at org.apache.commons.fileupload.FileUploadBase 
$FileItemIteratorImpl.init(FileUploadBase.java:885)
at  
org 
.apache 
.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java: 
331)
at  
org 
.apache 
.commons 
.fileupload 
.servlet.ServletFileUpload.getItemIterator(ServletFileUpload.java:148)
at  
com 
.formrunner 
.servlets.CSVFileUploadServlet.doPost(CSVFileUploadServlet.java:53)
at  
com 
.formrunner 
.servlets.CSVFileUploadServlet.doGet(CSVFileUploadServlet.java:29)

.

The browser actually displays the nextPage.jsp page.  However, if one  
then clicks any navigation button, you get a version of the warning  
above showing in the browser.


I'm assuming that the CSVFileUpload servlet processing of the upload  
has stripped the request; hence the errors.


My goal is to get from the CSVFileUpload servlet to the nextPage.jsp  
with the partialMatch data in hand.  In the normal use case, this
will only be a couple of text lines.  However, at the extreme it could  
be hundreds of lines of mismatches.  I really don't care how I
accomplish the transition to nextPage.jsp, so if there's a better way  
than what I'm attempting here, please let me know.
{Anything written on the web would be great.}  [I assume that I could  
store the partialMatch data in the db under some appropriate key,  
get to nextPage using response.sendRedirect, and then retrieve the  
info, but that seems like more of a hack than ought to be necessary  
here.]


Thanks in advance,
Ken





-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread Pid

On 19/04/2010 17:06, Ken Bowen wrote:
 Using Tomcat 6.0.18 (to be 6.0.26) and Google App Engine, for parallel
 development (different db tech)
 
 Short form:  I need to accept a file upload in a servlet, do some
 computations on the upload, and then transition to a JSP with some data
 resulting from the computations. I'm having some trouble with the
 transition at the end.  I need either to make the current code work or
 find an
 alternate way to achieve the goal.
 
 Long form.  Here's the html for file upload (vanilla):
 
 form name=csvUploadForm action=csvfileupload method=post
 enctype=multipart/form-data
 File:input type=file name=csvfile2uploadbr/br
 input type=submit name=Submit value=Upload CSV File
 onclick=uploadCSVFile();return false;
 /form
 
 Here's the servlet mapping for the receiving/processing servlet:
 
 servlet-mapping
 servlet-nameCSVFileUpload/servlet-name
 url-pattern/csvfileupload/*/url-pattern
 /servlet-mapping

Why use a wildcard instead of an absolute/explicit path, in both the
action attribute and in the url-pattern?

 The CSVFileUpload servlet doPost method uses
 org.apache.commons.fileupload.servlet.ServletFileUpload;
 to accept and process the upload, and puts the resulting data into the
 db.  If I don't care about duplicates or partial matches,
 this works fine; at the end of the servlet processing, I execute
 response.sendRedirect(/myStartPage.jsp);
 -- where response is the doPost's HttpServletResponse.
 
 Now I need to deal with partial matches between incoming data from the
 upload, and information already in the db. Specifically,
 I need to interact with the user to determine how to resolve the
 ambiguities.  I /thought/ I could create a bean with the necessary computed
 info, use setAttribute to attach the bean to request (the doPost's
 HttpServletRequest), and then forward to a JSP with code like this:

 String nextJSP = /nextPage.jsp;
 RequestDispatcher dispatcher =
 getServletContext().getRequestDispatcher(nextJSP);
 dispatcher.forward(request, response);

/myStartPage.jsp or /nextPage.jsp?

When does the redirect occur and when does the forward occur, it's not
entirely clear.

 Unfortunately, I'm running into the following errors showing in the log:
 
 WARNING: Nested in javax.servlet.ServletException:
 org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException: the
 request doesn't contain a multipart/form-data or multipart/mixed stream,
 content type header is null:
 org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException: the
 request doesn't contain a multipart/form-data or multipart/mixed stream,
 content type header is null
 at
 org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.init(FileUploadBase.java:885)
 
 at
 org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)
 
 at
 org.apache.commons.fileupload.servlet.ServletFileUpload.getItemIterator(ServletFileUpload.java:148)
 
 at
 com.formrunner.servlets.CSVFileUploadServlet.doPost(CSVFileUploadServlet.java:53)
 
 at
 com.formrunner.servlets.CSVFileUploadServlet.doGet(CSVFileUploadServlet.java:29)

What's at CSVFileUploadServlet.java:53?

 The browser actually displays the nextPage.jsp page.  However, if one
 then clicks any navigation button, you get a version of the warning
 above showing in the browser.

So you're catching the exception before executing the forward?

 I'm assuming that the CSVFileUpload servlet processing of the upload has
 stripped the request; hence the errors.

I'm not sure I concur, it seems a little unclear as to what is really
happening at this point.


p


 My goal is to get from the CSVFileUpload servlet to the nextPage.jsp
 with the partialMatch data in hand.  In the normal use case, this
 will only be a couple of text lines.  However, at the extreme it could
 be hundreds of lines of mismatches.  I really don't care how I
 accomplish the transition to nextPage.jsp, so if there's a better way
 than what I'm attempting here, please let me know.
 {Anything written on the web would be great.}  [I assume that I could
 store the partialMatch data in the db under some appropriate key, get
 to nextPage using response.sendRedirect, and then retrieve the info, but
 that seems like more of a hack than ought to be necessary here.]
 
 Thanks in advance,
 Ken
 
 
 
 
 
 -
 To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
 For additional commands, e-mail: users-h...@tomcat.apache.org
 




signature.asc
Description: OpenPGP digital signature

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread André Warnier


Ken Bowen wrote:
...



Long form.  Here's the html for file upload (vanilla):

form name=csvUploadForm action=csvfileupload method=post 
enctype=multipart/form-data

File:input type=file name=csvfile2uploadbr/br
input type=submit name=Submit value=Upload CSV File 
onclick=uploadCSVFile();return false;

/form

Without and before getting to the Java stuff, I do not really understand 
why above you have a button of type submit, but with an onClick 
event triggering a javascript function.
Either the button should be of type button and have an onClick event 
handler, or it should be of type submit and not have an event handler.
I suspect that by specifying both, you may be generating 2 actions and 
confusing the browser and/or the receiving end.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Ken,

On 4/19/2010 12:06 PM, Ken Bowen wrote:
form name=csvUploadForm action=csvfileupload method=post
enctype=multipart/form-data File:input type=file
name=csvfile2uploadbr/br input type=submit name=Submit
value=Upload CSV File onclick=uploadCSVFile();return false;
/form

Looks good, except for that uploadCSVFile javascript trigger. Why not
just do a regular file upload? Is this some kinda AJAX thing?

The CSVFileUpload servlet doPost method uses
org.apache.commons.fileupload.servlet.ServletFileUpload; to accept
and process the upload, and puts the resulting data into the db. If
I don't care about duplicates or partial matches, this works fine; at
the end of the servlet processing, I execute
response.sendRedirect(/myStartPage.jsp); -- where response is the
doPost's HttpServletResponse.

That seems fine (ignoring the mismatch between myStartPage.jsp and
nextPage.jsp).

String nextJSP = /nextPage.jsp; RequestDispatcher dispatcher =
getServletContext().getRequestDispatcher(nextJSP);
dispatcher.forward(request, response);

That's the correct way to do a forward.

Unfortunately, I'm running into the following errors showing in the
log:

WARNING: Nested in javax.servlet.ServletException:
org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException:
the request doesn't contain a multipart/form-data or multipart/mixed
stream, content type header is null:
org.apache.commons.fileupload.FileUploadBase$InvalidContentTypeException:
the request doesn't contain a multipart/form-data or multipart/mixed
stream, content type header is null at
org.apache.commons.fileupload.FileUploadBase$FileItemIteratorImpl.init(FileUploadBase.java:885)

at
org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)

at
org.apache.commons.fileupload.servlet.ServletFileUpload.getItemIterator(ServletFileUpload.java:148)

at
com.formrunner.servlets.CSVFileUploadServlet.doPost(CSVFileUploadServlet.java:53)

at
com.formrunner.servlets.CSVFileUploadServlet.doGet(CSVFileUploadServlet.java:29)

From

the information you've given, it looks like the
CSVFileUploadServlet is being invoked on a request that doesn't have the
proper formatting. You might want to change your CSVFileUploadServlet to
check the Content-Type of the request before invoking the
commons-file-upload stuff so you can give a better error message to your
users.

This could be happening due to a couple of reasons:

1. Your URL mapping is too wide-reaching and the CVS upload servlet is
handling the request to /nextPage.jsp, which would be weird.
2. Your are seeing an error for a request other than the one you think
you are.

I recommend checking for the Content-Type and then dumping a bunch of
information about the request if it's not multipart/form-data: things
like the URL, method and maybe the parameters, too.

The browser actually displays the nextPage.jsp page. However, if
one then clicks any navigation button, you get a version of the
warning above showing in the browser.

Do you have a link like a href= that could be going to the wrong
place? Presumably, navigation links shouldn't take you to the
CSVFileUploadServlet... only form POSTs.

My goal is to get from the CSVFileUpload servlet to the nextPage.jsp
with the partialMatch data in hand.

...whatever that is.

In the normal use case, this will only be a couple of text lines.
However, at the extreme it could be hundreds of lines of mismatches.
I really don't care how I accomplish the transition to nextPage.jsp,
so if there's a better way than what I'm attempting here, please let
me know. {Anything written on the web would be great.} [I assume
that I could store the partialMatch data in the db under some
appropriate key, get to nextPage using response.sendRedirect, and
then retrieve the info, but that seems like more of a hack than ought
to be necessary here.]

Keeping state in the request is always risky, because after it's over,
the user has to re-submit everything in order to basically see the same
result. It's idempotent, but not particularly elegant.

Keeping state in the session is always risky because the session can
expire /and/ you can also bust your heap if it's a lot of data. If
session timeouts are a concern, you have to encode a bunch of
information in the request to recover the session in those cases.

Putting the data into the database is not particularly convenient, but
it will save you from worries about memory exhaustion as well as having
to repeatedly shuttle lots of data from the client to the server and
back. Think: do you want your users to have to re-upload files after the
mismatches have been identified and resolved? Or, do you just want to
apply whatever mitigation steps have been chosen by the user on the data
already on the server?

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread Christopher Schultz

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

André,

On 4/19/2010 5:19 PM, André Warnier wrote:
 Ken Bowen wrote:
 ...
 

 Long form.  Here's the html for file upload (vanilla):

 form name=csvUploadForm action=csvfileupload method=post
 enctype=multipart/form-data
 File:input type=file name=csvfile2uploadbr/br
 input type=submit name=Submit value=Upload CSV File
 onclick=uploadCSVFile();return false;
 /form

 Without and before getting to the Java stuff, I do not really understand
 why above you have a button of type submit, but with an onClick
 event triggering a javascript function.
 Either the button should be of type button and have an onClick event
 handler, or it should be of type submit and not have an event handler.

An exception is the case where you want a page that works both with and
without Javascript enabled: the input type=submit allows the form to
submit in the standard way when no Javascript is present, but allows the
(presumably mind-blowingly fantastic) Javascript alternative to run when
appropriate.

You can't get this behavior from type=button.

 I suspect that by specifying both, you may be generating 2 actions and
 confusing the browser and/or the receiving end.

The return false; at the end of the trigger indicates that the event
handler will (duh) return false. When that happens, the browser is
supposed to basically drop the event and not send it to the default
handler, which would result in the form being submitted in the
traditional way. This is actually a pretty standard Javascript
implementation.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkvMyiUACgkQ9CaO5/Lv0PBb9wCeJZcEXufKdj7RG6MhAz8eIG0p
o3gAoMCjAaMOmzhjs2q8mzlZiwSs7Yx/
=1Z8E
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Problem moving from file upload servlet to JSP

2010-04-19 Thread Propes, Barry L

You sure about that? I've got mine that way (button of type submit, but with 
an onClick 
event triggering a javascript function.) and it works fine. 


 André Warnier wrote:
...

 
 Long form.  Here's the html for file upload (vanilla):
  
   form name=csvUploadForm action=csvfileupload method=post 
  enctype=multipart/form-data
 File:input type=file name=csvfile2uploadbr/br
   input type=submit name=Submit value=Upload CSV File 
   onclick=uploadCSVFile();return false; /form
 
Without and before getting to the Java stuff, I do not really understand why 
above you have a button of type submit, but with an onClick 
event triggering a javascript function.
Either the button should be of type button and have an onClick event handler, 
or it should be of type submit and not have an event handler.
I suspect that by specifying both, you may be generating 2 actions and 
confusing the browser and/or the receiving end.



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread André Warnier


Christopher Schultz wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

André,

On 4/19/2010 5:19 PM, André Warnier wrote:

Ken Bowen wrote:
...


Long form.  Here's the html for file upload (vanilla):

form name=csvUploadForm action=csvfileupload method=post
enctype=multipart/form-data
File:input type=file name=csvfile2uploadbr/br
input type=submit name=Submit value=Upload CSV File
onclick=uploadCSVFile();return false;
/form


Without and before getting to the Java stuff, I do not really understand
why above you have a button of type submit, but with an onClick
event triggering a javascript function.
Either the button should be of type button and have an onClick event
handler, or it should be of type submit and not have an event handler.


An exception is the case where you want a page that works both with and
without Javascript enabled: the input type=submit allows the form to
submit in the standard way when no Javascript is present, but allows the
(presumably mind-blowingly fantastic) Javascript alternative to run when
appropriate.

You can't get this behavior from type=button.


I suspect that by specifying both, you may be generating 2 actions and
confusing the browser and/or the receiving end.


The return false; at the end of the trigger indicates that the event
handler will (duh) return false. When that happens, the browser is
supposed to basically drop the event and not send it to the default
handler, which would result in the form being submitted in the
traditional way. This is actually a pretty standard Javascript
implementation.


Ok, I'll accept that explanation.
Which still leaves us in the dark concerning the content of that 
javascript uploadCSVFile() function which is probably the one used in 
this case then to post the form content to the server.
I must admit I still don't see the need of it in this case, since the 
form as it is without the onClick would submit the file just as well.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

RE: Problem moving from file upload servlet to JSP

2010-04-19 Thread Propes, Barry L

 My guess is that he may have other various pieces of validation tied to it, 
client side.

And he might even have some client-side validation intertwined for the type of 
file in an array, - i.e. .xls, .doc., .txt etc. as an acceptable file type to 
upload, though, like you, I have no idea not having seen all of his form and 
code accompanying it.

-Original Message-
From: André Warnier  wrote:

 The return false; at the end of the trigger indicates that the event 
 handler will (duh) return false. When that happens, the browser is 
 supposed to basically drop the event and not send it to the default
 handler, which would result in the form being submitted in the 
 traditional way. This is actually a pretty standard Javascript 
 implementation.
 
Ok, I'll accept that explanation.
Which still leaves us in the dark concerning the content of that javascript 
uploadCSVFile() function which is probably the one used in this case then to 
post the form content to the server.
I must admit I still don't see the need of it in this case, since the form as 
it is without the onClick would submit the file just as well.

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Problem moving from file upload servlet to JSP

Just aritfacts of step by step changes (as I come to understand  
various thingies):  should be type button, but gives no evidence of  
two events.


On Apr 19, 2010, at 5:19 PM, André Warnier wrote:


Ken Bowen wrote:
...


Long form.  Here's the html for file upload (vanilla):
form name=csvUploadForm action=csvfileupload method=post  
enctype=multipart/form-data

   File:input type=file name=csvfile2uploadbr/br
   input type=submit name=Submit value=Upload CSV File  
onclick=uploadCSVFile();return false;

/form
Without and before getting to the Java stuff, I do not really  
understand why above you have a button of type submit, but with an  
onClick event triggering a javascript function.
Either the button should be of type button and have an onClick  
event handler, or it should be of type submit and not have an  
event handler.
I suspect that by specifying both, you may be generating 2 actions  
and confusing the browser and/or the receiving end.




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

FollowUP: Problem moving from file upload servlet to JSP


Thanks for all the responses.
1.  After posting the original, I thought about it all at lunch, and  
was leaning towards the db solution, and Chris firmly pushed me over  
on that.  Now I just convert the byte stream to a (sometimes big)  
string, and stuff it in a temporary db place using the user's login  
and the current-time-in-millis as a combined key.  The I sendRedirect  
to nextPage.jsp?t=current-time-in-millis and do all the work at that  
page.


2.  Barry basically has it right as far as my using the call to  
javascript:  there is some validation and housekeeping concerning the  
file and user/organization logins going on before the actual upload.


thanks  cheers,
Ken

On Apr 19, 2010, at 5:59 PM, Propes, Barry L wrote:

My guess is that he may have other various pieces of validation tied  
to it, client side.


And he might even have some client-side validation intertwined for  
the type of file in an array, - i.e. .xls, .doc., .txt etc. as an  
acceptable file type to upload, though, like you, I have no idea not  
having seen all of his form and code accompanying it.


-Original Message-
From: André Warnier  wrote:

The return false; at the end of the trigger indicates that the  
event

handler will (duh) return false. When that happens, the browser is
supposed to basically drop the event and not send it to the default
handler, which would result in the form being submitted in the
traditional way. This is actually a pretty standard Javascript
implementation.


Ok, I'll accept that explanation.
Which still leaves us in the dark concerning the content of that  
javascript uploadCSVFile() function which is probably the one used  
in this case then to post the form content to the server.
I must admit I still don't see the need of it in this case, since  
the form as it is without the onClick would submit the file just as  
well.


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Re: Problem moving from file upload servlet to JSP

2010-04-19 Thread Pid

On 19/04/2010 22:21, Christopher Schultz wrote:
Ken,

Looks good, except for that uploadCSVFile javascript trigger. Why not
just do a regular file upload? Is this some kinda AJAX thing?

That seems fine (ignoring the mismatch between myStartPage.jsp and
nextPage.jsp).

... and that one is a redirect one is a forward?

p (confused)

String nextJSP = /nextPage.jsp; RequestDispatcher dispatcher =
getServletContext().getRequestDispatcher(nextJSP);
dispatcher.forward(request, response);

That's the correct way to do a forward.

Unfortunately, I'm running into the following errors showing in the
log:

at
org.apache.commons.fileupload.FileUploadBase.getItemIterator(FileUploadBase.java:331)

at
org.apache.commons.fileupload.servlet.ServletFileUpload.getItemIterator(ServletFileUpload.java:148)

at
com.formrunner.servlets.CSVFileUploadServlet.doPost(CSVFileUploadServlet.java:53)

at
com.formrunner.servlets.CSVFileUploadServlet.doGet(CSVFileUploadServlet.java:29)

From

This could be happening due to a couple of reasons:

I recommend checking for the Content-Type and then dumping a bunch of
information about the request if it's not multipart/form-data: things
like the URL, method and maybe the parameters, too.

The browser actually displays the nextPage.jsp page. However, if
one then clicks any navigation button, you get a version of the
warning above showing in the browser.

Do you have a link like a href= that could be going to the wrong
place? Presumably, navigation links shouldn't take you to the
CSVFileUploadServlet... only form POSTs.

My goal is to get from the CSVFileUpload servlet to the nextPage.jsp
with the partialMatch data in hand.

...whatever that is.

Keeping state in the request is always risky, because after it's over,
the user has to re-submit everything in order to basically see the same
result. It's idempotent, but not particularly elegant.

Re: Problem moving from file upload servlet to JSP