Re: Practical multipart handling
чт, 26 мар. 2020 г. в 18:03, Christopher Schultz : > > All, > > I'm developing my first multipart handler since .. I dunno, maybe > 2005? This is the first time I'll be using the Servlet 3.0 multipart > handling, of course through Tomcat. Some of these questions may have > answers which are "implementation-specific", so in this case, I would > like to know how things will behave in Tomcat specifically. Notes of > where the spec leaves things up to the implementation will be appreciate > d. > > I'd like to submit a form which has not only a large-ish file part, > but also some regular fields like . My > understanding is that I'll have to read those data by calling > Part.getInputStream(), wrapping the InputStream in an > InputStreamReader using the right charset, etc. I think that those are available via the standard request.getParameter(name) API. > [...] > > Can I rely on the client to send the fields in any particular order? > I'm not expecting to store the file on the server myself; I'd like to > process it in a "streaming" fashion and not touch the disk if > possible. I know that the server may store the file on the disk if it > decides to. I'm not terribly worried about that. I just don't want to > have to write the file to the disk TWICE, and I need information from > those other parameters in order to configure the stream-processing. Michael already answered this. There is a configurable threshold. Anything over it will be written to disk as a temporary file. The JavaDoc for Part.write() says that it can be implemented as moving the file. "This method is not guaranteed to succeed if called more than once" > When iterating over the Collection returned from > HttpServletRequest.getParts(), am I required to process each part in > order immediately? Or can I store a reference to a Part for later? > This kind of goes along with the previous question. You can store the reference, but your "for later" should be no longer than until the request processing ends. > When I'm done with a part, must I explicitly call Part.delete()? Tomcat deletes the files automatically (I implemented this feature in Tomcat 7.0.30 - see changelog). In my own web applications I delete the files explicitly (calling part.delete() in a cycle). Best regards, Konstantin Kolinko - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Does Tomcat/Java get around the problem of 64K maximum client source ports?
> -Original Message- > From: Olaf Kock > Sent: Thursday, March 26, 2020 2:06 PM > To: users@tomcat.apache.org > Subject: Re: Does Tomcat/Java get around the problem of 64K maximum > client source ports? > > Hi Eric, > > On 26.03.20 18:58, Eric Robinson wrote: > > Greetings, > > > > Many people say the maximum number of client ports is 64K. However, > TCP connections only require unique sockets, which are defined as... > > > > local_IP:local_port -> remote_ip:remote_port > > > > Theoretically, it is possible for a client process to keep using the same > > local > source port, as long as the connections are to a unique destinations. For > example on a local machine, the following connections should be possible... > > > > 192.168.5.100:1400 -> 192.168.5.200:3306 > > 192.168.5.100:1400 -> 192.168.5.201:3306 > > 192.168.5.100:1400 -> 192.168.5.202:3306 > > 192.168.5.100:1400 -> 192.168.5.203:3306 > > > > I've seen this demonstrated successfully here: > > > > https://serverfault.com/questions/326819/does-the-tcp-source-port-have > > -to-be-unique-per-host > > > > As someone on that page pointed out, while it is possible, it does not > commonly occur in practice "because most TCP APIs don't provide a way to > create more than one connection with the same source port, unless they > have different source IP addresses." This leads to the 64K maximum client > port range, but it is really a limitation of the APIs, not TCP. > > > > So how does tomcat handle things? Is it limited to a maximum 64K client > source ports, or is it 64K per destination, as it should be? > > To be honest, I can't remember to have seen a web- or application server > that accepts 64K concurrent connections at all, let alone to a single client. > > I can't come up with any reason to do so, I'd assume that there's a DOS attack > if I get 100 concurrent incoming connections from a single IP, and a > programming error if the server sets up more than 1K outgoing connections > > Maybe I'm missing the obvious, or have only administered meaningless > installations, but I fail to see the real world relevance of this question. > > I don't blame you for being puzzled, but this not about tomcat accepting connections. It's about tomcat acting as the client, where MySQL is the server. I'm referring to client connections from tomcat to MySQL. We have about 1800 instances of tomcat running. This question comes up once in a while when tomcat can't connect to MySQL. Trust me, it can be an issue. --Eric Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Does Tomcat/Java get around the problem of 64K maximum client source ports?
Hi Eric, On 26.03.20 18:58, Eric Robinson wrote: > Greetings, > > Many people say the maximum number of client ports is 64K. However, TCP > connections only require unique sockets, which are defined as... > > local_IP:local_port -> remote_ip:remote_port > > Theoretically, it is possible for a client process to keep using the same > local source port, as long as the connections are to a unique destinations. > For example on a local machine, the following connections should be > possible... > > 192.168.5.100:1400 -> 192.168.5.200:3306 > 192.168.5.100:1400 -> 192.168.5.201:3306 > 192.168.5.100:1400 -> 192.168.5.202:3306 > 192.168.5.100:1400 -> 192.168.5.203:3306 > > I've seen this demonstrated successfully here: > > https://serverfault.com/questions/326819/does-the-tcp-source-port-have-to-be-unique-per-host > > As someone on that page pointed out, while it is possible, it does not > commonly occur in practice "because most TCP APIs don't provide a way to > create more than one connection with the same source port, unless they have > different source IP addresses." This leads to the 64K maximum client port > range, but it is really a limitation of the APIs, not TCP. > > So how does tomcat handle things? Is it limited to a maximum 64K client > source ports, or is it 64K per destination, as it should be? To be honest, I can't remember to have seen a web- or application server that accepts 64K concurrent connections at all, let alone to a single client. I can't come up with any reason to do so, I'd assume that there's a DOS attack if I get 100 concurrent incoming connections from a single IP, and a programming error if the server sets up more than 1K outgoing connections Maybe I'm missing the obvious, or have only administered meaningless installations, but I fail to see the real world relevance of this question. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Strange side effect of "antiClickJackingOption" clause in "httpHeaderSecurity"
On 3/24/20 2:25 PM, Christopher Schultz wrote: I don't understand exactly how X-Frame-Options (which is what the HttpHeaderSecurityFilter is configuring) is being used by your application, but I believe X-Frame-Options is essentially being replaced by various features of Content-Security-Policy. You may want to talk to your engineers about using one of those versus the other; you may want to discontinue using the "anti click-jacking" features of this filter altogether in favor of something else. Dear Mr. Schultz (et al): Thanks. Our webapp team combined your answer with what we could glean from the customer's security audit, and has come up with a solution involving a Content-Security-Policy, using a class he added to the webapp. I'm not sure, but I think it can be built into the WAR file. On our own Tomcat server, we have another webapp that cannot have clickjack protection via a HTTPHeaderSecurity filter with antiClickJackingOption SAMEORIGIN because it's specifically designed to go into a frame, in a page served from an entirely different server. Is that what the "ALLOW-FROM" option and the associated "antiClickJackingUri" parameter are for? -- JHHL - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Practical multipart handling
Am 2020-03-26 um 16:03 schrieb Christopher Schultz: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, I'm developing my first multipart handler since .. I dunno, maybe 2005? This is the first time I'll be using the Servlet 3.0 multipart handling, of course through Tomcat. Some of these questions may have answers which are "implementation-specific", so in this case, I would like to know how things will behave in Tomcat specifically. Notes of where the spec leaves things up to the implementation will be appreciate d. I'd like to submit a form which has not only a large-ish file part, but also some regular fields like . My understanding is that I'll have to read those data by calling Part.getInputStream(), wrapping the InputStream in an InputStreamReader using the right charset, etc. Do I get any help from any existing APIs for doing that, or do I have to pull the Content-Type out of the Part-headers, look for a "charset" parameter, etc., etc., in order to get the encoding and convert to a String value? It seems like a pretty big hole in the multipart API not to have something like that already available. Can I rely on the client to send the fields in any particular order? I'm not expecting to store the file on the server myself; I'd like to process it in a "streaming" fashion and not touch the disk if possible. I know that the server may store the file on the disk if it decides to. I'm not terribly worried about that. I just don't want to have to write the file to the disk TWICE, and I need information from those other parameters in order to configure the stream-processing. When iterating over the Collection returned from HttpServletRequest.getParts(), am I required to process each part in order immediately? Or can I store a reference to a Part for later? This kind of goes along with the previous question. When I'm done with a part, must I explicitly call Part.delete()? A mere coincidence, we are currently designing an API around multipart/related and I have investigated multipart requests deeply after 10 years of abstinence. W/o going to much in detail: * Tomcat uses Commons File Upload w/ all its flaws and benefits * Commons FU starts writing to disk as soon as it hits a threshold * Don't expect an order unless specified by RFC 7578 * Browsers may order, read https://html.spec.whatwg.org/multipage/forms.html#forms * Streaming will be hard/impossible with Commons FU when not ordered. It reads off the non-repeatable input stream which has to be segmented on the fly into parts with headers and embedded payload. Consider that you have payload beyond your huge payload. You simply cannot access it unless you have consumed the huge one. In your case, if there are form fields after the huge payload you need to decide how to process the huge payload you *must* cache locally. * You must call delete, otherwise the temporary file will left on disk I think your best bet is either: http://commons.apache.org/proper/commons-fileupload/streaming.html or https://james.apache.org/mime4j/ Good luck, M - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Practical multipart handling
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 All, I'm developing my first multipart handler since .. I dunno, maybe 2005? This is the first time I'll be using the Servlet 3.0 multipart handling, of course through Tomcat. Some of these questions may have answers which are "implementation-specific", so in this case, I would like to know how things will behave in Tomcat specifically. Notes of where the spec leaves things up to the implementation will be appreciate d. I'd like to submit a form which has not only a large-ish file part, but also some regular fields like . My understanding is that I'll have to read those data by calling Part.getInputStream(), wrapping the InputStream in an InputStreamReader using the right charset, etc. Do I get any help from any existing APIs for doing that, or do I have to pull the Content-Type out of the Part-headers, look for a "charset" parameter, etc., etc., in order to get the encoding and convert to a String value? It seems like a pretty big hole in the multipart API not to have something like that already available. Can I rely on the client to send the fields in any particular order? I'm not expecting to store the file on the server myself; I'd like to process it in a "streaming" fashion and not touch the disk if possible. I know that the server may store the file on the disk if it decides to. I'm not terribly worried about that. I just don't want to have to write the file to the disk TWICE, and I need information from those other parameters in order to configure the stream-processing. When iterating over the Collection returned from HttpServletRequest.getParts(), am I required to process each part in order immediately? Or can I store a reference to a Part for later? This kind of goes along with the previous question. When I'm done with a part, must I explicitly call Part.delete()? - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl58xDUACgkQHPApP6U8 pFi6UxAAuwqHuLd7Xa1mo6yr+MwbltHo+3xauIgyr1IF9TIseAMeBAoNJYxgM7t2 TpZT6ZGupCQjblJ5Nd3m+W8IvtIEoOjFuILg7PIvmz+CzvPWNvmWMecIO55UW41Z khf5jX686eLelKqf8VkBkYTdwpYpvS/B+WaWgwwGS0wKsq2/Z4xiYBcr/lqaY9Hh SMeSmrfil7+ezGwf4YqdDANG8pORkGZi0NDDwrmlapS3fwKp9gQxKdsnu9Aq3YCI R4Dpydqo6q+XrMxT8jJXIFTjUwqCS8LosKaRoqhunzeQ2uv1guvKhjwmCwVyoK1k yPX6jojsR3CjChKbU5JCT0t7bierZ+Pem1Ihx5kv5fIfOf2hvhFwq7v/4MsCACZb y7PgHEXDf/r65X9w0vJ4phmQKPUWFjtKnU7GStCmD9Th/WIM9HslXDIZxzBehoOw HXQm6A4Terk+9183M5V2PIisHdR+WY2DhPBCat/j5s4+SMScSNcrt+ovpKQbpUAA hnosaCK+rbEKdnKFndT/2dWE5k2z2faGKwqSwAUnrFep0gSFXr3f6mrPSygw3dig 65N5tTicrmqLeXl7VjppaXuXya5wJtbC6nk+mREyyiDvni+lu+3wemqzKHAF5PT5 comqAGuLCWZiJEV53X6Xo5mfqjIklHbZIjHTHlxCqsJR/zvpg6Y= =aFQX -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Connector compressibleMimeType
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Martynas, On 3/26/20 07:17, Martynas Jusevičius wrote: > I enabled GZip compression on using 8.0.44. You need to upgrade. Support for Tomcat 8.0 ended nearly 2 years ago: http://tomcat.apache.org/tomcat-80-eol.html > Then I noticed that text/html documents are getting compressed, > but others were not. > > I guess this is due to the default compressibleMimeType? Which is > "text/html,text/xml,text/plain,text/css,text/javascript,application/ja vascript". > > https://tomcat.apache.org/tomcat-8.0-doc/config/http.html (We should probably add: application/json, application/xml, application/xhtml+xml, and application/soap+xml. Some Tomcat versions already have these in the default default.) > What would be the downside if I specified compressibleMimeType to > compress all media types? Including binary files etc. I guess that > would be compressibleMimeType="*/*". This will not work, since Tomcat doesn't perform MIME-type globbing or pattern-matching. The docs say it's a list of MIME types, not a pattern or anything like that. If you were to enable it for all types, you may compress already-compressed data. That usually results in an increase in size, so wasted effort all around. What kinds of media-types are you hoping to compress that aren't already on the default list? - -chris -BEGIN PGP SIGNATURE- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl58o+cACgkQHPApP6U8 pFi/QRAAgkcGyZNMAagBsO0/KT29N1MbA97B9mHsvvWARbFW1dFd6jtHCsG3vtTN dsN4ZoEtaeUQ3gUzXXP5z4BwQqtFROywyYC29Td7/uh5T2Ege7shL22Y8zvSUwel NxL2cveQuFfKde37fwlU/mvGvMhoQ6Q+ZmXTm1yk64QAyJ/p80pzleleMKrlWtS0 QJ7bVK5MYRagVcHIa7DsiEgovi4eeC715IOLK3rnNHWvr5o1Gxwsi0223+0pQppK x6ZmFAkC4t4EyZnuOyQ/5y2but0PswtVK5TYjkCyhQibyFGk8Y1g3eYdluZk2UXp M6ISQJSXwBORJXpX6apXBDGgnb7tjfRbfugNSVIlfCR9z+eRESiJ0hKbQdLqzW0L b9IckMIqB11/b3wIGpkFiIHnufDiBvdPsPiuhcHZL+RWxs5RcCRWtNc3oV14MTaC tU1rgSCcHJ+dIazC7OT8vz/MxiE2JAoUPPSHpFKLCaOSqTZvATDi0L7Dw5Fl14g0 /c9j6IGGhkeQStYX6f1dZo8P7HKuc8vNBM3bMxj0p1av9oIWtnpgjU1gRljIzbe0 IZn+9IRqUHZHqot712mF7J8OcUJBW2QdCzwTIKWundbFQmsbfW9QigqA01lNhNEe hFZJ565HxydDhr8VKEA6Cfvyec4CXdVv9CapS3DTuy4OhGvWIRs= =Vs9P -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Connector compressibleMimeType
Hi, I enabled GZip compression on using 8.0.44. Then I noticed that text/html documents are getting compressed, but others were not. I guess this is due to the default compressibleMimeType? Which is "text/html,text/xml,text/plain,text/css,text/javascript,application/javascript". https://tomcat.apache.org/tomcat-8.0-doc/config/http.html What would be the downside if I specified compressibleMimeType to compress all media types? Including binary files etc. I guess that would be compressibleMimeType="*/*". Thanks. Martynas - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
