Re: How to disable chunked encoding for the Http11NioProtocol connector.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/5/2011 4:29 PM, ilya goberman wrote: I was thinking more about it. What if Tomcat disables chunked encoding if response contains Connection: close header. I think that's what Mark has suggested several times, now: On 1/5/2011 12:54 PM, Mark Thomas wrote: As I said, an enhancement request to optionally use a non-chunked response when keep-alive is disabled is - in my view - a reasonable one. On 1/5/2011 4:29 PM, ilya goberman wrote: So in order to disable the encoding the Tomcat application will have to set just one response header. ... and set a configuration parameter in Tomcat to enable this feature. Apparently, the feature as you describe it was a problem fixed in a previous version of Tomcat and won't be reverted unless a serious regression is identified. Mark is offering to allow configuration to re-enable this behavior. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0l+AUACgkQ9CaO5/Lv0PAFbwCeNzfxtyfbDFvvzdKekcVCkSTC FQ0AoI4kc2YFrKta7Q+TFL8Cap68fc3z =j0Ua -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 André, On 1/5/2011 5:13 PM, André Warnier wrote: 4.3 Message Body ... Transfer-Encoding is a property of the message, not of the entity, and thus MAY be added or removed by any application along the request/response chain. Maybe the OP should use Apache httpd with mod_blow_chunks or something similar out in front. Does such a beast exist? I'm sure you could whip one up in a few minutes: all it takes is a little message buffering which, if the OP is serious about messages only being a few bytes, won't represent undue burden on the server. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0l+N8ACgkQ9CaO5/Lv0PC4gACfbZVIG8sJSML5byP7a+S6q5e5 2AwAn36XwVzERdyRlcnRSiDw9ZWm2pZW =E43B -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
I think what was suggested before is to control this behavior by setting keepAlive setting. I would not think this is the right way. I would still want to preserve keepAlive functiojnality for all other requests except long running comet response. So if it is request for a web page, using keepAlive is fine. Now in order to disable chunked encoding for a particular response, I would propose that a developer would set Connection:close header. Using connection close implies that closing the connection indicates the end of response and chunked encoding is not necessary in this case. Date: Thu, 6 Jan 2011 12:12:37 -0500 From: ch...@christopherschultz.net To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/5/2011 4:29 PM, ilya goberman wrote: I was thinking more about it. What if Tomcat disables chunked encoding if response contains Connection: close header. I think that's what Mark has suggested several times, now: On 1/5/2011 12:54 PM, Mark Thomas wrote: As I said, an enhancement request to optionally use a non-chunked response when keep-alive is disabled is - in my view - a reasonable one. On 1/5/2011 4:29 PM, ilya goberman wrote: So in order to disable the encoding the Tomcat application will have to set just one response header. ... and set a configuration parameter in Tomcat to enable this feature. Apparently, the feature as you describe it was a problem fixed in a previous version of Tomcat and won't be reverted unless a serious regression is identified. Mark is offering to allow configuration to re-enable this behavior. - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0l+AUACgkQ9CaO5/Lv0PAFbwCeNzfxtyfbDFvvzdKekcVCkSTC FQ0AoI4kc2YFrKta7Q+TFL8Cap68fc3z =j0Ua -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/6/2011 12:27 PM, ilya goberman wrote: I think what was suggested before is to control this behavior by setting keepAlive setting. I would not think this is the right way. Er, what's the difference between using keepAlive setting and Connection: close? AFAIK, they are the same thing. I would still want to preserve keepAlive functionality for all other requests except long running comet response. Of course. So, you have the client set Connection: close (which disables keep-alive) and then the configuration Mark proposed causes chunked encoding to be avoided. So if it is request for a web page, using keepAlive is fine. Now in order to disable chunked encoding for a particular response, I would propose that a developer would set Connection:close header. On the server side or from the client side? I'm not sure if Mark was thinking client or server. I suppose server would be safer, since the server can make the decision and not have to ensure that clients are compliant. Using connection close implies that closing the connection indicates the end of response and chunked encoding is not necessary in this case. So... everyone is happy, then, right? Are we arguing semantics, then? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0mL1QACgkQ9CaO5/Lv0PBxKwCgqjbxMKY5WKqdVLR8Ay9TpA+h ojwAmwdQKzXFqiumBd8j40zEgym9NOeR =2fnQ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
OK. All I wanted to say is that disabling keepAlive across the board is not necessary. If keepAlive can be applies to a single response (and to be honest I am not sure it is possible), it is fine. I was under impressing that the only way to disable keep alive is globally via: maxKeepAliveRequests=1 in server.xml Date: Thu, 6 Jan 2011 16:08:36 -0500 From: ch...@christopherschultz.net To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/6/2011 12:27 PM, ilya goberman wrote: I think what was suggested before is to control this behavior by setting keepAlive setting. I would not think this is the right way. Er, what's the difference between using keepAlive setting and Connection: close? AFAIK, they are the same thing. I would still want to preserve keepAlive functionality for all other requests except long running comet response. Of course. So, you have the client set Connection: close (which disables keep-alive) and then the configuration Mark proposed causes chunked encoding to be avoided. So if it is request for a web page, using keepAlive is fine. Now in order to disable chunked encoding for a particular response, I would propose that a developer would set Connection:close header. On the server side or from the client side? I'm not sure if Mark was thinking client or server. I suppose server would be safer, since the server can make the decision and not have to ensure that clients are compliant. Using connection close implies that closing the connection indicates the end of response and chunked encoding is not necessary in this case. So... everyone is happy, then, right? Are we arguing semantics, then? - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0mL1QACgkQ9CaO5/Lv0PBxKwCgqjbxMKY5WKqdVLR8Ay9TpA+h ojwAmwdQKzXFqiumBd8j40zEgym9NOeR =2fnQ -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/6/2011 4:17 PM, ilya goberman wrote: OK. All I wanted to say is that disabling keepAlive across the board is not necessary. Nobody is suggesting that. If keepAlive can be applies to a single response (and to be honest I am not sure it is possible), it is fine. Of course it can: headers can be set on a per-response basis. Otherwise, stuff like Content-Type and Content-Length would never work. I was under impressing that the only way to disable keep alive is globally via: maxKeepAliveRequests=1 in server.xml That's obviously the wrong approach. File another enhancement request to allow configuration to let client and/or server use of Connection:close to disable chunked response encoding and I'm sure Mark won't mark it as INVALID or WONTFIX. It's possible nobody will actually submit a patch for it any time soon, but you are free to do so. Enhancement requests with accompanying patches are much more likely to be fixed. ;) - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0mNXAACgkQ9CaO5/Lv0PCsPACfQpD5JEO8QEDyoaQQFg2TqN7f 3+MAn3N/RJa+j63Wp67tJL5Tw3J6twrY =u9Rn -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
Got it, thanks Date: Thu, 6 Jan 2011 16:34:40 -0500 From: ch...@christopherschultz.net To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Ilya, On 1/6/2011 4:17 PM, ilya goberman wrote: OK. All I wanted to say is that disabling keepAlive across the board is not necessary. Nobody is suggesting that. If keepAlive can be applies to a single response (and to be honest I am not sure it is possible), it is fine. Of course it can: headers can be set on a per-response basis. Otherwise, stuff like Content-Type and Content-Length would never work. I was under impressing that the only way to disable keep alive is globally via: maxKeepAliveRequests=1 in server.xml That's obviously the wrong approach. File another enhancement request to allow configuration to let client and/or server use of Connection:close to disable chunked response encoding and I'm sure Mark won't mark it as INVALID or WONTFIX. It's possible nobody will actually submit a patch for it any time soon, but you are free to do so. Enhancement requests with accompanying patches are much more likely to be fixed. ;) - -chris -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAk0mNXAACgkQ9CaO5/Lv0PCsPACfQpD5JEO8QEDyoaQQFg2TqN7f 3+MAn3N/RJa+j63Wp67tJL5Tw3J6twrY =u9Rn -END PGP SIGNATURE- - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
What is the overhead of ending a tcp connection and creating a new one? Because you are removing the benefits of keep-alive here. Compare that with sending 6 extra bytes in a IP-packet that you are sending anyway. Ronald. Op woensdag, 5 januari 2011 16:29 schreef ilya goberman gober...@msn.com: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
I see your point. But most clients will keep the application open for hours, so bandwidth saving may be more important than keep-alive. I think disabling chunked encoding is appropriate for the long running connections. Unfortunately, some browsers/ mobile devices have bugs associated with chunked encoding, so disabling it is the only way for these to work properly. Date: Wed, 5 Jan 2011 17:01:57 +0100 From: ronald-mailingl...@base.nl To: users@tomcat.apache.org Subject: RE: How to disable chunked encoding for the Http11NioProtocol connector. What is the overhead of ending a tcp connection and creating a new one? Because you are removing the benefits of keep-alive here. Compare that with sending 6 extra bytes in a IP-packet that you are sending anyway. Ronald. Op woensdag, 5 januari 2011 16:29 schreef ilya goberman gober...@msn.com: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
On 05/01/2011 15:29, ilya goberman wrote: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). You are ignoring the TCP/IP overhead. That is around 40 bytes per packet. More if you take account of the ACK. Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. I am well aware of that. I am also well aware that a client that declares itself HTTP 1.1 capable must support chunked encoding. I am frankly astonished that a client is so broken it can't handle chunked encoding. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. Which browsers? While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Sure, feel free to request an enhancement to optionally restore the non-chunked approach when keep-alive is disabled. I'm not sure how much interest there will be in implementing it though. Mark Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
Mark, 1) TCP/IP overhad? Not sure why you got this involved. It applies for both chunked and normal encoding. Certainly, TCP/IP packets can span across multiple chunks or one chunk can be split into multiple packets. Or maybe you are implying that chunked encoding will generate more packets and increase bandwidth even furhter? This is probably true. 2) I am astonished myself, but it is the fact. An example is Android browser: http://code.google.com/p/android/issues/detail?id=13044 3) I already requested an enhancement and you rejected it. Thanks Date: Wed, 5 Jan 2011 16:38:20 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 15:29, ilya goberman wrote: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). You are ignoring the TCP/IP overhead. That is around 40 bytes per packet. More if you take account of the ACK. Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. I am well aware of that. I am also well aware that a client that declares itself HTTP 1.1 capable must support chunked encoding. I am frankly astonished that a client is so broken it can't handle chunked encoding. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. Which browsers? While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Sure, feel free to request an enhancement to optionally restore the non-chunked approach when keep-alive is disabled. I'm not sure how much interest there will be in implementing it though. Mark Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
On 05/01/2011 17:43, ilya goberman wrote: Mark, 1) TCP/IP overhead? Not sure why you got this involved. Because of with the number of bytes in this use case the TCP overhead is significant. It significantly alters the % overhead when comparing chunked and non-chunked. It may or may not alter it enough for you to change your view on the benefits of non-chunked. 2) I am astonished myself, but it is the fact. An example is Android browser: http://code.google.com/p/android/issues/detail?id=13044 Jeez. And I was considering getting an Android phone when my current contract expires. 3) I already requested an enhancement and you rejected it. I rejected your first set of questions since they were support questions and I rejected your request to force a HTTP 1.0 response since there are ways of doing that already. As I said, an enhancement request to optionally use a non-chunked response when keep-alive is disabled is - in my view - a reasonable one. The whole point of directing you to the users list was to have this discussion on the users list so it is in the user archives for future reference. Bugzilla should be focussed on fixing specific issues and is not intended for more general discussion. Mark Thanks Date: Wed, 5 Jan 2011 16:38:20 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 15:29, ilya goberman wrote: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). You are ignoring the TCP/IP overhead. That is around 40 bytes per packet. More if you take account of the ACK. Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. I am well aware of that. I am also well aware that a client that declares itself HTTP 1.1 capable must support chunked encoding. I am frankly astonished that a client is so broken it can't handle chunked encoding. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. Which browsers? While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Sure, feel free to request an enhancement to optionally restore the non-chunked approach when keep-alive is disabled. I'm not sure how much interest there will be in implementing it though. Mark Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
OK, it is fair, thanks. Date: Wed, 5 Jan 2011 17:54:53 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 17:43, ilya goberman wrote: Mark, 1) TCP/IP overhead? Not sure why you got this involved. Because of with the number of bytes in this use case the TCP overhead is significant. It significantly alters the % overhead when comparing chunked and non-chunked. It may or may not alter it enough for you to change your view on the benefits of non-chunked. 2) I am astonished myself, but it is the fact. An example is Android browser: http://code.google.com/p/android/issues/detail?id=13044 Jeez. And I was considering getting an Android phone when my current contract expires. 3) I already requested an enhancement and you rejected it. I rejected your first set of questions since they were support questions and I rejected your request to force a HTTP 1.0 response since there are ways of doing that already. As I said, an enhancement request to optionally use a non-chunked response when keep-alive is disabled is - in my view - a reasonable one. The whole point of directing you to the users list was to have this discussion on the users list so it is in the user archives for future reference. Bugzilla should be focussed on fixing specific issues and is not intended for more general discussion. Mark Thanks Date: Wed, 5 Jan 2011 16:38:20 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 15:29, ilya goberman wrote: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). You are ignoring the TCP/IP overhead. That is around 40 bytes per packet. More if you take account of the ACK. Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. I am well aware of that. I am also well aware that a client that declares itself HTTP 1.1 capable must support chunked encoding. I am frankly astonished that a client is so broken it can't handle chunked encoding. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. Which browsers? While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Sure, feel free to request an enhancement to optionally restore the non-chunked approach when keep-alive is disabled. I'm not sure how much interest there will be in implementing it though. Mark Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view
RE: How to disable chunked encoding for the Http11NioProtocol connector.
I was thinking more about it. What if Tomcat disables chunked encoding if response contains Connection: close header. So in order to disable the encoding the Tomcat application will have to set just one response header. I think it is a reasonable enhancement to do. If Connection: close is not set, keep alive functionality is preserved. Any objections? Date: Wed, 5 Jan 2011 17:54:53 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 17:43, ilya goberman wrote: Mark, 1) TCP/IP overhead? Not sure why you got this involved. Because of with the number of bytes in this use case the TCP overhead is significant. It significantly alters the % overhead when comparing chunked and non-chunked. It may or may not alter it enough for you to change your view on the benefits of non-chunked. 2) I am astonished myself, but it is the fact. An example is Android browser: http://code.google.com/p/android/issues/detail?id=13044 Jeez. And I was considering getting an Android phone when my current contract expires. 3) I already requested an enhancement and you rejected it. I rejected your first set of questions since they were support questions and I rejected your request to force a HTTP 1.0 response since there are ways of doing that already. As I said, an enhancement request to optionally use a non-chunked response when keep-alive is disabled is - in my view - a reasonable one. The whole point of directing you to the users list was to have this discussion on the users list so it is in the user archives for future reference. Bugzilla should be focussed on fixing specific issues and is not intended for more general discussion. Mark Thanks Date: Wed, 5 Jan 2011 16:38:20 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 15:29, ilya goberman wrote: Mark, overhead of chunked encoding can be significant. My typical message is about 50 bytes and chunked encoding takes 6 bytes per message: about 12%. I use JSON protocol that is already compressed (the way JSON can be compressed). You are ignoring the TCP/IP overhead. That is around 40 bytes per packet. More if you take account of the ACK. Using Connection: close with Content-Length header omitted is perfectly valid from HTTP perspective. The end of response is detected by terminating connection on the server side. I am well aware of that. I am also well aware that a client that declares itself HTTP 1.1 capable must support chunked encoding. I am frankly astonished that a client is so broken it can't handle chunked encoding. In fact some browsers have problems detecting connection termination (and host of other issues) related to the chunked encoding. Which browsers? While I understand it is not a Tomcat issue, it will score some points for Tomcat if this is addressed by adding a configuration option. Sure, feel free to request an enhancement to optionally restore the non-chunked approach when keep-alive is disabled. I'm not sure how much interest there will be in implementing it though. Mark Thanks Date: Wed, 5 Jan 2011 06:14:18 + From: ma...@apache.org To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked
Re: How to disable chunked encoding for the Http11NioProtocol connector.
ilya goberman wrote: I was thinking more about it. What if Tomcat disables chunked encoding if response contains Connection: close header. So in order to disable the encoding the Tomcat application will have to set just one response header. I think it is a reasonable enhancement to do. If Connection: close is not set, keep alive functionality is preserved. Any objections? Yes. The sysadmin may decide, for whatever reason, to disable keep-alive connections. In that case Tomcat will add a Connection: close to any response, and close the connection at the end of the response. But that does not mean that the one response sent back, could not be from some application generating a dynamic response for which the size is not known in advance, thus triggering a chunked response. Now in addition I am thinking of this : sputnik /\ / \ /\ client -- something -- -- something -- tomcat The somethings in-between could decide that to go through the sputnik, it makes no sense to have the message in one piece - or in multiple pieces - and re-slice it or re-assemble it in the way best suited to go up and down. Since the transfer-encoding is just that, they are allowed to do that. So basically any scheme that relies on the transmission being chunked or not, is doomed, no ? I love ascii art. :-) - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
Correct me if I am wrong, but it seems to me that both in the case of this post, and another simultaneous one entitled Tomcat and HTTP chunk extensions, the OP's are trying to use HTTP in a way that is not really part of the protocol design. The transfer-encoding is not supposed to be something over which the application has real control. Its purpose is to allow the safe transport of the message from end to end, and is basically the choice of any intermediate agent. RFC 2616 : 3.6 Transfer Codings Transfer-coding values are used to indicate an encoding transformation that has been, can be, or may need to be applied to an entity-body in order to ensure safe transport through the network. and 4.3 Message Body ... Transfer-Encoding is a property of the message, not of the entity, and thus MAY be added or removed by any application along the request/response chain. and 4.4 Message Length ... All HTTP/1.1 applications that receive entities MUST accept the chunked transfer-coding (section 3.6), thus allowing this mechanism to be used for messages when the message length cannot be determined in advance. ... I am not saying that the OP's do not have valid reasons to want what they want, but maybe they should consider writing their own specialised server to do that, rather than asking for modifications to spec-respectful servers such as Apache httpd or tomcat ? After all, both are open-source and can be used as base to do that. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: How to disable chunked encoding for the Http11NioProtocol connector.
This is getting philosophical. spec-respectful does not mean it has to support only one valid protocol out of 2. If both protocol A (chunked-encoding) and B (no chunked encoding) is allowed, why not give an ability to use whatever user prefers. As far as sputnik example is concerned, I have never heard of a proxy server adding chunked encoding to a non-chunked encoding input (most proxy servers are still HTTP 1.0 compliant). Even if it does, it will be very small percent of overall users. Anyway, from the practical perspective, if I can save 10% of bandwidth and work around some bugs, I would take it. Thanks Date: Wed, 5 Jan 2011 23:13:07 +0100 From: a...@ice-sa.com To: users@tomcat.apache.org Subject: Re: How to disable chunked encoding for the Http11NioProtocol connector. Correct me if I am wrong, but it seems to me that both in the case of this post, and another simultaneous one entitled Tomcat and HTTP chunk extensions, the OP's are trying to use HTTP in a way that is not really part of the protocol design. The transfer-encoding is not supposed to be something over which the application has real control. Its purpose is to allow the safe transport of the message from end to end, and is basically the choice of any intermediate agent. RFC 2616 : 3.6 Transfer Codings Transfer-coding values are used to indicate an encoding transformation that has been, can be, or may need to be applied to an entity-body in order to ensure safe transport through the network. and 4.3 Message Body ... Transfer-Encoding is a property of the message, not of the entity, and thus MAY be added or removed by any application along the request/response chain. and 4.4 Message Length ... All HTTP/1.1 applications that receive entities MUST accept the chunked transfer-coding (section 3.6), thus allowing this mechanism to be used for messages when the message length cannot be determined in advance. ... I am not saying that the OP's do not have valid reasons to want what they want, but maybe they should consider writing their own specialised server to do that, rather than asking for modifications to spec-respectful servers such as Apache httpd or tomcat ? After all, both are open-source and can be used as base to do that. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: How to disable chunked encoding for the Http11NioProtocol connector.
On 05/01/2011 05:04, ilya goberman wrote: Hi, I use NIO HTTP Tomcat connector org.apache.coyote.Http11NioProtocol to implement Comet streaming to browsers and mobile devices. I would like to disable HTTP response chunked encoding to reduce bandwidth. How significant is the overhead with chunking in your case? I'd expect it to be pretty small unless only a few bytes are sent at a time (and even then there is the overhead for the packet). Is there any mileage in using compression to reduce bandwidth instead? Issues with flushing compressed output streams [1] were fixed last year. The response will have header Connection: close with Content-Length header omitted. Is there a way to do it besides having client send HTTP 1.0 request (that is not possible in the majority of cases)? Having looked at the relevant source code the only two ways I can see are: - sending an HTTP 1.0 request - declaring a content length It used to be possible to control this by disabling keep-alive but that was changed back in April last year [2],[3] as a result a discussion on the dev list [4]. If your Tomcat version is old enough, you may still be able to use the disable keep-alive trick. My own view was then, and is now, that the extra bytes with chunking are a price worth paying for the client to be able to determine if the request is complete. That said, an option on the connector to revert to non-chunked responses when keep-alive is disabled for use cases where reducing bandwidth is more important than knowing if the response is complete seems reasonable to me. Mark [1] http://issues.apache.org/bugzilla/show_bug.cgi?id=48738 [2] http://svn.apache.org/viewvc?rev=931709view=rev [3] http://svn.apache.org/viewvc?rev=932913view=rev [4] http://markmail.org/message/pim62zhlw4cii7ve - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
