Dear Wiki user, You have subscribed to a wiki page or wiki category on "Httpd Wiki" for change notification.
The following page has been changed by wrowe: http://wiki.apache.org/httpd/ReInflating The comment on the change is: Color commentary and the play-by-play ------------------------------------------------------------------------------ + So you want to handle the text from a proxy or other app which has deflate'd the content? - #FORMAT plain - I've lost decorative text once already, explanations to follow - LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression + The first most valuable thing for diagnosing any issues is to create a compression log + which will show you the Accept-Encoding (AE) permitted by the client (browser) along + with the Transfer-Encoding (TE) and Content-Encoding (CE) for the request body with the + total input byte count. We'll prefix each entry with the client ip/host and their request + line. And the meat of the response to the client are the response size (bytes transmitted), + response Transfer-Encoding and response Content-Encoding. Throughout these examples we'll + be sharing the results as gathered with the following config snippet; - CustomLog logs/compression.log compression + {{{ + LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression + CustomLog logs/compression.log compression + }}} + The first thing is to provide a server to test against which deflates it's output. We can + do this in the same instance of Apache, simply set up an alias to the htdocs for /deflated/ + requests for the very same documents in /. - Alias /deflated "/path-to/htdocs" - <Location "/deflated"> - SetOutputFilter DEFLATE - </Location> - 127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 412 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /local/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-" + {{{ + Alias /deflated "/path-to/htdocs" + <Location "/deflated"> + SetOutputFilter DEFLATE + </Location> + }}} + Requesting the same LICENSE.txt from both / and /deflated/ reveals the impact deflate can offer; - ProxyPass /proxy/ http://127.0.0.1/ - ProxyPassReverse /proxy/ http://127.0.0.1/ + {{{ + 127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 412 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /local/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-" + }}} + Note the response body has shrunk nearly to a quarter of it's original size. - 127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 501 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-" + For our first experiment (and we will keep it in our config as the LAST snippet, + because this would override any other example below if it appears first), we will + create a reverse-proxy (gateway) to our own server's content, everything requested + from /proxy/ will actually be proxied from / of our server. + {{{ - ProxyPass /proxy/nodeflate/ http://127.0.0.1/deflated/ + ProxyPass /proxy/ http://127.0.0.1/ + ProxyPassReverse /proxy/ http://127.0.0.1/ + }}} - <Location "/proxy/nodeflate/"> - RequestHeader unset Accept-Encoding - </Location> - 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 479 AE:"-" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/nodeflate/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 428 AE:"-" TE:"-" CE:"-" + The compression.log gets interesting here. The first request to be satisfied is our + back-end proxied request from ourself, once it's completed the /proxy/ request can + be completed. In this case, the responses are identical, we aren't manipulating + the Content-Encoding as the request passes through the proxy; + {{{ + 127.0.0.1 "GET /LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 501 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/LICENSE.txt HTTP/1.1" 200 39858 (40146) TE:"-" CE:"-" <-- 418 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-" + }}} + + Now the quick method of disabling compression is to never request a compressed + document from the back-end machine, which we can accomplish by unset'ting the + Accept-Encoding header from the client. + + {{{ + ProxyPass /proxy/nodeflate/ http://127.0.0.1/deflated/ + <Location "/proxy/nodeflate/"> + RequestHeader unset Accept-Encoding + </Location> + }}} + + And lo, neither the backend or proxy request appears to support gzip encoding; + + {{{ + 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 479 AE:"-" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/nodeflate/LICENSE.txt HTTP/1.1" 200 39858 (40169) TE:"-" CE:"-" <-- 428 AE:"-" TE:"-" CE:"-" + }}} + + But if we want to continue to accept deflated content from the back end server + for bandwidth conservation, it's possible to use mod_deflate to perform response + re-inflation. We only want to do this if the back end server provided a response + in Content-Encoding:gzip, or else we will be trying to unpack an apparently corrupt + data stream; + + {{{ FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip ProxyPass /proxy/reinflate/ http://127.0.0.1/deflated/ <Location "/proxy/reinflate/"> FilterChain gzinflate </Location> + }}} - 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/reinflate/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-" + Now when a request is made through /proxy/reinflate/, we will take the deflated + back end content and explode it. Note that the (2nd) response to the client is + nearly four times larger, again, than the backend request; + {{{ + 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 510 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/reinflate/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-" + }}} - FilterProvider ucase CaseFilter Content-Type $text/ + Let's take this one step further, and add on a transformation. There's an experimental + filter mod_case_filter which upper-cases the response, and it's perfect for illustrating + the transformation. We can't upper-case the deflated binary stream, so we'll have to + inflate it, then transform it. - ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/ - <Location "/proxy/ucase/"> - FilterChain +gzinflate +ucase - </Location> - - 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 513 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-" - - ** NOTE One line bug in mod_case_filter.c that must be fixed in 2.2.6, + ** NOTE A one line bug in mod_case_filter.c that must be fixed in 2.2.6! Insert the line; - insert the line; + {{{ apr_brigade_cleanup(pbbIn); + }}} immediately before this line; + {{{ return ap_pass_brigade(f->next, pbbOut); + }}} and rebuild the module. + So using the example above, we'll build on it with another FilterProvider we'll name ucase, + and apply both gzinflate and ucase in that order to all /proxy/ucase/ requests; - FilterProvider gzdeflate DEFLATE Content-Type $text/ + {{{ + FilterProvider ucase CaseFilter Content-Type $text/ - ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/ + ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/ - <Location "/proxy/ucase/"> + <Location "/proxy/ucase/"> - FilterChain +gzinflate +ucase +gzdeflate + FilterChain +gzinflate +ucase - </Location> + </Location> + }}} - 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 514 AE:"gzip,deflate" TE:"-" CE:"-" - 127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 9600 (9934) TE:"-" CE:"gzip" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-" + And the resulting log shows that the response to the client remains undeflated, and + if you inspect it in the browser, it's irritatingly in all caps; + {{{ + 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 513 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 39858 (40219) TE:"chunked" CE:"-" <-- 427 AE:"gzip,deflate" TE:"-" CE:"-" + }}} - LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression - FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip + Building on those two examples, we'll take this one step further, and ensure the + final response can in fact be deflated to the client, so we'll name our redeflate + filter gzdeflate and apply all three filters, in specific order, to inflate, + manipulate, and re-deflate the response; + {{{ + FilterProvider gzdeflate DEFLATE Content-Type $text/ - <VirtualHost 192.168.0.254:80> - ErrorLog logs/proxyerror.log - CustomLog logs/proxyaccess.log common - CustomLog logs/proxycompression.log compression - ProxyRequests On - ProxyVia On + ProxyPass /proxy/ucase/ http://127.0.0.1/deflated/ + <Location "/proxy/ucase/"> + FilterChain +gzinflate +ucase +gzdeflate + </Location> + }}} - <Proxy http:*> - FilterChain gzinflate - Order deny,allow - Deny from all - Allow from 127. 10. 192.168. - </Proxy> - </VirtualHost> + Note a fun side effect on deflation of upper-casing the document; + + {{{ + 127.0.0.1 "GET /deflated/LICENSE.txt HTTP/1.1" 200 10629 (10964) TE:"-" CE:"gzip" <-- 514 AE:"gzip,deflate" TE:"-" CE:"-" + 127.0.0.1 "GET /proxy/ucase/LICENSE.txt HTTP/1.1" 200 9600 (9934) TE:"-" CE:"gzip" <-- 428 AE:"gzip,deflate" TE:"-" CE:"-" + }}} + + With only 26 alpha characters rather than 52 (upper and lower case) it compresses down + more efficiently than the back end had provided it to us. + + We can actually do this for forward proxies as well. Here is an example proxy server + which inflating every C-E:gzip compressed response and serves only plain text to the + clients using our proxy, we save about 10%. + + {{{ + LogFormat "%h \"%r\" %>s %b (%O) TE:\"%{Transfer-Encoding}o\" CE:\"%{Content-Encoding}o\" <-- %I AE:\"%{Accept-Encoding}i\" TE:\"%{Transfer-Encoding}i\" CE:\"%{Content-Encoding}i\"" compression + FilterProvider gzinflate INFLATE resp=Content-Encoding $gzip + + <VirtualHost 192.168.0.254:80> + ErrorLog logs/proxyerror.log + CustomLog logs/proxyaccess.log common + CustomLog logs/proxycompression.log compression + ProxyRequests On + ProxyVia On + + <Proxy http:*> + FilterChain gzinflate + Order deny,allow + Deny from all + Allow from 127. 10. 192.168. + </Proxy> + </VirtualHost> + }}} +
