A comparison between de.wikipedia.org and wiki.openstreetmap.org - the option --save-headers prepends the answer's headers to "file":
1) wget -O file --save-headers http://de.wikipedia.org/wiki/Test && more file HTTP/1.0 200 OK Date: Fri, 28 Aug 2009 12:30:51 GMT Server: Apache Cache-Control: private, s-maxage=0, max-age=0, must-revalidate Content-Language: de Vary: Accept-Encoding,Cookie Last-Modified: Sat, 22 Aug 2009 06:27:38 GMT Content-Length: 25984 Content-Type: text/html; charset=utf-8 Age: 19534 Connection: keep-alive <!DOCTYPE html ... 2) wget -O file --save-headers http://wiki.openstreetmap.org/wiki/Map_Features && more file HTTP/1.0 200 OK Date: Fri, 28 Aug 2009 17:45:43 GMT Server: Apache X-Powered-By: PHP/5.2.4-2ubuntu5.6 Content-Language: en ETag: W/"wiki:pcache:idhash:1156-0!1!0!!en!2!edit=0--20090828060905" Vary: Accept-Encoding,Cookie X-Vary-Options: Accept-Encoding;list-contains=gzip,Cookie;string-contains=wikiTo ken;string-contains=wikiLoggedOut;string-contains=wiki_session Cache-Control: s-maxage=18000, must-revalidate, max-age=0 Last-Modified: Fri, 28 Aug 2009 10:49:00 GMT Content-Encoding: gzip Content-Length: 74290 Content-Type: text/html; charset=UTF-8 Age: 875 X-Cache: HIT from ross.wwood.co.uk X-Cache-Lookup: HIT from ross.wwood.co.uk:3128 Via: 1.0 ross.wwood.co.uk:3128 (squid/2.6.STABLE18) Connection: keep-alive ^_<8B>^...@^@^...@^@^...@^@^C<EC><FD>[<8F>$?&<86>=<CF><FE>^U~r<AB><F7><AE>?<91>^^y<BF>t e<8C><B2><AA><AB><BA>jwUw< .... wiki.openstreetmap.org sends gzipped content anyway - and wget does not care about the response header "Content-Encoding: gzip" 3) Even further, my wget does not care about gzip at all: wget -O file --header="Accept-Encoding: gzip" http://de.wikipedia.org/wiki/Test && more file ^_<8B>^...@^@^...@^@^...@^@^C<C5>][o#Gv~<B6>~E^M7<B3>^Z9"<9B><92>Fs<91>D.4<92>?<F5><C8> <D6>Z^ZO<E2>^YCh<B2><8B>d^O<9B><DD>?<88><92>^L^C^N<90>?^P<E4>% ^V<B9>a^P^Dy<C8> <E3>^L^DX<FD>^A<EF><93><DF><F7><97> <E4>;<A7><AA><AB><AB>4D<CD>:<D9>] @<D3>?:Uu <EA><DC><CF><F1>?<FD><CF><F7>N<FF><F2><F8>@^L<D2>Q <8E>_={y<B8>'ju<C7>y<BD><B1> <E7>8<FB><A7><FB><E2>/^<9C>^^<BD>^Tk<8D><A6>8<8D><DD>0<F1>S? <DD><C0>q^N><AB><89><DA> M<C7>[<8E>3<99>L^Z<93><8D>F^T<F7><9D><D3>/<9C>^K^Zk<8D> ><U+058F><F5><D4><FA><B2><E1><A5> .... Anyone who can solve this puzzle, so that we can download with simple commands ?-) Andi Marc Schütz schrieb: > -------- Original-Nachricht -------- > >> Datum: Fri, 28 Aug 2009 11:59:15 +0200 >> Von: Roland Olbricht <roland.olbri...@gmx.de> >> An: dev@openstreetmap.org >> Betreff: Re: [OSM-dev] Problem retrieving wiki pages >> > > >>> Of course, but the server (or more likely the proxy) is still >>> >> mis-behaving: >> >>> if the client does not send an 'Accept-Encoding' header, the server must >>> return plain text, not gzipped or deflated text. >>> >> No. Have a look at the HTTP 1.1 specification >> http://tools.ietf.org/html/rfc2616#section-14.3 >> >> "If no Accept-Encoding field is present in a request, the server MAY >> assume that the client will accept any content coding." >> >> > > If you read on, it also gives a recommendation for this case: > "If no Accept-Encoding field is present in a request, the server MAY > assume that the client will accept any content coding. In this case, > if "identity" is one of the available content-codings, then the > server SHOULD use the "identity" content-coding, unless it has > additional information that a different content-coding is meaningful > to the client." > > So you are right; strictly speaking, the behaviour of the proxy is not in > violation of the RFC. However, it is recommended to use the identity encoding > whenever possible. > > >> If you think of a highly frequented server like the wiki, it's a good >> decision >> to compress the data whenever possible. >> > > I agree, but in this case the client gave no indication that it actually > understands compressed replies. > _______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
