On Thu, Aug 18, 2011 at 7:43 AM, Paul Marquess <paul.marqu...@btinternet.com> wrote: >> >> >> -----Original Message----- >> From: Peng Yu [mailto:pengyu...@gmail.com] >> Sent: 17 August 2011 23:32 >> To: Andy Lester >> Cc: libwww@perl.org >> Subject: Re: Subclass of both WWW::Mechanize::GZip and > WWW::Mechanize::Sleepy >> >> On Wed, Aug 17, 2011 at 5:19 PM, Andy Lester <a...@petdance.com> wrote: >> > >> > On Aug 17, 2011, at 5:05 PM, Peng Yu wrote: >> > >> > Hi, >> > >> > I'd like to use the features of both WWW::Mechanize::GZip and >> > WWW::Mechanize::Sleepy. Does anybody let me know what I should use? Is >> > there a subclass of both? >> > >> > If ::GZip just ungzips content as it comes in, that's now a built-in >> > feature of Mech. And I know that the fucntionality of ::Sleepy is >> > pretty trivial, and could be incorporated into your own subclass of >> > Mech with minimal effort. >> > xoa >> >> According to man WWW::Mechanize, there is an internal method. >> >> $mech->_modify_request( $req ) >> Modifies a HTTP::Request before the request is sent out, for both > GET and POST requests. >> >> We add a "Referer" header, as well as header to note that we can > accept gzip encoded content, if Compress::Zlib is installed. >> >> >> I have Compress::Zlib installed. But it seems that I have to explicit > called it to modify the request. But I don't want to do so as it is an > internal method. >> >> Currently, I do the following. But this wastes some bandwidth. Would you > please let me know how to use WWW::Mechanize with gzip? >> >> $browser->add_header('Accept-Encoding' => 'identity'); my $response = > $browser->get($uri); > > Looking at the WWW::Mechanize code, it looks like it already handles gzip. > > If you want an origin server to return gzipped content you can't get around > having to add this header to the HTTP request > > Accept-Encoding: gzip > > This is what WWW::Mechanize will do if it detects Compress::Zlib is > available. > > Note - by using the "identity" content-encoding you are requesting that the > origin server does not return gzipped content. I assume that isn't what you > want to happen. > > If WWW::Mechanize handles gzip content for you, does that mean that > WWW::Mechanize::Sleepy meets your requirements
I don't necessarily need the server to return gzipped content. But if the server does (and the server that I try returns zipped content by default, unless I tell it not to do so), I need to some how get the gunzipped content. As I far as I can see, WWW::Mechanize::Sleepy doesn't automoatically gunzip the content, because when I print the $response->content, I get some unreadable characters, which is apparently gzipped. Besides the way that I use (i.e., telling the server not to send me gzipped content), would you please let me know what the code that you that you can automatically get the content gunzipped? -- Regards, Peng
