[Bug-wget] [bug #44817] http/gzip compression

[Darshit Shah]

Update of bug #44817 (project wget):

  Status:None => Fixed  
 Open/Closed:Open => Closed 
 Release:  1.16.3 => 1.19.1 


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Bug-wget] [bug #44817] http/gzip compression

[Ander Juaristi]

Follow-up Comment #1, bug #44817 (project wget):

For me, the best place to decompress the response body would be at
fd_read_body(), just after the call to fd_read(). Let fd_read() return, and
uncompress the read bytes. Also, we should consider whether the data will be
written to warc compressed or uncompressed. In the latter, we should discuss
how to handle fd_read_line(), since it expects to encounter a line separator.

We could add an extra command line switch to let the user decide the supported
compression algorithms.

Finally, the best way to implement this would be using zlib, since it supports
the two most used compression algorithms on the web: gzip and deflate. Don't
forget to update the dependency list for bootstrap/autoconf.

___

Reply to this item at:

  http://savannah.gnu.org/bugs/?44817

___
  Message sent via/by Savannah
  http://savannah.gnu.org/

[Bug-wget] [bug #44817] http/gzip compression

[anonymous]

URL:
  http://savannah.gnu.org/bugs/?44817

 Summary: http/gzip compression
 Project: GNU Wget
Submitted by: None
Submitted on: Sun 12 Apr 2015 11:33:43 AM UTC
Category: Feature Request
Severity: 3 - Normal
Priority: 5 - Normal
  Status: None
 Privacy: Public
 Assigned to: None
 Originator Name: 
Originator Email: 
 Open/Closed: Open
 Discussion Lock: Any
 Release: 1.16.3
Operating System: None
 Reproducibility: Every Time
   Fixed Release: None
 Planned Release: None
  Regression: None
   Work Required: None
  Patch Included: No

___

Details:

Why does wget still not have any http/gzip compression?

http://en.wikipedia.org/wiki/HTTP_compression

It's an essential feature for a web crawler because it saves both the server
and the downloader huge amounts of bandwidth. Server admins are banning wget
now because it requests everything uncompressed and uses 4 times the bandwidth
as other programs. 




___

Reply to this item at:

  http://savannah.gnu.org/bugs/?44817

___
  Message sent via/by Savannah
  http://savannah.gnu.org/