When using wget 1.14 to generate warc.gz files, e.g.

wget -O tempname --warc-file="output"  "http://example.com";

the files this creates do not play back well using the Internet Archives 
warc.gz parsers, throwing errors like 

"Invalid FExtra length/records". 

It appears wget may be creating slightly malformed GZIP skip-length 
fields - see 

https://github.com/ukwa/warc-discovery/issues/1 

for details.

It's likely that we'll need to make the warc.gz parsers a bit more 
robust, but I thought I'd mention it here in case this is 
actually a bug in wget.

Thanks for your time.

Andy Jackson


Reply via email to