On Wed, Mar 10, 2010 at 5:51 PM, Eric Uhrhane <er...@google.com> wrote: > On Wed, Mar 10, 2010 at 12:28 AM, timeless <timel...@gmail.com> wrote: >> http://www.pkware.com/documents/casestudies/APPNOTE.TXT V. General >> Format of a .ZIP file >> >> the zip format is fairly streaming friendly, the directory is at the >> end of the file. And if you're actually generating a file which has so >> many records that you can't remember all of them, you're probably >> trying to attack my user agent, so I'm quite happy that you'd fail. > > Isn't a format that has its directory at the end about as > streaming-UNfriendly as you can get? You need to pull the whole thing > down before you can take it apart. With a .tar.gz, you can unpack > files as they arrive.
Each file's compressed data is preceded with a header with enough information to decompress it (filename etc), and then that information is duplicated in the central directory at the end, so I believe you can still do streaming decompression (as well as doing random access once you've got the directory). And you can still do streaming compression without even buffering a single file, by setting a flag and moving a part of the file header (lengths and checksum) to just after the compressed file data. (But I never understood why pkunzip asked me to put in the last floppy disk of a multi-disk zip before it would start decompressing the first - maybe there's some reason that streaming decompression doesn't quite work perfectly in practice?) -- Philip Taylor exc...@gmail.com
