Working with the LZMA compression code from 7zip a lot right now, I can
attest that the decompression code is pretty easy to mold C code. The
compression code is a much larger chunk of C++ code, but I might have a C
based port soon too.
Thank you,
Clay
On Fri, 1 Feb 2008, Danek Duvall wrote:
> On Fri, Feb 01, 2008 at 10:08:05PM -0600, Mike Gerdts wrote:
>
> > I would like to see something different than tar, because...
> >
> > - The logical thing to do with a tar file is to compress it. If you
> > are working on a system with slow single thread performance,
> > processing compressed tar (or cpio) archives is extremely slow.
>
> We could, of course, compress each file in the archive individually, which
> would allow for multiple threads to do the decompression in parallel. Of
> course, you still have to read through the archivve to find out where
> things are, unless you tack on a header that contains the table of
> contents.
>
> > >From my recollection of the zap discussion and my own frustration with
> > single-threaded performance on US-T1 systems, I would like to see:
> >
> > - All metadata at the beginning of the file. This is helpful when the
> > file is delivered via a standard HTTP download (no IPS server) to
> > determine which byte ranges need to be downloaded.
>
> This is one of my big beefs with zip-based formats. There's no reason that
> a smart protocol couldn't download the TOC first, place it at the end of a
> holey file, and fill in the rest afterwards. Seems a bit icky, but at
> least opensource zip tools are very widely available (including the python
> module), so we wouldn't have to write anything new.
>
> > - No barriers in the file format to having a multi-threaded pkg
> > command that can simultaneously uncompress, checksum, verify
> > signatures, etc. multiple files from the same package at the same
> > time.
>
> Zip archives compress the files individually, as I understand it, so it's
> at least capable of better multithreaded operations, but I don't know if
> there are any zip utilities or libraries that actually do this. (Or 7z,
> for that matter, since you brought it up.)
>
> RAR would be another possibility, but it's not free. 7z seems like it
> would be ideal (individually compressed files if desired, LGPL), but all
> the implementations are in C++, and nasty C++ at that. But we could
> probably work with it if we had to.
>
> Danek
> _______________________________________________
> pkg-discuss mailing list
> [email protected]
> http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
>
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
