On 05/19/10 08:14 AM, Darren J Moffat wrote:
First a general comment, I'm really impressed with how "simple" this
actually looks particularly given you have been able to use an existing
archive format.
I assume when you mention ZIP as a format that was considered the JAR
variant of it was too.
Yes, although they're essentially the same format. However, I might
note that the ZIP64 extensions which would be required for consumers to
use JAR (to get large file support) weren't available until OpenJDK7
build(b55). I don't even know if that's part of an official build yet.
Even if that was the case, there currently aren't any existing Python
libraries (that are already integrated) that support the creation of and
access to JAR files (I suppose you could use the zipfile module but that
means we'd have to layer our own JAR variant management on top).
Finally, all of the same problems noted about ZIP files apply to JAR
files as well. In short, they provide no real advantage for our use
case over standard ZIP files.
I really have only one comment and that is about integrity protection of
the on disk format.
Lets assume for a moment that the package contents themselves are
cryptographically signed so they are covered.
I assume you're referring to the (potentially) signed manifests within?
However the archive format version doesn't have any additional integrity
protection. In particular there doesn't appear to be any protection that
the index is pointing to the correct places and there doesn't appear to
be any integrity protection of content like the pub.p5i file that aren't
actually package contents.
Is there really a risk here ?
The ustar headers used by the PAX format have built in checksums for
basic integrity checking of the headers themselves (which should
reasonably protect against basic file corruption, but obviously doesn't
handle malicious manipulation).
Package files are stored using a digest hash and have to match what's in
the package manifest (if present) and manifests will eventually be signed.
The catalog has its own digest as well, but since it is simply a subset
of the manifests contained within, its contents could be verified by
relying on the manifests which will eventually be signed.
So that just leaves the .p5i file and the index. As far as the p5i
file, that particular item doesn't have any specific protections that
the others do but doesn't contain any sensitive information other than
(optionally) URIs for publisher origin(s) and/or mirror(s).
As for the index file, that's much simpler to deal with. Since the
index file contains the name of the archive entry and the size of the
archive entry, if the header block at the specified offset doesn't
match, it's pretty easy to tell something's wrong.
So, generally speaking, I don't believe accidental (as opposed to
malicious) corruption will be a concern.
Compare this to what ZFS does. It uses a Merkle tree of checksums going
all they way back to the uberblock.
If archive integrity verification is needed, (beyond simple accidental
protection), my inclination would be to simply sign the archive using
gpg and provide the resulting '.asc' file alongside the archive. That's
a fairly standard, accepted practice as far as I can tell.
As much as possible, I'm trying to avoid anything that can't be done
using some simple shell scripting so that users could choose to create
package archives using their own tools and could access and verify the
content themselves using their own tools.
Cheers,
-Shawn
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
