On 19/05/2010 17:36, Shawn Walker wrote:
On 05/19/10 08:14 AM, Darren J Moffat wrote:
First a general comment, I'm really impressed with how "simple" this
actually looks particularly given you have been able to use an existing
archive format.
I assume when you mention ZIP as a format that was considered the JAR
variant of it was too.
Yes, although they're essentially the same format. However, I might note
that the ZIP64 extensions which would be required for consumers to use
JAR (to get large file support) weren't available until OpenJDK7
build(b55). I don't even know if that's part of an official build yet.
Even if that was the case, there currently aren't any existing Python
libraries (that are already integrated) that support the creation of and
access to JAR files (I suppose you could use the zipfile module but that
means we'd have to layer our own JAR variant management on top).
Finally, all of the same problems noted about ZIP files apply to JAR
files as well. In short, they provide no real advantage for our use case
over standard ZIP files.
Thanks, that is exactly what I expected you would say!
I really have only one comment and that is about integrity protection of
the on disk format.
Lets assume for a moment that the package contents themselves are
cryptographically signed so they are covered.
I assume you're referring to the (potentially) signed manifests within?
Yes.
However the archive format version doesn't have any additional integrity
protection. In particular there doesn't appear to be any protection that
the index is pointing to the correct places and there doesn't appear to
be any integrity protection of content like the pub.p5i file that aren't
actually package contents.
Is there really a risk here ?
The ustar headers used by the PAX format have built in checksums for
basic integrity checking of the headers themselves (which should
reasonably protect against basic file corruption, but obviously doesn't
handle malicious manipulation).
Package files are stored using a digest hash and have to match what's in
the package manifest (if present) and manifests will eventually be signed.
The catalog has its own digest as well, but since it is simply a subset
of the manifests contained within, its contents could be verified by
relying on the manifests which will eventually be signed.
So far that looks sufficient to me.
So that just leaves the .p5i file and the index. As far as the p5i file,
that particular item doesn't have any specific protections that the
others do but doesn't contain any sensitive information other than
(optionally) URIs for publisher origin(s) and/or mirror(s).
As for the index file, that's much simpler to deal with. Since the index
file contains the name of the archive entry and the size of the archive
entry, if the header block at the specified offset doesn't match, it's
pretty easy to tell something's wrong.
Great.
So, generally speaking, I don't believe accidental (as opposed to
malicious) corruption will be a concern.
That is good.
Compare this to what ZFS does. It uses a Merkle tree of checksums going
all they way back to the uberblock.
If archive integrity verification is needed, (beyond simple accidental
protection), my inclination would be to simply sign the archive using
gpg and provide the resulting '.asc' file alongside the archive. That's
a fairly standard, accepted practice as far as I can tell.
That sounds sufficient to me, please include that in the document as a
suggested method.
As much as possible, I'm trying to avoid anything that can't be done
using some simple shell scripting so that users could choose to create
package archives using their own tools and could access and verify the
content themselves using their own tools.
That would be good to add to the document as a goal - I think that is a
great goal.
--
Darren J Moffat
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
