Greetings,
The following webrev contains fixes and changes for the following RFEs/bugs:
11065 client v1 catalog support for v0 catalogs
11075 catalog should provide manifest signature data
11074 manifests need sha-1 signature
11072 client double instantiates image on startup
11071 pkg list -v doesn't show full FMRI for those from a preferred
publisher
10041 catalog attrs can be out of sync with catalog
9248 client traceback if preferred publisher not in catalog cache
8473 cached catalog data can be wrong if mixing client versions
8341 extract catalog cache from image
5562 catalog uses local time instead of UTC
3045 pkg list tracebacks when installed state directory is out of
sync with pkg directory
2763 pkg.catalog should have a function to split entries into
appropriate fields
318 Unify depot and client pkg format
269 Catalog should include package dependencies
webrev:
http://cr.opensolaris.org/~swalker/pkg-cat-p2/
Summary:
This changeset implements almost all of the v1 catalog support work
needed for the client. It does by changing the client to retrieve v0
catalogs, and then transform into v1 catalogs for internal usage.
This *will constitute a flag day when putback* as it also changes the
format of the image such that older pkg clients won't recognise an image
created with (or upgraded by) this new client as an image anymore (since
the catalog directory is being removed).
The changes denoted above are all of those needed to implement the
recently discussed Catalog format proposal [1] (with some minor
deviations for the updatelog and with respect to signatures [2]).
There is a significant amount of change here, but part of that is
because the catalog was so inter-twined among most of the codebase (that
is less true now that the Catalog has most of that code).
Significant performance improvements are present for operations such as
pkg info, list, install, and uninstall [3]. Operations like
image-update take about the same amount of time. However, pkg refresh
can take twice as much time as it did previously (cost of v0 -> v1
catalog transformation plus re-creating state information); on my system
that means 18 seconds becomes 34 seconds roughly.
No serious performance work beyond achieving performance parity (where
possible) has been done yet, so there likely remains a lot of
additional, possible optimisation to be done. Although at .3 seconds
real time for pkg list on my system now, I suspect that optimisation
will mainly be in other areas :)
For reference, it takes about 40-50 seconds for the new pkg client to
upgrade an existing image. This too could be optimised; most of the
cost is in v0 -> v1 catalog transformation (FMRI object construction).
However, once that upgrade has been performed, all future operations
will work rather speedily.
The new pkg client is able to use read-only images of a previous format,
and will still work, but very slowly as noted above. I haven't yet put
in a performance warning for that, as I wanted some feedback about it first.
Please note that this changeset also has a nice side-effect of
significantly reducing client memory usage (85 megabytes to 35 megabytes
for pkg list in one case) for many operations because the client now
only constructs FMRI objects on an as-needed basis instead of
constructing one for every possible package on startup and keeping them
in-memory.
There are likely lots of additional test cases to add, and the only one
possibly missing I can think of is for the image upgrade logic which I'm
currently testing by hand. I have no problems in adding an automated
test for that if so desired.
Reviewers: There is a lot of material here, but I need review comments
at your earliest convenience as this work is a blocker for a lot of
other work. The parent revision for this changeset is currently rev 1291.
Cheers,
--
Shawn Walker
[1] http://cr.opensolaris.org/~swalker/catalog.txt
[2] http://cr.opensolaris.org/~swalker/catalog-v4-v5.txt
[3] http://markmail.org/thread/n6kcsllmegupxmjn
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
