At
http://cr.opensolaris.org/~dduvall/pkg-newlist/
you will find the webrev for a release candidate to fix
1867 'pkg list' takes *WAY* too long
There are two separate changes. The first is a wad I took over from
Stephen to move package state files into a single directory, making
operations like getting the list of all installed packages much quicker.
The second is the main list wad, which introduces a data structure to make
lookups fast. I've gotten rid of a fair amount of code, and reduced two
codepaths to one (in almost all cases). I think that the new execution
flow is easier to understand, even if __inventory() is kind of hairy (I
hope I commented it well enough).
There are also a handful of new XXX comments which I'd like some input on.
As for what you get out of this ... my machine has two authorities, roughly
duplicates of each other, but in total, there are 17 builds between the
two, encompassing 17000 package versions. Prior to my changes, "pkg list"
and "pkg list -a" take 3:15 and 3:23, respectively (terminal output
redirected to /dev/null), while after my changes, those times change to
1.8s and 3.9s. The constant overhead (loading the cached catalog data
structure off disk) is about .67s.
For fun, I loaded up an image with ten copies of cyber's catalog, for a
total of 91k versions. list -a took 13s.
It's a tad slower than I'd like, but I'm not sure where else to shave.
It's also running on a fast machine, but I expect that everyone will see
similar factors of speed increase.
Upgrade should be handled smoothly and quietly, though until you run one of
the pkg subcommands with sufficient privilege to write the new data
structure files, it'll be a bit slower (but not by a *whole* lot).
Executing "pkg list -a", the original code maintains 14-17MB resident for
most of the duration, but spikes up to about 30MB right before it exits.
The new code seems to hang out at 32MB for the duration. This should serve
us for a while, at least, if for no other reason than the number was
exactly the same for the 91k version test, too.
Thanks,
Danek
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
