On 01/12/10 02:10 AM, Jan Damborsky wrote:
Hi Shawn,
On 01/11/10 05:39 PM, Shawn Walker wrote:
On 01/11/10 09:41 AM, Frédéric VANNIERE wrote:
Another thing that might be contributing to this is how AI is
performing the installs.
Can you provide the full install log?
It's here : http://pastebin.com/m3c26ca80
Note: The PWKinstall-opensolaris package is downloaded from a custom
IPS repository and is post-install my customization script (I'll post
about this latter) .
That helps quite a bit; AI appears to be executing the pkg command to
install each package separately and for other tasks (over 90 times).
Yep - that is how AI currently installs packages listed in AI manifest.
That's going to be much slower than simply executing the pkg(1)
command once for all packages to be installed. Not only that, AI
should really be using the python pkg.client.api that's provided
instead as that would further reduce the overhead of the operations it
is performing.
That is interesting - thank you for pointing this out.
Do you happen to have more data about what the difference is
with respect to the overhead of particular approaches you mentioned:
* calling pkg(1) for each package
If you call pkg(1) for each package, you are significantly increasing
install times due to the following:
* search indices update cannot be coalesced for the set of packages
being installed
* dependency solver overhead means you pay a higher price for individual
install instead of a single install
* transport system depends upon observed performance metrics per session
to determine optimum path and relies on persistent connections to
optimise file transfer
* python interpreter startup cost
* calling pkg(1) for all packages at once
This is the best option if you cannot use the API.
* consuming directly pkg.client.api
Specific benefits of using the API include:
* access to additional information that is not easily
available/parseable via the CLI
* ability to control various transport retry/timeout errors
* ability to perform granular error-case handling (far more expressive
than the limited exit code failures provided by cli)
* ability to retrieve package architecture and zone variant information
* in the future, ability to retrieve package size information before
plan preparation or execution
We are mostly interested in memory consumption as well as well
time execution.
As far as stability of those APIs is concerned, are there any
significant differences between pkg(1) and pkg.client.api ?
I mean if we switch to pkg.client.api, should we assume
to be hit by API changes more often than we currently are
with pkg(1) ?
More often, yes, but the changes should all be documented in
doc/client_api_versions.txt in the pkg gate. And even though changes
might affect you more often, it is usually a span of at least weeks and
sometimes months before it changes incompatibly.
The pkg.client.api interface is the intended interface for external
consumers.
The interface is also versioned, so if it changes incompatibly, your
program should fail as soon as it tries to load the API; not
mysteriously somewhere in the middle.
Also, pkg(1) man page is useful source of information and often
used as API documentation, is pkg.client.api API documented
somewhere as well ?
See 'pydoc pkg.client.api' and 'pydoc pkg.api_common'. Note that you
need to use the python2.6 version of pydoc as of b129.
For example:
/usr/lib/python2.6/pydoc.py pkg.client.api
--
Shawn Walker
_______________________________________________
pkg-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
