Hi Ethan,
Hey, I sent an initially late reply in so thanks for considering
my ideas at all. Responses in-line.
Thank you,
Clay
On Fri, 5 Feb 2010, Ethan Quach wrote:
> Clay,
>
> Thanks for the comments. (really late reply here, but comments below...)
>
> On 01/14/10 17:39, Clay Baenziger wrote:
>> Hello Ethan,
>> Thank you for getting this out. I think there's some great ideas here. I
>> am concerned that we're trying to move away from industry standards or
>> XML standards in a few places, however. I really like some of the UI
>> suggestions for an editor. More comments in-line.
>>
>> Thank you,
>> Clay
>>
>> On Wed, 23 Dec 2009, Ethan Quach wrote:
>>
>>> caimanians,
>>>
>>> Below is a strawman for how derived manifests could work. Would
>>> appreciate comments/thoughts by 1/13.
>>>
>>>
>>> thanks,
>>> -ethan
>>>
>>>
>>>
>>> From Ethan.Quach at Sun.COM Wed Dec 23 23:21:55 2009
>>> Date: Wed, 23 Dec 2009 22:20:48 -0800
>>> From: Ethan Quach <Ethan.Quach at Sun.COM>
>>> To: caiman-discuss <caiman-discuss at opensolaris.org>
>>> Subject: [caiman-discuss] strawman for derived manifests / manifest
>>> update
>>>
>>> caimanians,
>>>
>>> Below is a strawman for how derived manifests could work. Would
>>> appreciate comments/thoughts by 1/13.
>>>
>>>
>>> thanks,
>>> -ethan
>>>
>>>
>>>
>>> [ Part 2: "Attached Text" ]
>>>
>>>
>>> 1. Introduction
>>>
>>> An AI manifest provides the set of input that describes to an automated
>>> installer how to install onto a client system -- selection of target
>>> disk(s)
>>> to install onto (can be a local disk or remote, i.e. iSCSI),
>>> configuration for
>>> disk partitioning, pool layout, etc. It also provides input for what to
>>> install -- a description of the software in the form of a package
>>> list, and a
>>> location of where to get the packages.
>>>
>>> For the relevant case here of using AI over the network with an AI
>>> install
>>> service (this proposal is not considering the use of AI directly
>>> booted from
>>> media for the moment), a client system gets handed an AI manifest from an
>>> install service. The administrator maintains manifest files on the
>>> server,
>>> and assigns a criteria set to each manifest, essentially mapping the
>>> manifests
>>> to the clients that match the criteria set. An example of a criteria set
>>> would be "ARCH=i386,MEM=2048", or something that maps to a single
>>> machine,
>>> "MAC=0:1c:25:2e:64:be". In general, criteria are attributes describing
>>> the
>>> client.
>>>
>>> This use of criteria and static manifest files on the server is an
>>> acceptable
>>> solution for cases where the number of manifests is small, and
>>> administratively
>>> manageable on the server regardless of how many client systems are
>>> actaully
>>> being deployed. The use of criteria in this case can be used to map
>>> multiple
>>> clients to common manifests.
>>>
>>> However, for the case where the number of unique manifests approaches
>>> a large
>>> scale number of clients, maintaining the static instances of those
>>> manifest
>>> files on the server can be unmanageable. The mapping of a client to a
>>> manifest may be based on criteria (client system attribute) that's
>>> just not
>>> supported or on an extremely complex combination of criteria, and
>>> secondly the
>>> pure number of manifest instance files that would have to be
>>> maintained on the
>>> server in this case is simply too large for an administrator to handle. A
>>> better solution is needed.
>>>
>>>
>>> 1.1 Reference docs
>>>
>>> [1] http://hub.opensolaris.org/bin/view/Project+caiman/DerivedManifests
>>> [2]
>>> http://hub.opensolaris.org/bin/download/Project+caiman/auto_install/manifestsimplificationproposalv3.txt
>>>
>>>
>>>
>>> 2. Proposed Solution
>>>
>>> The proposed solution is to enable the user to dynamically modify the
>>> contents of an AI manifest at client run time, and thereby able to base
>>> the modifications on the client attributes, before the client engine uses
>>> the manifest to start an installation. To do this we will introduce a
>>> manifest input module that can dynamically update the contents of the
>>> AI manifest that the client engine is using.
>>>
>>>
>>> 2.1 Client-side spec
>>>
>>> Currently, the AI client engine has support for taking in only one
>>> type of
>>> input -- a regular manifest file which is parsed as an xml document. The
>>> proposal here is for the AI client engine to support a different type of
>>> input - a dynamic manifest script. The AI Service choosing engine (run by
>>> the manifest-locator service) is what negotiates a manifest from an AI
>>> install
>>> service today (which only yields a manifest file). The data it
>>> receives will
>>> be augmented to include:
>>>
>>> 1. the manifest
>>> - which now can be a regular manifest file
>>> instance, or a dynamic manifest script
>>> 2. a metadata file - containing data describing what's been shipped
>>> over via
>>> this service. This includes the distinction of what
>>> type of manifest #1 is -- manifest instance, or script.
>>> - (*** metadata from the service -> client could have other usages
>>> - if and where to push its remote logging / progress reporting.
>>> - validation (hash values) of everything the client's downloaded
>>> from this service?
>>> - in anticipation of the system configuration work using SMF
>>> profiles as the specification, a service will have to handle
>>> this file as a separate entity from the AI manifest.
>>>
>>
>> I think through out the proposal there's a hard-line distinction between
>> "static" and "dynamic" manifests. I think this is a dichotomy which
>> doesn't need to be made.
>>
>> I think to prevent jumpstart style failures which aren't seen until a
>> client is installing we should limit manifest manipulations to a set of
>> queried system data. (We could alway publish an API for folks to add
>> more data if necessary -- and caution we can't support external code.)
>> However, this would limit the challenges of arbitrary scripting.
>
> Yes, providing a known set of system data that'd typically be used in
> this context will be extremely useful; a certain set will be provided.
> The design will propose and spell out exactly what. A starting place
> there would be to provide at least the data that jumpstart provided in
> this environment context. (yes, jumpstart did provide such a set
> for derived manifests.)
>
> Can you expand on what you mean by publishing an API to add to this
> set of data without supporting external code? Or how you're seeing
> that to work...
My thoughts on this are that we need to understand how folks can tie in a
configuration management database (if that is indeed a requirement folks
have) and any other material they'll expect to have for deriving
manifests. I suspect ODBC or some other simple API is enough for such a
requirement and would allow customers to pump in arbitrary data without us
having to build a framework to support arbitrary scripts or intentions;
XSLT would allow them to arbitrarily load in their data and manipulate it.
>>
>>> When the AI client engine is started with a dynamic manifest script
>>> instead of a regular manifest file, the client engine will instantiate
>>> a manifest input module which will be very similar to, or based off
>>> of, what
>>> the ManifestServ object provides for us today. It will be instantiated
>>> with
>>> the AI schema and the default AI manifest included with the AI image. The
>>> manifest input module will then execute the dynamic manifest script which
>>> contains specific commands to manipulate the contents of what the
>>> ManifestServ
>>> object has already loaded in as its manifest. Because a default
>>> (vanilla/generic) manifest is loaded to start with, the dynamic manifest
>>> script only needs to add/modify things in the manifest it needs to
>>> change,
>>> rather than having to set every required piece of manifest input.
>>
>> I have strong reservations about continuing to build off ManifestServ
>> opposed to using third-party code we don't have to invest maintenance
>> time in.
>
> The reference to manifestserv here to try to give a better description
> of the functionality that is desired. We don't have to use the existing
> manifestserv, or even base the described manifest input module off
> that. I think that'll be an implementation decision when we get there.
Thank you for clarifying no hard dependencies are here. I think as we look
into XSLT more too, we'll have other ideas on how to work with an XML
document and modify them too.
>> Further, I'd much rather that we can output the final manifest
>> so the sys. admin. can debug why something didn't work as expected (and
>> for developers to do the same in case the engine takes an improper
>> action too.)
>
> Agreed. It's not stated here, but that is the intent, i.e. the resultant
> manifest that ends up getting used can get dumped into file. This
> functionality
> would be used for successful install cases, as well as any failure cases for
> further debug purposes, etc. We should explicitly state those uses; I'll
> include
> this in the design.
Thank you for adding these cases to the design. I think calling them out
will be useful to direct our implementation.
>>
>>> To give the script access to manipulate the contents of the ManifestServ
>>> object, we introduce a new command that communicates directly with the
>>> ManifestServ object. This new command will only be available to the AI
>>> microroot. It will be executed in the context of the manifest input
>>> module,
>>> and will be explicitly tied to the specific process space. (This could be
>>> acheived by setting variables in the environment where the dynamic
>>> manifest
>>> script gets run.) We want to do this so that we can reserve the
>>> possibility
>>> to be able to run multiple instances of an AI client engine on the
>>> same system.
>>> (The reason for this would be for the potential of using the AI client
>>> engine
>>> to install zones on a server.)
>>
>> Again, instead of opening this up to arbitrary scripting, I'd like to
>> find a way to constrain the actions to well-defined actions on data
>> provided (but the ability to extend the gathered data easily). (This
>> reduces the problem to a well known space of combining two XML documents
>> which XSLT provides us without any re-invention on our part, other than
>> customer facing tooling.)
>
> I can see how that can provide for the input mechanism that describes the
> manifest delta that is desired (i.e. would replace the use of a shell script
> calling a command to inject the deltas), but I'm not following how that would
> provide the method to calculate the manifest delta itself. I see your other
> mail regarding the use of XSLT though, and will take a closer look at that.
Yes, here one would use XSLT anywhere they need a derived value and the
XML processor would take advantage and process that in-line.
>>
>>> The new command, /usr/sbin/aimanifest, will have the following usage:
>>>
>>> /usr/sbin/aimanifest
>>> load <manifest file> - This can optionally be used to load in some
>>> other manifest file to start with, and overwrite
>>> the original default AI manifest loaded.
>>>
>>> add [-i <nodepath=value>,...] <nodepath=value>
>>> - Add a new nodepath. For the case where multiple
>>> instances of the heirarchical parent nodepath
>>> are valid, nodepath must be uniquely identified
>>> using the -i option.
>>>
>>> modify [-i <nodepath=value>,...] <nodepath=value>
>>> - Modify an existing nodepath value. For cases
>>> where multiple instances of nodepath are valid,
>>> The existing nodepath must be uniquely identified
>>> using the -i option.
>>>
>>> verify
>>> - Verify the current manifest input.
>>>
>>> A dynamic manifest script will contain calls to the 'aimanifest'
>>> command to
>>> set nodepaths to whatever values the user wants to set them to.
>>> Because it is
>>> a script, custom logic can be included in the script to dynamically
>>> derive the
>>> values the user may want to use for this particular client.
>>>
>>> (*** Could referencing nodepaths get unusable? Another consideration
>>> is to define a mapping or aliases exposed through the aimanifest command,
>>> instead of referencing the nodepaths.)
>>
>> When you mention nodepath you mean an Xpath expression or something
>> else? (This is one of my concerns of not using what has already been
>> standardized for us by the XML community.)
>
> Yes, an xpath expression.
Cool, then I think your note regarding a question of node-paths should be
handled pretty gracefully as XPath can be written pretty readably in the
cases I've used it.
>
>>
>>> Upon the dynamic manifest script exiting successfully, the ManifestServ
>>> object validates its current content (which will be what is used as the
>>> manifest) and the AI client engine proceeds from there.
>>
>> What do you envision happening in the case of a script failing?
>
> The manifest input module will report failure and the installation
> process stops. A pointer to the script and a log of the script
> execution will be pointed to.
Okay, so any server side validation will pay dividends for customers not
requiring them to worry about bailing while installing.
>>
>>> Note: because of the reliance on the new metadata file gotten from the AI
>>> install service, this design only accomodates the use of derived
>>> manifests
>>> for AI over the network, with an AI install service. Specifying a dynamic
>>> manifest script from the prompt when booting the AI media directly is not
>>> supported in this design. (It would be good to get feedback as to whether
>>> or not we want to support this.)
>>
>> I'd like to see us keep XML for our manifests, so that this bifurcation
>> doesn't happen.
>
> This proposal doesn't move us away from our manifests being XML. Or are
> you stating something else?
My comment was if we remain 100% XML then the bootable AI should be fine
to accept the manifest -- as the need for a metadata file will be
obviated.
>>
>>> 2.2 Server-side spec
>>>
>>> Adding a dynamic manifest script to an install service mustn't be complex
>>> (or any more complex than adding a regular manifest.) Conceptually, a
>>> dynamic manifest script should be viewed as an optional replacement for
>>> a reqular manifest file. As such, association of a dynamic manifest
>>> script
>>> to an install service is equivalent, and follows the same relationship
>>> rules
>>> as that of an regular manifest to an install service. The same
>>> server-side
>>> criteria mechanism will be used as the selection mechanism by the install
>>> service to hand dynamic manifest scripts to a client (to keep it simple).
>>> It should also be possible to denote that a dynamic manifest script is
>>> the
>>> "default" manifest being served by an install service.
>>
>> I like this!
>>
>>> A new -s option to the 'installadm add' command (also changing 'add' to
>>> 'add-manifest') will allow a user to specify adding a dynamic manifest
>>> script
>>> to an install service:
>>>
>>> installadm add-manifest -s <manifest> -n <svcname>
>>>
>>>
>>> With -s the dynamic manifest script is not validated against the AI
>>> manifest
>>> schema. A copy of the file will be stored internally, which is how a
>>> regular
>>> manifest is also currently handled. (*** Since there's no validation,
>>> do we
>>> really need to copy the file internally for dynamic manifest scripts?)
>>
>> I don't see why we have to expose the difference to the user via an
>> option. The structural difference should make this obvious.
>
> yeah, others have made this comment on this as well.
>
>>
>>> Denotion of which manifest is the "default" manifest for an install
>>> service
>>> will become a property of the service. A manifest can be set to be the
>>> "default" for the service at add time using a new -d option, or
>>> subsequently
>>> by setting a property on the service:
>>>
>>> installadm add-manifest -d -m <manifest> -n <svcname>
>>>
>>> installadm modify-service -p default-manifest=<manifest> -n <svcname>
>>>
>>>
>>> For dynamic manifest scripts, the name of the file itself will be used as
>>> its manifest name tracked by the install service. 'list' output will
>>> display
>>> its file name as the manifest name when requested to display manifests
>>> associated with an install service.
>>
>> How does this differ from manifests currently?
>
> It differs in that currently, the name specified inside the manifest content
> is used. (Internally, we make a copy of the manifest and name that copy
> of the file by the name mentioned above, but that is internal to the install
> service, and not known to the user.) So the pov of the user is that the
> internal name specified in the manifest content is what's listed.
I think if we keep with the current XML format we should keep the same
behavior (or at least change it for all manifests).
>>
>>> What about criteria? (Most of this is repeated from the previous proposal
>>> in [1]; the aspect of pulling out the criteria from the file is important
>>> to derived manifest design since there's no criteria spec in a dynamic
>>> manifest script. So including that aspect of that design again here.)
>>>
>>> Criteria specification for a manifest will be pulled out from the
>>> file, and
>>> into command line options. Namely, a new -c option is added to the
>>> 'installadm add-manifest' command to specify the criteria associated with
>>> an added manifest.
>>>
>>> installadm add-manifest -m <manfiest> -n <svcname> \
>>> -c <crit=value|range>,<crit=value|range>,...
>>>
>>> installadm add-manifest -s <manifest> -n <svcname>
>>> -c <crit=value|range>,<crit=value|range>,...
>>
>> I find that the criteria being on the command line could be quite
>> unwieldy and hard to ensure accuracy versus being in a more readable
>> format. E.g.
>> installadm add-manifest -m foo_bar_manifest.xml -n foo_bar_service -c
>> mem=1024-2048,mac=c0ffeec0ffee,ipv4=172.20.24.0-172.20.48.255,arch=sun4u,platform="SUNW,Sun-Fire-880"
>>
>
> The example you've given is quite a beast. In practice, from data seen
> from rules usage in jumpstart, there usually aren't more than a couple
> of criteria for a given manifest. Usually, its just one.
Do we anticipate broader customer usage? Certainly, I did pick an example
to show the point but if I were describing my old desktop at the
University I would have to be very specific. I see XML (despite complaints
of unreadability sill grouping info. better than on a command line):
<ai_criteria name="mem">
<range>1024 2048</range>
</ai_criteria>
<!-- This is our generic coffee maker MAC -->
<ai_criteria name="mac">
<value>c0:ff:ee:c0:ff:ee</value>
</ai_criteria>
<!-- Only pick machines on Brad's network -->
<ai_criteria name="ipv4">
<range>172.20.24.0 172.20.48.255</range>
</ai_criteria>
<!-- Only SPARC coffee makers -->
<ai_criteria name="arch">
<value>sun4u</value>
</ai_criteria>
<!-- Only those with lots of expansion -->
<ai_criteria name="platform">
<value>SUNW,Sun-Fire-880</value>
</ai_criteria>
>> Let alone trying to enter more than one criteria set per manifest in one
>> operation which would be a very nice addition for users; I'd believe.
>
> I agree with this point; we do need to provide a means to provide
> more than one criteria set per manifest. And or'ed set of and'ed
> criteria, is what you mean I presume. Being able to specify multiple
> -c options could provide for that.
I'd be really annoyed typing out two criteria sets on the command line.
This is especially true when even one MAC based criteria is upwards of 25
characters (-c mac=c0:ff:ee:c0:ff:ee); I'd much rather be able to provide
a file (or standard in so anyone who wants to type can; or if someone
wants to write a script).
>>
>>> The use of criteria by the install service to select which manifest
>>> should be given to a client is unchanged. The difference now is that
>>> the manifest may be a regular manifest, or a dynamic manifest script.
>>>
>>>
>>> 3. Manifest tags
>>>
>>> With this proposal, the grammar of the AI manifest is also being visited
>>> to shorten element/tag names where possible. Because we're proposing the
>>> usage of dynamic manifest scripts to reference nodepaths from the AI
>>> manifest,
>>> having shorter, simpler names makes this easier to use. In addition to
>>> this
>>> reason, there are currently elements who's names unneccessarily
>>> include its
>>> heirarchical parent element name again. For example,
>>> <ai_target_device>
>>> <target_device_name>
>>> <target_device_type>
>>> <target_device_vendor>
>>> <target_device_size>
>>> ...
>>> ..
>>>
>>> could be simplified to
>>>
>>> <target_device>
>>> <name>
>>> <type>
>>> <vendor>
>>> <size>
>>
>> Yea! This is certainly needed!
>>
>>> 4. Creating/modifying manifests on the server.
>>>
>>> This part is really a separate proposal from the above, but I would
>>> like to
>>> get people's input on a more user-friendly way create and modify
>>> manifests
>>> on the AI server. Currently, to create an AI manifest, one usually takes
>>> a copy of the "generic/vanilla" AI manifest, use a text-editor (e.g.
>>> vi) to
>>> make edits to the XML (which by some opinions, is not friendly) and adds
>>> the manifest to an install service. Once added, the user's original
>>> copy of
>>> the file is left alone, and it is really unclear what to be done with
>>> it as
>>> the install service does not care for it or track it.
>>
>> Perhaps a terminology change could help (i.e. import-manifest)?
>>
>>> Updating/modifying a published manifest is not any easier. In fact, there
>>> is no official support to update a published manifest with the
>>> 'installadm'
>>> provided tool. A user pretty much has to start from scratch. If you do
>>> have
>>> the original copy of what you added, you're in a little bit better
>>> shape as
>>> you can make additional edits to that, 'installadm remove' the one in the
>>> service, and then 'installadm add' your newly editted copy again.
>>> Otherwise,
>>> you really have to start from scratch. In addition, once criteria is
>>> moved
>>> out of the file, re-publishing a manifest will lose that data.
>>>
>>> A solution for this could be to just provide the 'export-manifest'
>>> subcommand
>>> to get back out the published AI manifest in a file. And then to
>>> provide an
>>> 'update-manifest' subcommand to update the contents of the manifest
>>> without
>>> disrupting any already set criteria.
>>>
>>> Alternatively, we could propose here an interactive CLI to create/modify
>>> manifests in an install service. The CLI could look and behave much
>>> like the
>>> zonecfg(1M) command, which is the tool to administer zone configurations.
>>>
>>> The command to add a new manifest to a service can be
>>>
>>> # installadm add-manifest -m <manifest> -n <svcname>
>>
>> I think overloading the term add here is not as good as calling this
>> invocation create-manifest. Whereas the non-CLI version, which adds a
>> manifests should remain add?
>
> That sounds reasonable.
>
>>
>>> which will now launch an interactive CLI. The AI manifest schema and
>>> the default AI manifest included in the AI image for the service will
>>> be used
>>> to initialize the manifest content. The user will only have to set the
>>> values they want to change.
>>>
>>> For example, if a user wants to create, and add to svc1, a custom
>>> manifest
>>> based on the default manifest, but just modify the pkg repository, the
>>> command could look like:
>>>
>>> # installadm add-manifest -m custom_man1 -n svc1
>>> aimanifest:custom_man1> select pkgdefpub
>>> aimanifest:custom_man1:pkgdefpub> set url=http://pkg.opensolaris.org/dev
>>> aimanifest:custom_man1:pkgdefpub> commit
>>> aimanifest:custom_man1> commit
>>> aimanifest:custom_man1> exit
>>> #
>>
>> This is a cool way to interact with XML! So we'd be able to select on
>> tags and attributes to drill down to the data we want to change?
>
> That's the proposed intent.
>
>
> Thanks for comments,
> -ethan
>
>
>>
>>> Editing an already published manifest can also be interactive. The
>>> command to do this can be
>>>
>>> # installadm edit-manifest -m <manifest> -n <svcname>
>>>
>>> which launches the same interactive CLI, loads the AI schema and
>>> specified
>>> manifest content. An example of modifying the above published manifest to
>>> add another package to the pkg list could look like:
>>>
>>> # installadm edit-manifest -m custom_man1 -n svc1
>>> aimanifest:custom_man1> select install_packages
>>> aimanifest:custom_man1:install_packages> add name=SUNWblah
>>> aimanifest:custom_man1:install_packages> info
>>> install_packages:
>>> name: entire
>>> name: SUNWcsd
>>> name: SUNWcs
>>> name: babel_install
>>> name: SUNWblah
>>> aimanifest:custom_man1:install_packages>commit
>>> aimanifest:custom_man1>commit
>>> aimanifest:custom_man1>exit
>>> #
>>>
>>>
>>> The usage of the raw xml file format as input to add a manifest to an
>>> install service could still be supported, using a different option, -f
>>>
>>> # installadm add-manifest -f <manifest> -n svc1
>>>
>>>
>>> [ Part 3: "Attached Text" ]
>>>
>>> _______________________________________________
>>> caiman-discuss mailing list
>>> caiman-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/caiman-discuss
>>
>
>
