Hi Dave.
On 05/20/09 10:30, Dave Miner wrote:
> Somewhat belatedly following up:
>
> Jack Schwartz wrote:
>> Hi Dave.
>>
>> On 05/18/09 08:41, Dave Miner wrote:
>>> Jack Schwartz wrote:
>>>> HI everyone.
>>>>
>>>> Minutes of this meeting are posted at:
>>>> http://www.opensolaris.org/os/project/caiman/auto_install/AI_mtg/Minutes/XML_parser_rework_minutes_090515.txt
>>>>
>>>>
>>>>
>>>
>>> A couple of comments on sections I've pasted in:
>>>
>>>> Provide capability via enhanced SMF profiles to specify custom info
>>>> not used by
>>>> installer?
>>>> -------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>> Would we ever want to provide a user capability to specify custom
>>>> info via an
>>>> enhanced profile, where the installer isn't using that info? This
>>>> way, one can
>>>> customize the install (e.g. enable ssh).
>>>>
>>>> --> Conclusion: If this capability is implemented, we would want to
>>>> take the
>>>> custom info via a separate file.
>>>
>>> I think some elaboration here is required, as it appears to me that
>>> you're proposing to support only a limited subset of enhanced
>>> profile functionality. Why would we not, by default, allow
>>> customization of any SMF setting via the profiles supplied here?
>> My understanding is that AI will make use of some standard subset of
>> enhanced profile functionality, and there will be a delivered SMF
>> profile which contains this subset. The installer will need to break
>> this out into individual SMF services. For those services which are
>> not yet implemented, the installer will invoke ICTs.
>>
>> A user can set up a second profile to supplement the "standard" one,
>> which can set up properties for additional configuration. These
>> properties would just be passed through the installer.
>>
>> Does this make sense, or am I missing something?
>
> I don't think this is quite the right way to look at it. I would
> phrase it more as "the system is meant to be configured using enhanced
> profiles functionality; AI will apply whatever is configured via that
> means". Exceptions to this rule should be rare.
Yes, my track has changed since I wrote this email, and I agree. Use a
tool to set up the enhanced SMF profile, then let AI install it.
>
> I'll add that, in the context of AI, almost all ICT's are a bug. The
> desire is to install and update systems using delivered packages
> without "installer magic", and that magic is what ICT's represent.
> The install from the live CD is different, in that the ICT's there
> should represent reversing the modifications applied to the
> package-installed image to construct the live CD.
Fair enough. This is out of scope of XML parsing, but would affect
other parts of AI though. Not sure how AI will handle unimplemented
service bundles until SMF implements their service.
>
>>>
>>>
>>>> Since there could be conditions where the manifest could still work
>>>> even though
>>>> there is a version mismatch, the conclusion is to print a warning
>>>> on version
>>>> mismatch but try to continue. Then if validation fails or other
>>>> errors occur,
>>>> the user has been alerted to a version mismatch as a possible cause.
>>>
>>> What specific, useful scenarios do we think this loose versioning
>>> supports? The notes don't mention any, but I'd expect some
>>> elaboration on why a strict check is not appropriate.
>> Don't we want to have the installation work if it can? It is
>> possible that a revved version will still work, and why not let it?
>> The warning will simply say that a version mismatch has been detected
>> and the installation could fail.
>>
>
> Predictability and correctness represent higher goals in my view;
> failure when the known constraints are violated (as represented in
> this case by versioning mismatches) would seem to lead to better
> results on both counts.
OK. So noted.
>
>> If the installation succeeds, so much the better. If it fails, the
>> user can point to the warning and know what to check. Failures are
>> likely to show as manifest validation errors, and the warning will
>> help explain why the errors happened.
>>
>> As a specific example, consider the case where a new optional feature
>> is added. An old manifest will still work, and we want it to,
>> right? If the feature added were not optional, the same warning
>> would print, but the validation would fail and the warning would give
>> the reason.
>>
>> In many cases the versioning check is not necessary, since installadm
>> will be labeling the manifest when it validates, and will pair the
>> manifest with an image when creating a service. Versioning serves as
>> a check in case the manifest gets paired up with a different
>> installer/schema.
>
> I don't see any specific use cases here, only a general "we might want
> this" statement. I'd expect a specific example to justify it.
I'll note to take a closer look at use cases for this...
>
>>>
>>>> Problem statement 4: Semantic validation is needed for AI.
>>>> ==========================================================
>>>> - Lack of it means failures further down the installation process
>>>> instead of up front, or misconfiguration.
>>>>
>>> ...
>>>> --> Conclusion:
>>>> If not too much extra work involved, do it for server and client.
>>>> Else, do it for the client only.
>>>>
>>>
>>> I'd view it as an important requirement that we be able to perform
>>> full semantic verification on the server. Why is this regarded as
>>> optional?
>> It's not that it's optional on the server, it's that the server
>> cannot provide the full client context to do full validation. Of
>> course we want to do as much validation as we can on both the server
>> and the client.
>>
>
> I would expect that a user could supply whatever context we desire to
> use, either synthetically or via importation of some client context
> that was generated from a real client. From the point of view of the
> validator, the client is just a bunch of parameters, which can also be
> supplied via other avenues. Requiring real, live clients in order to
> validate thus seems unnecessary.
>
From a subsequent email, I see you are not talking about contacting a
client, but through a manifest development tool which could mimic a
client's settings and export them for validation purposes.
>> In response to your reply to Clay:
>> > Wouldn't it be desirable to be able to construct a virtual validation
>> > "environment" on the server based on user input? Let them specify
>> > the disk configuration, the memory, etc. and validate against it.
>> Such
>> > functionality would seem to be necessary in order to effectively test
>> > this, wouldn't it?
>>
>> Please elaborate, because as I understand, I'm not sure this is
>> feasible so I may just not get what you're saying.
>>
>> For example, as Clay said in his first response, in the case of a
>> disk configuration, how would the user know that such a disk by that
>> name really exists on the client? We can't assume the client is up
>> and available, or even if it is up, that the disk would have that
>> name in its currently-booted OS instance.
>
> See my note above about synthetic environments. If our only answer to
> testing out sets of criteria is to "boot and hope", then I think we've
> fallen far short of what's possible.
>
>>>
>>>> Problem statement 1: AI's multiple parsers present unneeded
>>>> complexity and unmaintainability.
>>>> ===========================================================
>>>> - Things to consider for a single parser:
>>>> - functionality for data retrieval and search, schema compatibility,
>>>> how supported / maintainable is the parser
>>> ...
>>>> The main argument for using lxml would be that it is supported from
>>>> the
>>>> outside. However, that ManifestServ is supported by us means we
>>>> can more
>>>> easily tweek it to add any needed functionality which remains. If
>>>> lxml were
>>>> used, we would have to fix our code which interfaced with it if it
>>>> changed, but
>>>> we would have to do the same for ManifestServ if libxml2 changed.
>>>>
>>>
>>> It wasn't entirely clear from the discussion in the notes that we've
>>> actually identified requirements that lxml doesn't meet. Do we have
>>> that?
>> lxml meets most of the requirements we would need, but it would
>> require some work writing new wrapper code to use it, and
>> ManifestServ has this already done. For example, lxml does have
>> something we can use for semantic validation but would require event
>> handlers and other code to make it work. ManifestServ already
>> presents a complete system for this using libxml2. lxml won't buy us
>> anything more than what ManifestServ already delivers, and will
>> require an effort to make it useful which isn't required with
>> ManifestServ.
>>
>> One shortcoming of lxml vs ManifestServ is that it doesn't have as
>> good of a capability for narrowing of matches based on values of
>> multiple child nodes.
>>
>
> I don't have any particular bias one way or another, but there's
> up-front vs. on-going effort involved in any choice made, and I'd
> expect to see some evaluation of those trade-offs in the design proposal.
OK. The strawman will be given this information.
Thanks,
Jack
>
> Dave
