On 02/20/2013 02:18 PM, Dan Callaghan wrote:
I've just published the initial version my design proposal for a stable
harness API [1], following Beaker's new design proposal process. The API
it describes is mostly the same as my original draft from an earlier
thread on this list [2]. I added a "none" result ("N/A" was awkward
because of the slash and "Not Applicable" because of its two words) and
a way for the harness to attach arbitrary URLs as logs, as suggested by
Bill [3].
Thanks for getting this written up Dan! It will give us a good basis for
discussion with our users.
As a general point, these days we probably want to use application/json
as the POST format rather than x-www-form-urlencoded form data.
As far as providing the recipe ID goes, perhaps a BEAKER_RECIPE_ID
environment var would be better than a file? That should be easier to
make consistent cross-platform, and also matches the way we provide URL
details.
For the other two settings, how about BEAKER_API and BEAKER_HUB?
The intent of BEAKER_API would be that API access is simply $BEAKER_API
+ documented API location (e.g. allowing us to expose different versions
at different URLs if we decide we need to upgrade at some point in the
future).
I'm not clear on why we need to provide BEAKER_HUB (don't we want all
access routed through the stable harness API?), but I prefer some kind
of suffix over a bare BEAKER environment variable and HUB makes sense
since we're going with --hub for the CLI option.
I think Nick also has some suggestions about the alternative harness
selection too... so, let the discussions begin!
Oh yes. (Although it's at least partly Bill's fault, he gave me ideas...)
One common complaint from folks attempting to use the current private
XML-RPC API to create a custom harness is that there's no way for the
code running on the system under test to add tasks to the recipe. This
means that an entire test run using an alternate harness will be
collapsed into a single task result in Beaker. That's ugly, and makes it
hard to upload good logs, provide decent progress reporting, granular
test results, etc.
What would be highly desirable is to have a way for the Beaker job XML
to contain *just* enough detail to get an alternate harness up and
running, and then the harness tells *Beaker* what the tasks are rather
than the other way around.
The design goals I think we should be aiming for when it comes to
alternate harnesses:
1. Transport arbitrary XML configuration settings for the harness
through the recipe definition
2. No required references to the Beaker task library in the recipe at
all (unless a particular harness chooses to rely on it)
3. Encourage harnesses to add all their tasks to the recipe before
starting execution, but also allow them to add them incrementally
4. Allow users to automatically request reservation of the system upon
completion, independently of the harness used
So I think the approach to selecting alternate harnesses in the first
draft doesn't quite work, as it doesn't easily support configuration of
the harness itself, doesn't eliminate the need to define at least one
task for each recipe and doesn't provide a harness independent way to
reserve the system at the end. We also need to define an extra API
endpoint to submit a new task.
I'll cover the latter first, as I don't think it needs to be
particularly complicated:
POST /recipes/{recipeid}/tasks/
Content-type: application/json
label=Something meaningful for the harness used
-> 302 See Other
Location: /recipes/{recipeid}/tasks/{taskid}
This would append a new task to the current recipe. New tasks would
always be appended, with no mechanism for inserting them into the middle
of the existing task list. Harnesses would be encouraged to use this to
prepopulate the full list of tasks before they start running anything,
but they would also be allowed to interleave the addition of new tasks
with the completion of previous ones.
We may also want to allow harnesses to populate the "role" and "params"
parts of the task definition.
Now, on to the more complex part of the idea, the problem of deciding
which harness to install and providing it with arbitrary configuration
settings without Beaker needing to care what those settings are.
The initial draft of the alternate harness design has the XML look like
this:
<recipe ks_meta="harness=mylittleharness">
<repos>
<repo name="mylittleharness"
url="http://example.com/mylittleharness/el6/"; />
</repos>
...
</recipe>
However, this leaves in place the current requirement that every recipe
definition must include at least one task, and doesn't provide a task
library independent way to request reservation of the system after the
recipe executes.
My alternative suggestion is that it could look like this (the repos
element would be the same as above):
<recipe>
<harness label="My Little Harness" yuminstall="mylittleharness"/>
...
</recipe>
Or, if you want to reserve the system afterwards:
<recipe>
<harness label="My Little Harness" yuminstall="mylittleharness">
<reservesys/>
</harness>
...
</recipe>
In detail, my proposed change to the recipe_contents RELAX-NG schema
(http://beaker-project.org/schema/beaker-job.rng) is to replace the current
simple "<oneOrMore><ref name="task"/></oneOrMore>" part of the scheme with:
# In the recipe_contents definition
<choice>
<ref name="task_list"/>
<ref name="harness"/>
</choice>
# Elsewhere
<define name="task_list">
<oneOrMore>
<ref name="task"/>
</oneOrMore>
</define>
<define name="harness">
<element name="harness">
<optional>
<attribute name="display"/>
</optional>
<attribute name="yuminstall"/>
<optional>
<element name="reservesys"/>
</optional>
<choice>
<ref name="task_list"/>
<element name="config">
<ref name="arbitrary_attributes_and_elements"/>
</element>
</choice>
</element>
</define>
<define name="arbitrary_attributes_and_elements">
See http://relaxng.org/tutorial-20011203.html#IDAFLZR
and http://books.xmlschemata.org/relaxng/relax-CHP-11-SECT-4.html
</define>
At the "recipe_contents" level, the "task_list" part will preserve
compatibility with Beaker's current capabilities, and implies the usage
of the standard beah harness. The new "harness" element then explicitly
indicates the desire to bypass the normal beah-based task execution
behaviour and use a different harness.
At the harness level, the interesting parts would be:
1. The required "yuminstall" attribute of the harness element replaces
the ks_meta="harness=<yum arg>" part of Dan's initial design
The explicit nature of the "yuminstall" name leaves the door open to
less packaging system specific alternatives in the future
2. The optional "label" attribute gives us something human readable to
show in the UI (defaulting to the "yuminstall" value)
3. The ability to request reservation of the system after execution of
the recipe is separated out from the definition of what the harness
should do
4. "reservesys" is an *element* to leave the door open to possible
future feature like <reservesys onsuccess="false"> to only reserve it in
some cases.
5. The configuration mechanism for a custom harness is a choice between:
- a Beaker task list (for the benefit of custom harnesses that were
originally built to work within Beaker's current limitations)
- a custom harness-specific "config" element which can contain
arbitrary XML (such as a Git URL reference).
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
_______________________________________________
Beaker-devel mailing list
Beaker-devel@lists.fedorahosted.org
https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
