On 28/10/13 16:26 -0700, Clint Byrum wrote:

[snip]

>
My components proposals had no hosted_on, but I've been thinking about
the implications of implementing software configuration as resources,
and one of the natural consequences might be that hosted_on is the best
way of establishing the relationship with compute and its
configurations. Let me elaborate.

Lets say that Heat has resource types for software configuration, with
the following behaviours:
* like other resources, a config resource goes into CREATE IN_PROGRESS
as soon as its dependencies are satisfied (these dependencies may be
values signalled from other config resources)
* a config resource goes to state CREATE COMPLETE when it receives a
signal that configuration on the compute resource is complete (by some
mechanism; wait condition, marconi message, whatevs)
* the relationship between config resources and compute resources are
achieved with existing intrinsic functions (get_resource, get_attr)

This lifecycle behaviour means that a configuration resource can only
run on a single compute resource, and that relationship needs to be
established somehow. Config resources will have a quirk in that they
need to provide 2 sources of configuration data at different times:
1) cloud-init config-data (or other boot-only mechanism), which must be
available when the compute resource goes into CREATE IN_PROGRESS
2) the actual configuration data (oac metadata, puppet manifest, chef
recipe) which the compute resource needs to be able to fetch and execute
whenever it becomes available.

The data in 1) implies that the compute needs to depend on the config,
but then all concurrency is lost (this won't matter for a
cloud-init-only config resource).  Either way, the data for 1) will need
to be available when the config resource is still in state INIT
COMPLETE, which may impose limitations on how that is defined (ie
get_resource, get_attr not allowed).

So, 2 concrete examples for handling config/compute dependencies:

hosted_on
---------
resources:
  configA
    type: Heat::ScriptConfig
    properties:
       hosted_on:
         get_resource: the_server
       script: |
         #!/bin/bash
         logger "1. hello config A"
  configB
    type: Heat::ScriptConfig
    properties:
       hosted_on:
         get_resource: the_server
       script: |
         #!/bin/bash
         logger "1. hello config B"
  the_server:
    type: OS::Nova::Server

Here, configA and configB go into CREATE IN_PROGRESS as soon as
the_server is CREATE COMPLETE. configA and configB go into CREATE
COMPLETE when heat-engine receives a signal that they are complete. This
signal may include values that other resources depend on to start their
creation.


My biggest objection to hosted_on is that if I want to reuse configA.. I
can't without an additional level of abstraction. This feels awkward,
and will be an awkward thing for users. In addition, it is very unclear
what what will actually be on "the_server" when looking at "the_server".

Nobody has answered my question where else this sort of pattern is used.

OS::Nova::Server config_resources
---------------------------------
resources:
  configA
    type: Heat::ScriptConfig
    properties:
       script: |
         #!/bin/bash
         logger "1. hello config A"
  configB
    type: Heat::ScriptConfig
    properties:
       script: |
         #!/bin/bash
         logger "1. hello config B"
  the_server:
    type: OS::Nova::Server
    properties:
      config_resources:
        - {get_resource: configA}
        - {get_resource: configB}

Here there would need to be some bypassing of dependency calculation to
allow configA and configB to be created after the_server, maybe by:
* special treatment of config_resources to prevent dependencies being
created
* a get_resource variant which doesn't create a hard dependency
(get_resource_deferred?)


IMO no, these are not best expressed as resources in the same way that
python classes are not expressed as objects.

The template above reads more like code. If used as a component, configA
remains reusable throughout the rest of the template as it grows and
changes, and it is very obvious what is expected on "the_server". Each
server using configA gets its very own completion waitcondition for that
instantiation of the component.

This is why I liked the components concept, rather than have them as
resources. It seems clear to me that components are parse-time objects,
so if I refer to the same component 4 times, I really want 4 copies of
it. Resources, on the other hand, are real things that are instantiated.

Neither the hosted_on nor the config_resources behaviours are ideal, but
I'm leaning towards hosted_on at the moment since it doesn't require any
new soft-dependency mechanism.


IMO hosted_on is a new dependency mechanism, so avoiding the more clear
paradigm for the reason of not introducing dependency mechanism seems a
bit backwards too.

As for composability, what *actually* needs to be composable is the
contents of the script (manifest, recipe) property. Everything else in a
config resource is just stack-specific wiring.  There are a couple of
ways this composability could be achieved:
1) resource provider in the user environment which specifies the script
property
2) __include__ or some equivalent client-side inclusion mechanism


At first it seems like all of it is just per-stack wiring, but the point
is that even just inside a stack, you will want to reuse whole components,
not just the script/cloud-config/etc.

I've done a client side __include__ method for TripleO. It works fine
actually. But since we're developing a language, we need to think
longer and harder than "works for me", otherwise we will make the PHP
of orchestration languages.

Zane had an idea for a "FileRef" of some kind that I nealy had time
to implement. This would work is a simerlar way to the way template
resources are uploaded into the "files" structure.


I'd like to see components as a first class facility in HOT so that users
can compose templates in a logical straight forward manner. The proposals
that you have laid out provide workable but less discoverable mechanisms.

At the moment we have something that alters a server in a somewhat
simlar way to configuring - volume attaching.

resources:
  server:
    bla...
  volume:
    bla...
  volume_attachment:
    volume_id: {Ref: volume}
    server_id: {Ref: server}


We could model config install in a simerlar way

resources:
  server:
    bla...
  swift_file:
    bla...
  config_installer:
    config_url: {Ref: swift_file}
    server_id: {Ref: server}

This could be shortened to a url:

resources:
  server:
    bla...
  config_installer:
    server_id: {Ref: server}
config_content: str_replace: template: {get_file: config_url}
        params:
          foo: whatever

- you could reuse the config in multiple places
- the installer could pass variables into config
  (and potentially get outputs out for dynamic attributes)


[I hope we get big whiteboards at summit :-) ]

-Angus

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Reply via email to