On 14/07/14 12:21, Tomas Sedovic wrote:
On 12/07/14 06:41, Zane Bitter wrote:
On 11/07/14 09:37, Tomas Sedovic wrote:
[snip]
3. List of IP addresses of all controllers:
https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L405
We cannot do this, because resource group doesn't support extended
attributes.
Would need something like:
{get_attr: [controller_group, networks, ctlplane, 0]}
(ctlplane is the network controller_group servers are on)
I was going to give an explanation of how we could implement this, but
then I realised a patch was going to be easier:
https://review.openstack.org/#/c/106541/
https://review.openstack.org/#/c/106542/
Thanks, that looks great.
4. IP address of the first node in the resource group:
https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/swift-deploy.yaml#L29
Can't do: extended attributes are not supported for the n-th node for
the group either.
I believe this is possible today using:
{get_attr: [controller_group, resource.0.networks, ctlplane, 0]}
Yeah, I've missed this. I have actually checked the ResourceGroup's
GetAtt method but didn't realise the connection with the GetAtt function
so I hadn't tried it before.
[snip]
Alternatively, we could extend the ResourceGroup's get_attr behaviour:
{get_attr: [controller_group, resource.0.networks.ctlplane.0]}
but the former is a bit cleaner and more generic.
I wrote a patch that implements this (and also handles (3) above in a
similar manner), but in the end I decided that this:
{get_attr: [controller_group, resource.0, networks, ctlplane, 0]}
would be better than either that or the current syntax (which was
obviously obscure enough that you didn't discover it). My only
reservation was that it might make things a little weird when we have an
autoscaling API to get attributes from compared with the dotted syntax
that you suggest, but I soon got over it ;)
So now that I understand how this works, I'm not against keeping things
the way we are. There is a consistency there, we just need to document
it and perhaps show some examples.
It kind of fell out of the work I was doing on the patches above anyway.
It would be harder to _not_ implement this (and the existing way still
works too).
---
That was the easy stuff, where we can get by with the current
functionality (plus a few fixes).
What follows are examples that really need new intrinsic functions (or
seriously complicating the ResourceGroup attribute code and syntax).
5. Building a list of {ip: ..., name: ...} dictionaries to configure
haproxy:
https://github.com/openstack/tripleo-heat-templates/blob/a7f2a2c928e9c78a18defb68feb40da8c7eb95d6/overcloud-source.yaml#L478
This really calls for a mapping/for-each kind of functionality. Trying
to invent a ResourceGroup syntax for this would be perverse.
Here's what it could look like under Clint's `map` proposal:
map:
- ip: {get_attr: [{get_resource: "$1"}, networks, ctlplane, 0]
name: {get_attr: [{get_resource: "$1"}, name]}
- {get_attr: [compute_group, refs]}
This has always been the tricky one :D
IMHO the real problem here is that we're trying to collate the data at
the point where it is consumed, not the point where it is produced. It's
not like we were just given a big blob of data and now have to somehow
extract the useful parts; we produced it ourselves by combining data
from the scaled units. If we didn't get the right data, we have only
ourselves to blame ;)
So if the provider template that defines e.g. a compute node contains
the section:
outputs:
host_entry:
value:
ip: {get_attr: [compute_server, networks, ctlplane, 0]}
name: {get_attr: [compute_server, name]}
Then in your main template all you need to do is:
{getattr: [compute_group, host_entry]}
Oh this is absolutely wonderful. I've had all the pieces in my head but
I didn't make the connection.
You're completely right about using a provider template here anyway --
that's what I planned to do, but I didn't fully appreciate the
connection between outputs and attributes (even though I knew about it).
to get the list of {ip: ..., name: ...} dicts. (As a bonus, this is
about as straightforward to read as I can imagine it ever getting.)
Note that you *will* want to be using a provider template as the scaled
unit _anyway_, because each compute_server will have associated software
deployments and quite possibly a bunch of other resources that need to
be in the scaled unit.
Yep, exactly.
There is one aspect of this that probably doesn't work yet: originally
outputs and attributes were only allowed to be strings. We changed that
for attributes, but probably not yet for outputs (even though outputs of
provider templates become attributes of the facade resource). But that
should be easy to fix. (And if your data can be returned as a string, it
should already work.)
Unless I misunderstood what you're saying, it seems to be working now:
controller.yaml:
outputs:
hosts_entry:
description: An IP address and a hostname of the server
value:
ip: {get_attr: [controller_server, networks, private, 0]}
name: {get_attr: [controller_server, name]}
environment.yaml:
resource_registry:
OS::TripleO::Controller: controller.yaml
test-resource-group.yaml:
resources:
servers:
type: OS::Heat::ResourceGroup
properties:
count: 3
resource_def:
type: OS::TripleO::Controller
properties:
key_name: {get_param: key_name}
image: {get_param: image_id}
outputs:
hosts:
description: "/etc/hosts entries for each server"
value: {get_attr: [servers, hosts_entry]}
Heat stack-show test-resource-group:
{
"output_value": [
"{u'ip': u'10.0.0.4', u'name':
u'rg-7heh-0-tweejsvubaht-controller_server-mscy33sbtirn'}",
"{u'ip': u'10.0.0.3', u'name':
u'rg-7heh-1-o4szl7lry27d-controller_server-sxpkalgi27ii'}",
"{u'ip': u'10.0.0.2', u'name':
u'rg-7heh-2-l2y6rqxml2fi-controller_server-u4jcjacjdrea'}"
],
"description": "/etc/hosts entries for each server",
"output_key": "hosts"
},
It looks like the dicts are being converted to strings by Python, so
there probably is a small bug here to be fixed. (At the very least, if
we're converting to strings we should do so using json.dumps(), not repr().)
[snip]
So this boils down to 4 features proposals:
1. Support extended attributes in ResourceGroup's members
Sorted.
Yep
2. Allow a way to use a Resource ID (e.g. what you get by {get_attr:
[ResourceGroup, refs]} or {get_attr: [ResourceGroup, resource.0]}) with
existing intrinsic functions (get_resource, get_attr)
No dice, but (1) solves the problem anyway.
Agreed
3. A `map` intrinsic function that turns a list of items to another list
by doing operations on each item
There may be a better solution available to us already, so IMO
investigate that first. If that turns out not to be the case then we'll
need to reach a consensus on whether map is something we want.
You're right. I no longer think map (or anything like it) is necessary.
That's the kind of thing I love to hear :D
4. A `concat_list` intrinsic function that joins multiple lists into one.
Low priority.
Yeah.
I think the first two are not controversial. What about the other two?
I've shown you some examples where we would find a good use in the
TripleO templates. The lack of `map` actually blocks us from going
all-Heat.
Hopefully that's not actually the case.
The alternative would be to say that this sort of stuff to be done
inside the instance by os-apply-config et al. It would complicate things
for TripleO, but oh well.
It seems to me that the alternative is not necessarily to modify
os-apply-config, but rather to provide a software config with a script
that converts the data from whatever format Heat can supply to whatever
format is needed by the application. Although I don't think it's even
required in the specific case you mention, I find that a much better
answer for the general case than turning the template format into the
JSON equivalent of XSLT ;)
That's what I thought -- not rewriting os-apply-config but simply
passing the raw data to it and changing the tripleo image elements to
output the configuration format required by the applications.
+1 on the XSLT sentiment, I just didn't realise we pretty much solve
everything with the existing model (hat off to the design btw, I love
how it works together -- we just maybe need to communicate it a bit more).
Yeah, we're thinking of trying to put something together for the summit
to explain how the pieces fit together. You're not the only one looking
for that documentation, I suspect :)
cheers,
Zane.
_______________________________________________
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
