On 07/06/16 23:53, Hongbin Lu wrote:
Hi Heat team,
A question inline.
Best regards,
Hongbin
-----Original Message-----
From: Steven Hardy [mailto:sha...@redhat.com]
Sent: March-03-16 3:57 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum][heat] spawn a group of nodes on
different availability zones
On Wed, Mar 02, 2016 at 05:40:20PM -0500, Zane Bitter wrote:
On 02/03/16 05:50, Mathieu Velten wrote:
Hi all,
I am looking at a way to spawn nodes in different specified
availability zones when deploying a cluster with Magnum.
Currently Magnum directly uses predefined Heat templates with Heat
parameters to handle configuration.
I tried to reach my goal by sticking to this model, however I
couldn't find a suitable Heat construct that would allow that.
Here are the details of my investigation :
- OS::Heat::ResourceGroup doesn't allow to specify a list as a
variable that would be iterated over, so we would need one
ResourceGroup by AZ
- OS::Nova::ServerGroup only allows restriction at the hypervisor
level
- OS::Heat::InstanceGroup has an AZs parameter but it is marked
unimplemented , and is CFN specific.
- OS::Nova::HostAggregate only seems to allow adding some metadatas
to a group of hosts in a defined availability zone
- repeat function only works inside the properties section of a
resource and can't be used at the resource level itself, hence
something like that is not allowed :
resources:
repeat:
for_each:
<%az%>: { get_param: availability_zones }
template:
rg-<%az%>:
type: OS::Heat::ResourceGroup
properties:
count: 2
resource_def:
type: hot_single_server.yaml
properties:
availability_zone: <%az%>
The only possibility that I see is generating a ResourceGroup by AZ,
but it would induce some big changes in Magnum to handle
modification/generation of templates.
Any ideas ?
This is a long-standing missing feature in Heat. There are two
blueprints for this (I'm not sure why):
https://blueprints.launchpad.net/heat/+spec/autoscaling-
availabilityzo
nes-impl
https://blueprints.launchpad.net/heat/+spec/implement-
autoscalinggroup
-availabilityzones
The latter had a spec with quite a lot of discussion:
https://review.openstack.org/#/c/105907
And even an attempted implementation:
https://review.openstack.org/#/c/116139/
which was making some progress but is long out of date and would need
serious work to rebase. The good news is that some of the changes I
made in Liberty like https://review.openstack.org/#/c/213555/ should
hopefully make it simpler.
All of which is to say, if you want to help then I think it would be
totally do-able to land support for this relatively early in Newton :)
Failing that, the only think I can think to try is something I am
pretty sure won't work: a ResourceGroup with something like:
availability_zone: {get_param: [AZ_map, "%i"]}
where AZ_map looks something like {"0": "az-1", "1": "az-2", "2":
"az-1", ...} and you're using the member index to pick out the AZ to
use from the parameter. I don't think that works (if "%i" is resolved
after get_param then it won't, and I suspect that's the case) but
it's
worth a try if you need a solution in Mitaka.
Yeah, this won't work if you attempt to do the map/index lookup in the
top-level template where the ResourceGroup is defined, but it *does*
work if you pass both the map and the index into the nested stack, e.g
something like this (untested):
$ cat rg_az_map.yaml
heat_template_version: 2015-04-30
parameters:
az_map:
type: json
default:
'0': az1
'1': az2
resources:
AGroup:
type: OS::Heat::ResourceGroup
properties:
count: 2
resource_def:
type: server_mapped_az.yaml
properties:
availability_zone_map: {get_param: az_map}
index: '%index%'
$ cat server_mapped_az.yaml
heat_template_version: 2015-04-30
parameters:
availability_zone_map:
type: json
index:
type: string
resources:
server:
type: OS::Nova::Server
properties:
image: the_image
flavor: m1.foo
availability_zone: {get_param: [availability_zone_map, {get_param:
index}]}
This is nice. It seems to address our heterogeneity requirement at *deploy*
time. However, I wonder what is the runtime behavior. For example, I deploy a
stack by:
$ heat stack-create -f rg_az_map.yaml -P az_map='{"0":"az1","1":"az2"}'
Then, I want to remove a sever by:
$ heat stack-update -f rg_az_map.yaml -P az_map='{"0":"az1"}'
Will Heat remove resources in index "1" only (with resources in index "0"
untouched)? Also, I wonder if we can dynamically add resources (with existing resources untouched).
For example, add a server by:
$ heat stack-update -f rg_az_map.yaml -P
az_map='{"0":"az1","1":"az2","2":"az3"}'
Removing members from the end of a ResourceGroup works fairly well. It's
when you have to remove members from anywhere else in the list that
things go very very bad very very quickly, which is the reason that I
don't recommend ResourceGroup to anybody.
In addition, I want to point out that spreading the availability zones is not
the only use case. Magnum has generic use cases to manage heterogeneous set of
resources. For example:
$ heat stack-create -f rg_az_map.yaml -P
az_map='{"resource_gorup_1":{"availability_zone":"az1","count":"2","flavor":"m1.foo",...},"resource_gorup_2":{"availability_zone":"az2","count":"3","flavor":"m2.foo",...},...}"
Is it a reasonable to expect Heat to support that?
IMHO no, it's not. If you want a stack of heterogeneous resources, just
create a template with a bunch of heterogeneous resources. (We're hoping
to build a library[1] that will make this even easier, but it's pretty
straightforward even without that.)
Think of ResourceGroup and even Autoscaling groups as the world's
dumbest template generator. The reason to use them is if you need to
orchestration capabilities that they have tightly integrated with Heat -
e.g. batched rolling upgrades. Turning them into general-purpose
template generators is very much beside the point and the results will
never be entirely satisfactory.
cheers,
Zane.
[1] https://review.openstack.org/#/c/328822/
FWIW we already use this technique in some TripleO templates, and it
works pretty well.
https://github.com/openstack/tripleo-heat-
templates/blob/master/network/ports/external_from_pool.yaml#L35
Steve
_______________________________________________________________________
___
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-
requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__________________________________________________________________________
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
