Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Chris Friesen chris.frie...@windriver.com wrote on 04/28/2014 10:44:46 AM: On 04/26/2014 09:41 PM, Jay Lau wrote: ... My idea is that can we add a new field such as PlacemenetPolicy to AutoScalingGroup? If the value is affinity, then when heat engine create the AutoScalingGroup, it will first create a server group with affinity policy, then when create VM instance for the AutoScalingGroup, heat engine will transfer the server group id as scheduler hints so as to make sure all the VM instances in the AutoScalingGroup can be created with affinity policy. While I personally like this sort of idea from the perspective of simplifying things for heat users, I see two problems. First, my impression is that heat tries to provide a direct mapping of nova resources to heat resources. That matches my understanding too. But autoscaling groups (all three kinds) are already broken in that regard: they are not Nova resources, nor resources of any other service, but purely creatures of Heat's creation. There is a blueprint for fixing this, but it is only very partially implemented at the moment. Using a property of a heat resource to trigger the creation of a nova resource would not fit that model. For the sake of your argument, let's pretend that the new ASG blueprint has been fully implemented. That means an ASG is an ordinary virtual resource. In all likelihood the implementation will generate templates and make nested stacks. I think it is fairly natural to suppose that the generated template could include a Nova server group. Second, it seems less well positioned for exposing possible server group enhancements in nova. For example, one enhancement that has been discussed is to add a server group option to make the group scheduling policy a weighting factor if it can't be satisfied as a filter. With the server group as an explicit resource there is a natural way to extend it. Abstractly an autoscaling group is a sub-class of group of servers (ignoring the generalization of server in the relevant cases), so it would seem natural to me that the properties of an autoscaling group would include the properties of a server group. As the latter evolves, so would the former. OTOH, I find nothing particularly bad with doing it as you suggest. BTW, this is just the beginning. What about resources of type AWS::CloudFormation::Stack? What about Trove and Sahara? Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 04/30/2014 03:41 PM, Mike Spreitzer wrote: Chris Friesen chris.frie...@windriver.com wrote on 04/28/2014 10:44:46 AM: Using a property of a heat resource to trigger the creation of a nova resource would not fit that model. For the sake of your argument, let's pretend that the new ASG blueprint has been fully implemented. That means an ASG is an ordinary virtual resource. In all likelihood the implementation will generate templates and make nested stacks. I think it is fairly natural to suppose that the generated template could include a Nova server group. Second, it seems less well positioned for exposing possible server group enhancements in nova. For example, one enhancement that has been discussed is to add a server group option to make the group scheduling policy a weighting factor if it can't be satisfied as a filter. With the server group as an explicit resource there is a natural way to extend it. Abstractly an autoscaling group is a sub-class of group of servers (ignoring the generalization of server in the relevant cases), so it would seem natural to me that the properties of an autoscaling group would include the properties of a server group. As the latter evolves, so would the former. OTOH, I find nothing particularly bad with doing it as you suggest. BTW, this is just the beginning. What about resources of type AWS::CloudFormation::Stack? What about Trove and Sahara? If we go with what Zane suggested (using the already-exposed scheduler_hints) then by implementing a single server group resource we basically get support for server groups for free in any resource that exposes scheduler hints. That seems to me to be an excellent reason to go that route rather than modifying all the different group-like resources that heat supports. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Chris Friesen chris.frie...@windriver.com wrote on 04/30/2014 06:07:49 PM: If we go with what Zane suggested (using the already-exposed scheduler_hints) then by implementing a single server group resource we basically get support for server groups for free in any resource that exposes scheduler hints. That seems to me to be an excellent reason to go that route rather than modifying all the different group-like resources that heat supports. Yes, I agree there is economy of implementation in that approach. However it seems (to me, anyway) a little awkward from the point of view of the template author. Not terribly so, but a little. I am generally wary of analogies, but I will try to make one that is not much of a stretch. Consider a multiple-inheritance class hierarchy where class C inherits from A and B; when constructing a C, the user passes the constructor parameters of both A and B. That's fairly natural and usable. Now consider an alternative language that forbids multiple inheritance of classes; class A knows that it might work together with a B to accomplish what the forbidden C would do; to use the combined functionality the user has to construct a B and pass it to the constructor of A. That works, but the language with multiple inheritance is more convenient to use. Trove and Sahara are not implemented by Heat. They are going to be more interesting cases. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 04/26/2014 09:41 PM, Jay Lau wrote:
Just noticed this email, I have already filed a blueprint related to
this topic
https://blueprints.launchpad.net/heat/+spec/vm-instance-group-support
My idea is that can we add a new field such as PlacemenetPolicy to
AutoScalingGroup? If the value is affinity, then when heat engine create
the AutoScalingGroup, it will first create a server group with affinity
policy, then when create VM instance for the AutoScalingGroup, heat
engine will transfer the server group id as scheduler hints so as to
make sure all the VM instances in the AutoScalingGroup can be created
with affinity policy.
resources:
WorkloadGroup:
type: AWS::AutoScaling::AutoScalingGroup
properties:
AvailabilityZones: [nova]
LaunchConfigurationName: {Ref: LaunchConfig}
PlacementPolicy: [affinity]
MaxSize: 3
MinSize: 2
While I personally like this sort of idea from the perspective of
simplifying things for heat users, I see two problems.
First, my impression is that heat tries to provide a direct mapping of
nova resources to heat resources. Using a property of a heat resource
to trigger the creation of a nova resource would not fit that model.
Second, it seems less well positioned for exposing possible server group
enhancements in nova. For example, one enhancement that has been
discussed is to add a server group option to make the group scheduling
policy a weighting factor if it can't be satisfied as a filter. With
the server group as an explicit resource there is a natural way to
extend it.
Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Just noticed this email, I have already filed a blueprint related to this
topic https://blueprints.launchpad.net/heat/+spec/vm-instance-group-support
My idea is that can we add a new field such as PlacemenetPolicy to
AutoScalingGroup? If the value is affinity, then when heat engine create
the AutoScalingGroup, it will first create a server group with affinity
policy, then when create VM instance for the AutoScalingGroup, heat engine
will transfer the server group id as scheduler hints so as to make sure all
the VM instances in the AutoScalingGroup can be created with affinity
policy.
resources:
WorkloadGroup:
type: AWS::AutoScaling::AutoScalingGroup
properties:
AvailabilityZones: [nova]
LaunchConfigurationName: {Ref: LaunchConfig}
PlacementPolicy: [affinity]
MaxSize: 3
MinSize: 2
2014-04-26 5:27 GMT+08:00 Zane Bitter zbit...@redhat.com:
On 25/04/14 16:07, Chris Friesen wrote:
On 04/25/2014 12:00 PM, Zane Bitter wrote:
On 25/04/14 13:50, Chris Friesen wrote:
In the nova boot command we pass the group uuid like this:
--hint group=e4cf5dea-4831-49a1-867d-e263f2579dd0
If we were to make use of the scheduler hints, how would that look?
Something like this? (I'm not up to speed on my YAML, so forgive me if
this isn't quite right.) And how would this look if we wanted to
specify other scheduler hints as well?
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints: {group: { get_resource: my_heat_group }}
Something like that (I don't think you need the quotes around group).
Or, equivalently:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints:
group: { get_resource: my_heat_group }
Okay...assuming it works like that then that looks fine to me.
Cool, +1 for that then.
If we go this route then the changes are confined to a single new file.
Given that, do we need a blueprint or can I just submit the code for
review once I port it to the current codebase?
I guess wearing my PTL hat I ought to say that you should still raise a
blueprint (no real content necessary though, or just link to this thread).
Wearing my core team hat, I personally couldn't care less either way ;)
The change is self-explanatory and you've already done a good job of
consulting on the changes before submitting them.
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Thanks,
Jay
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Jay Lau jay.lau@gmail.com wrote on 04/26/2014 11:41:25 PM:
Just noticed this email, I have already filed a blueprint related to
this topic https://blueprints.launchpad.net/heat/+spec/vm-instance-
group-support
My idea is that can we add a new field such as PlacemenetPolicy to
AutoScalingGroup? If the value is affinity, then when heat engine
create the AutoScalingGroup, it will first create a server group
with affinity policy, then when create VM instance for the
AutoScalingGroup, heat engine will transfer the server group id as
scheduler hints so as to make sure all the VM instances in the
AutoScalingGroup can be created with affinity policy.
resources:
WorkloadGroup:
type: AWS::AutoScaling::AutoScalingGroup
properties:
AvailabilityZones: [nova]
LaunchConfigurationName: {Ref: LaunchConfig}
PlacementPolicy: [affinity]
MaxSize: 3
MinSize: 2
Remember that Heat has two resource types named
something::something::AutoScalingGroup, plus OS::Heat::InstanceGroup,
and also OS::Heat::ResourceGroup. Two of those four kinds of group are
almost (remember the indirection through ScaledResource) certainly groups
of Compute instances; the other two are more open about their elements. I
think the thing you are clearly saying that might be new in this
discussion is the proposal to make one or more of those four kinds of
groups use Nova's server group feature. The relevant kind of group would
then gain all the properties, attributes, privileges, and responsibilities
of a server group. At least, that makes sense to me. The biggest
questions in my mind are which kinds of groups should get this treatment
and how to cope with the cases where the element is NOT a Compute
instance. A sub-case of that last issue is: the scaled element is a
composite thing that includes one or more Compute instances.
Regards,
Mike
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
I'm looking to add support for server groups to heat. I've got working code, but I thought I'd post the overall design here in case people had objections. Basically, what I propose is to add a class NovaServerGroup resource. Currently it would only support a policy property to store the scheduler policy for the server group. The scheduler policy would not support updating on the fly. The LaunchConfiguration and Instance classes would be extended with an optional ServerGroup property. In the Instance class if the ServerGroup property is set then the group name is added to the scheduler_hints when building the instance. The Server class would be extended with an optional server_group property. If it is set then the group name is added to the scheduler_hints when building the server. All in all, its around a hundred lines of code. Any comments? Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Chris Friesen chris.frie...@windriver.com wrote on 04/25/2014 12:23:00 PM: I'm looking to add support for server groups to heat. I've got working code, but I thought I'd post the overall design here in case people had objections. Basically, what I propose is to add a class NovaServerGroup resource. Currently it would only support a policy property to store the scheduler policy for the server group. The scheduler policy would not support updating on the fly. The LaunchConfiguration and Instance classes would be extended with an optional ServerGroup property. In the Instance class if the ServerGroup property is set then the group name is added to the scheduler_hints when building the instance. The Server class would be extended with an optional server_group property. If it is set then the group name is added to the scheduler_hints when building the server. All in all, its around a hundred lines of code. Any comments? Sounds exactly right to me, and timely too. Cheers, Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 25/04/14 12:23, Chris Friesen wrote: I'm looking to add support for server groups to heat. I've got working code, but I thought I'd post the overall design here in case people had objections. Basically, what I propose is to add a class NovaServerGroup resource. Currently it would only support a policy property to store the scheduler policy for the server group. The scheduler policy would not support updating on the fly. If I correctly understood Mike when he previously talked about this, a server group policy is an actual thing in Nova with a UUID that gets passed to servers when you create them. If that is the case, then +1 for this design. The LaunchConfiguration and Instance classes would be extended with an optional ServerGroup property. In the Instance class if the ServerGroup property is set then the group name is added to the scheduler_hints when building the instance. -1 for making changes to AWS resources. These only exist for portability from/to CloudFormation; if people want to use OpenStack-only features then they should use the native resource types. In the case of autoscaling, I'd say you probably want to add the property to e.g. InstanceGroup rather than to the LaunchConfiguration. (I guess this will become somewhat academic in the future, as I believe the plan for new native autoscaling resources is to have the launch configuration defined as part of the scaling group.) The Server class would be extended with an optional server_group property. If it is set then the group name is added to the scheduler_hints when building the server. Given that we already expose the scheduler_hints directly, can you talk about why it would be advantageous to have a separate property as well? (e.g. syntax would be really finicky?) cheers, Zane. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
Zane Bitter zbit...@redhat.com wrote on 04/25/2014 12:36:00 PM: On 25/04/14 12:23, Chris Friesen wrote: ... The LaunchConfiguration and Instance classes would be extended with an optional ServerGroup property. In the Instance class if the ServerGroup property is set then the group name is added to the scheduler_hints when building the instance. -1 for making changes to AWS resources. These only exist for portability from/to CloudFormation; if people want to use OpenStack-only features then they should use the native resource types. Oh yes, I overlooked that in my enthusiasm. Good catch. In the case of autoscaling, I'd say you probably want to add the property to e.g. InstanceGroup rather than to the LaunchConfiguration. (I guess this will become somewhat academic in the future, as I believe the plan for new native autoscaling resources is to have the launch configuration defined as part of the scaling group.) Two of our current four kinds of group have already dispatched LaunchConfig to, well, pick your favorite from http://www.nytimes.com/1983/10/16/magazine/on-language-dust-heaps-of-history.html As pointed out above, one of the two LaunchConfig-philes, AWS::AutoScaling::AutoScalingGroup, should be left alone. That leaves OS::Heat::InstanceGroup --- which is, in the Python, a superclass of the AWS ASG --- so it would be oddly irregular to add something to InstanceGroup but not the AWS ASG. More important is Zane's following question. The Server class would be extended with an optional server_group property. If it is set then the group name is added to the scheduler_hints when building the server. Given that we already expose the scheduler_hints directly, can you talk about why it would be advantageous to have a separate property as well? (e.g. syntax would be really finicky?) Mike___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 04/25/2014 11:01 AM, Mike Spreitzer wrote:
Zane Bitter zbit...@redhat.com wrote on 04/25/2014 12:36:00 PM:
On 25/04/14 12:23, Chris Friesen wrote:
More important is Zane's following question.
The Server class would be extended with an optional server_group
property. If it is set then the group name is added to the
scheduler_hints when building the server.
Given that we already expose the scheduler_hints directly, can you talk
about why it would be advantageous to have a separate property as well?
(e.g. syntax would be really finicky?)
I was thinking it'd be more intuitive for the end-user (and more
future-proof if novaclient changes), but maybe I'm wrong.
In the version I have currently it looks something like this:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
server_group: { get_resource: my_heat_group }
In the nova boot command we pass the group uuid like this:
--hint group=e4cf5dea-4831-49a1-867d-e263f2579dd0
If we were to make use of the scheduler hints, how would that look?
Something like this? (I'm not up to speed on my YAML, so forgive me if
this isn't quite right.) And how would this look if we wanted to
specify other scheduler hints as well?
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints: {group: { get_resource: my_heat_group }}
Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 25/04/14 13:50, Chris Friesen wrote:
On 04/25/2014 11:01 AM, Mike Spreitzer wrote:
Zane Bitter zbit...@redhat.com wrote on 04/25/2014 12:36:00 PM:
On 25/04/14 12:23, Chris Friesen wrote:
More important is Zane's following question.
The Server class would be extended with an optional server_group
property. If it is set then the group name is added to the
scheduler_hints when building the server.
Given that we already expose the scheduler_hints directly, can you
talk
about why it would be advantageous to have a separate property as
well?
(e.g. syntax would be really finicky?)
I was thinking it'd be more intuitive for the end-user (and more
future-proof if novaclient changes), but maybe I'm wrong.
In the version I have currently it looks something like this:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
server_group: { get_resource: my_heat_group }
In the nova boot command we pass the group uuid like this:
--hint group=e4cf5dea-4831-49a1-867d-e263f2579dd0
If we were to make use of the scheduler hints, how would that look?
Something like this? (I'm not up to speed on my YAML, so forgive me if
this isn't quite right.) And how would this look if we wanted to
specify other scheduler hints as well?
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints: {group: { get_resource: my_heat_group }}
Something like that (I don't think you need the quotes around group).
Or, equivalently:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints:
group: { get_resource: my_heat_group }
- ZB
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 04/25/2014 12:00 PM, Zane Bitter wrote:
On 25/04/14 13:50, Chris Friesen wrote:
In the nova boot command we pass the group uuid like this:
--hint group=e4cf5dea-4831-49a1-867d-e263f2579dd0
If we were to make use of the scheduler hints, how would that look?
Something like this? (I'm not up to speed on my YAML, so forgive me if
this isn't quite right.) And how would this look if we wanted to
specify other scheduler hints as well?
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints: {group: { get_resource: my_heat_group }}
Something like that (I don't think you need the quotes around group).
Or, equivalently:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints:
group: { get_resource: my_heat_group }
Okay...assuming it works like that then that looks fine to me.
If we go this route then the changes are confined to a single new file.
Given that, do we need a blueprint or can I just submit the code for
review once I port it to the current codebase?
Chris
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat] looking to add support for server groups to heat...any comments?
On 25/04/14 16:07, Chris Friesen wrote:
On 04/25/2014 12:00 PM, Zane Bitter wrote:
On 25/04/14 13:50, Chris Friesen wrote:
In the nova boot command we pass the group uuid like this:
--hint group=e4cf5dea-4831-49a1-867d-e263f2579dd0
If we were to make use of the scheduler hints, how would that look?
Something like this? (I'm not up to speed on my YAML, so forgive me if
this isn't quite right.) And how would this look if we wanted to
specify other scheduler hints as well?
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints: {group: { get_resource: my_heat_group }}
Something like that (I don't think you need the quotes around group).
Or, equivalently:
cirros_server1:
type: OS::Nova::Server
properties:
name: cirros1
image: 'cirros'
flavor: 'm1.tiny'
scheduler_hints:
group: { get_resource: my_heat_group }
Okay...assuming it works like that then that looks fine to me.
Cool, +1 for that then.
If we go this route then the changes are confined to a single new file.
Given that, do we need a blueprint or can I just submit the code for
review once I port it to the current codebase?
I guess wearing my PTL hat I ought to say that you should still raise a
blueprint (no real content necessary though, or just link to this thread).
Wearing my core team hat, I personally couldn't care less either way ;)
The change is self-explanatory and you've already done a good job of
consulting on the changes before submitting them.
cheers,
Zane.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
