Re: [openstack-dev] [Heat] Passing a list of ResourceGroup's attributes back to its members
On 14/08/14 00:00, Tomas Sedovic wrote: On 12/08/14 01:06, Steve Baker wrote: On 09/08/14 11:15, Zane Bitter wrote: On 08/08/14 11:07, Tomas Sedovic wrote: On 08/08/14 00:53, Zane Bitter wrote: On 07/08/14 13:22, Tomas Sedovic wrote: Hi all, I have a ResourceGroup which wraps a custom resource defined in another template: servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: my_custom_server properties: prop_1: ... prop_2: ... ... And a corresponding provider template and environment file. Now I can get say the list of IP addresses or any custom value of each server from the ResourceGroup by using `{get_attr: [servers, ip_address]}` and outputs defined in the provider template. But I can't figure out how to pass that list back to each server in the group. This is something we use in TripleO for things like building a MySQL cluster, where each node in the cluster (the ResourceGroup) needs the addresses of all the other nodes. Yeah, this is kind of the perpetual problem with clusters. I've been hoping that DNSaaS will show up in OpenStack soon and that that will be a way to fix this issue. The other option is to have the cluster members discover each other somehow (mDNS?), but people seem loath to do that. Right now, we have the servers ungrouped in the top-level template so we can build this list manually. But if we move to ResourceGroups (or any other scaling mechanism, I think), this is no longer possible. So I believe the current solution is to abuse a Launch Config resource as a store for the data, and then later retrieve it somehow? Possibly you could do something along similar lines, but it's unclear how the 'later retrieval' part would work... presumably it would have to involve something outside of Heat closing the loop :( Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble figuring out how would that work. LaunchConfig represents an instance, right? We can't pass the list to ResourceGroup's `resource_def` section because that causes a circular dependency. And I'm not aware of a way to attach a SoftwareConfig to a ResourceGroup. SoftwareDeployment only allows attaching a config to a single server. Yeah, and that would be a tricky thing to implement well, because a resource group may not be a group of servers (but in many cases it may be a group of nested stacks that each contain one or more servers, and you'd want to be able to handle that too). Yeah, I worried about that, too :-(. Here's a proposal that might actually work, though: The provider resource exposes the reference to its inner instance by declaring it as one of its outputs. A SoftwareDeployment would learn to accept a list of Nova servers, too. Provider template: resources: my_server: type: OS::Nova::Server properties: ... ... (some other resource hidden in the provider template) outputs: inner_server: value: {get_resource: my_server} ip_address: value: {get_attr: [controller_server, networks, private, 0]} Based on my limited testing, this already makes it possible to use the inner server with a SoftwareDeployment from another template that uses my_server as a provider resource. E.g.: a_cluster_of_my_servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: custom::my_server ... some_deploy: type: OS::Heat::StructuredDeployment properties: server: {get_attr: [a_cluster_of_my_servers, resource.0.inner_server]} config: {get_resource: some_config} So what if we allowed SoftwareDeployment to accept a list of servers in addition to accepting just one server? Or add another resource that does that. I approve of that in principle. Only Steve Baker can tell us for sure if there are any technical roadblocks in the way of that, but I don't see any. Maybe if we had a new resource type that was internally implemented as a nested stack... that might give us a way of tracking the individual deployment statuses for free. cheers, Zane. Then we could do: mysql_cluster_deployment: type: OS::Heat::StructuredDeployment properties: server_list: {get_attr: [a_cluster_of_my_servers, inner_server]} config: {get_resource: mysql_cluster_config} input_values: cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers, ip_address]} This isn't that different from having a SoftwareDeployment accepting a single server and doesn't have any of the problems of allowing a ResourceGroup as a SoftwareDeployment target. What do you think? All the other solutions I can think of will result in circular issues. I'll start
Re: [openstack-dev] [Heat] Passing a list of ResourceGroup's attributes back to its members
On 12/08/14 01:06, Steve Baker wrote: On 09/08/14 11:15, Zane Bitter wrote: On 08/08/14 11:07, Tomas Sedovic wrote: On 08/08/14 00:53, Zane Bitter wrote: On 07/08/14 13:22, Tomas Sedovic wrote: Hi all, I have a ResourceGroup which wraps a custom resource defined in another template: servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: my_custom_server properties: prop_1: ... prop_2: ... ... And a corresponding provider template and environment file. Now I can get say the list of IP addresses or any custom value of each server from the ResourceGroup by using `{get_attr: [servers, ip_address]}` and outputs defined in the provider template. But I can't figure out how to pass that list back to each server in the group. This is something we use in TripleO for things like building a MySQL cluster, where each node in the cluster (the ResourceGroup) needs the addresses of all the other nodes. Yeah, this is kind of the perpetual problem with clusters. I've been hoping that DNSaaS will show up in OpenStack soon and that that will be a way to fix this issue. The other option is to have the cluster members discover each other somehow (mDNS?), but people seem loath to do that. Right now, we have the servers ungrouped in the top-level template so we can build this list manually. But if we move to ResourceGroups (or any other scaling mechanism, I think), this is no longer possible. So I believe the current solution is to abuse a Launch Config resource as a store for the data, and then later retrieve it somehow? Possibly you could do something along similar lines, but it's unclear how the 'later retrieval' part would work... presumably it would have to involve something outside of Heat closing the loop :( Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble figuring out how would that work. LaunchConfig represents an instance, right? We can't pass the list to ResourceGroup's `resource_def` section because that causes a circular dependency. And I'm not aware of a way to attach a SoftwareConfig to a ResourceGroup. SoftwareDeployment only allows attaching a config to a single server. Yeah, and that would be a tricky thing to implement well, because a resource group may not be a group of servers (but in many cases it may be a group of nested stacks that each contain one or more servers, and you'd want to be able to handle that too). Yeah, I worried about that, too :-(. Here's a proposal that might actually work, though: The provider resource exposes the reference to its inner instance by declaring it as one of its outputs. A SoftwareDeployment would learn to accept a list of Nova servers, too. Provider template: resources: my_server: type: OS::Nova::Server properties: ... ... (some other resource hidden in the provider template) outputs: inner_server: value: {get_resource: my_server} ip_address: value: {get_attr: [controller_server, networks, private, 0]} Based on my limited testing, this already makes it possible to use the inner server with a SoftwareDeployment from another template that uses my_server as a provider resource. E.g.: a_cluster_of_my_servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: custom::my_server ... some_deploy: type: OS::Heat::StructuredDeployment properties: server: {get_attr: [a_cluster_of_my_servers, resource.0.inner_server]} config: {get_resource: some_config} So what if we allowed SoftwareDeployment to accept a list of servers in addition to accepting just one server? Or add another resource that does that. I approve of that in principle. Only Steve Baker can tell us for sure if there are any technical roadblocks in the way of that, but I don't see any. Maybe if we had a new resource type that was internally implemented as a nested stack... that might give us a way of tracking the individual deployment statuses for free. cheers, Zane. Then we could do: mysql_cluster_deployment: type: OS::Heat::StructuredDeployment properties: server_list: {get_attr: [a_cluster_of_my_servers, inner_server]} config: {get_resource: mysql_cluster_config} input_values: cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers, ip_address]} This isn't that different from having a SoftwareDeployment accepting a single server and doesn't have any of the problems of allowing a ResourceGroup as a SoftwareDeployment target. What do you think? All the other solutions I can think of will result in circular issues. I'll start looking at a spec to create a resource
Re: [openstack-dev] [Heat] Passing a list of ResourceGroup's attributes back to its members
On 09/08/14 11:15, Zane Bitter wrote: On 08/08/14 11:07, Tomas Sedovic wrote: On 08/08/14 00:53, Zane Bitter wrote: On 07/08/14 13:22, Tomas Sedovic wrote: Hi all, I have a ResourceGroup which wraps a custom resource defined in another template: servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: my_custom_server properties: prop_1: ... prop_2: ... ... And a corresponding provider template and environment file. Now I can get say the list of IP addresses or any custom value of each server from the ResourceGroup by using `{get_attr: [servers, ip_address]}` and outputs defined in the provider template. But I can't figure out how to pass that list back to each server in the group. This is something we use in TripleO for things like building a MySQL cluster, where each node in the cluster (the ResourceGroup) needs the addresses of all the other nodes. Yeah, this is kind of the perpetual problem with clusters. I've been hoping that DNSaaS will show up in OpenStack soon and that that will be a way to fix this issue. The other option is to have the cluster members discover each other somehow (mDNS?), but people seem loath to do that. Right now, we have the servers ungrouped in the top-level template so we can build this list manually. But if we move to ResourceGroups (or any other scaling mechanism, I think), this is no longer possible. So I believe the current solution is to abuse a Launch Config resource as a store for the data, and then later retrieve it somehow? Possibly you could do something along similar lines, but it's unclear how the 'later retrieval' part would work... presumably it would have to involve something outside of Heat closing the loop :( Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble figuring out how would that work. LaunchConfig represents an instance, right? We can't pass the list to ResourceGroup's `resource_def` section because that causes a circular dependency. And I'm not aware of a way to attach a SoftwareConfig to a ResourceGroup. SoftwareDeployment only allows attaching a config to a single server. Yeah, and that would be a tricky thing to implement well, because a resource group may not be a group of servers (but in many cases it may be a group of nested stacks that each contain one or more servers, and you'd want to be able to handle that too). Yeah, I worried about that, too :-(. Here's a proposal that might actually work, though: The provider resource exposes the reference to its inner instance by declaring it as one of its outputs. A SoftwareDeployment would learn to accept a list of Nova servers, too. Provider template: resources: my_server: type: OS::Nova::Server properties: ... ... (some other resource hidden in the provider template) outputs: inner_server: value: {get_resource: my_server} ip_address: value: {get_attr: [controller_server, networks, private, 0]} Based on my limited testing, this already makes it possible to use the inner server with a SoftwareDeployment from another template that uses my_server as a provider resource. E.g.: a_cluster_of_my_servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: custom::my_server ... some_deploy: type: OS::Heat::StructuredDeployment properties: server: {get_attr: [a_cluster_of_my_servers, resource.0.inner_server]} config: {get_resource: some_config} So what if we allowed SoftwareDeployment to accept a list of servers in addition to accepting just one server? Or add another resource that does that. I approve of that in principle. Only Steve Baker can tell us for sure if there are any technical roadblocks in the way of that, but I don't see any. Maybe if we had a new resource type that was internally implemented as a nested stack... that might give us a way of tracking the individual deployment statuses for free. cheers, Zane. Then we could do: mysql_cluster_deployment: type: OS::Heat::StructuredDeployment properties: server_list: {get_attr: [a_cluster_of_my_servers, inner_server]} config: {get_resource: mysql_cluster_config} input_values: cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers, ip_address]} This isn't that different from having a SoftwareDeployment accepting a single server and doesn't have any of the problems of allowing a ResourceGroup as a SoftwareDeployment target. What do you think? All the other solutions I can think of will result in circular issues. I'll start looking at a spec to create a resource which is capable of deploying a config to
Re: [openstack-dev] [Heat] Passing a list of ResourceGroup's attributes back to its members
On 08/08/14 11:07, Tomas Sedovic wrote: On 08/08/14 00:53, Zane Bitter wrote: On 07/08/14 13:22, Tomas Sedovic wrote: Hi all, I have a ResourceGroup which wraps a custom resource defined in another template: servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: my_custom_server properties: prop_1: ... prop_2: ... ... And a corresponding provider template and environment file. Now I can get say the list of IP addresses or any custom value of each server from the ResourceGroup by using `{get_attr: [servers, ip_address]}` and outputs defined in the provider template. But I can't figure out how to pass that list back to each server in the group. This is something we use in TripleO for things like building a MySQL cluster, where each node in the cluster (the ResourceGroup) needs the addresses of all the other nodes. Yeah, this is kind of the perpetual problem with clusters. I've been hoping that DNSaaS will show up in OpenStack soon and that that will be a way to fix this issue. The other option is to have the cluster members discover each other somehow (mDNS?), but people seem loath to do that. Right now, we have the servers ungrouped in the top-level template so we can build this list manually. But if we move to ResourceGroups (or any other scaling mechanism, I think), this is no longer possible. So I believe the current solution is to abuse a Launch Config resource as a store for the data, and then later retrieve it somehow? Possibly you could do something along similar lines, but it's unclear how the 'later retrieval' part would work... presumably it would have to involve something outside of Heat closing the loop :( Do you mean AWS::AutoScaling::LaunchConfiguration? I'm having trouble figuring out how would that work. LaunchConfig represents an instance, right? We can't pass the list to ResourceGroup's `resource_def` section because that causes a circular dependency. And I'm not aware of a way to attach a SoftwareConfig to a ResourceGroup. SoftwareDeployment only allows attaching a config to a single server. Yeah, and that would be a tricky thing to implement well, because a resource group may not be a group of servers (but in many cases it may be a group of nested stacks that each contain one or more servers, and you'd want to be able to handle that too). Yeah, I worried about that, too :-(. Here's a proposal that might actually work, though: The provider resource exposes the reference to its inner instance by declaring it as one of its outputs. A SoftwareDeployment would learn to accept a list of Nova servers, too. Provider template: resources: my_server: type: OS::Nova::Server properties: ... ... (some other resource hidden in the provider template) outputs: inner_server: value: {get_resource: my_server} ip_address: value: {get_attr: [controller_server, networks, private, 0]} Based on my limited testing, this already makes it possible to use the inner server with a SoftwareDeployment from another template that uses my_server as a provider resource. E.g.: a_cluster_of_my_servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: custom::my_server ... some_deploy: type: OS::Heat::StructuredDeployment properties: server: {get_attr: [a_cluster_of_my_servers, resource.0.inner_server]} config: {get_resource: some_config} So what if we allowed SoftwareDeployment to accept a list of servers in addition to accepting just one server? Or add another resource that does that. I approve of that in principle. Only Steve Baker can tell us for sure if there are any technical roadblocks in the way of that, but I don't see any. Maybe if we had a new resource type that was internally implemented as a nested stack... that might give us a way of tracking the individual deployment statuses for free. cheers, Zane. Then we could do: mysql_cluster_deployment: type: OS::Heat::StructuredDeployment properties: server_list: {get_attr: [a_cluster_of_my_servers, inner_server]} config: {get_resource: mysql_cluster_config} input_values: cluster_ip_addresses: {get_attr: [a_cluster_of_my_servers, ip_address]} This isn't that different from having a SoftwareDeployment accepting a single server and doesn't have any of the problems of allowing a ResourceGroup as a SoftwareDeployment target. What do you think? Is there a way to do this that I'm missing? And if there isn't, is this something we could add to Heat? E.g. extending a SoftwareDeployment to accept ResourceGroups or adding another resource for that purpose. Thanks, Tomas ___ OpenStack-dev
Re: [openstack-dev] [Heat] Passing a list of ResourceGroup's attributes back to its members
On 07/08/14 13:22, Tomas Sedovic wrote: Hi all, I have a ResourceGroup which wraps a custom resource defined in another template: servers: type: OS::Heat::ResourceGroup properties: count: 10 resource_def: type: my_custom_server properties: prop_1: ... prop_2: ... ... And a corresponding provider template and environment file. Now I can get say the list of IP addresses or any custom value of each server from the ResourceGroup by using `{get_attr: [servers, ip_address]}` and outputs defined in the provider template. But I can't figure out how to pass that list back to each server in the group. This is something we use in TripleO for things like building a MySQL cluster, where each node in the cluster (the ResourceGroup) needs the addresses of all the other nodes. Yeah, this is kind of the perpetual problem with clusters. I've been hoping that DNSaaS will show up in OpenStack soon and that that will be a way to fix this issue. The other option is to have the cluster members discover each other somehow (mDNS?), but people seem loath to do that. Right now, we have the servers ungrouped in the top-level template so we can build this list manually. But if we move to ResourceGroups (or any other scaling mechanism, I think), this is no longer possible. So I believe the current solution is to abuse a Launch Config resource as a store for the data, and then later retrieve it somehow? Possibly you could do something along similar lines, but it's unclear how the 'later retrieval' part would work... presumably it would have to involve something outside of Heat closing the loop :( We can't pass the list to ResourceGroup's `resource_def` section because that causes a circular dependency. And I'm not aware of a way to attach a SoftwareConfig to a ResourceGroup. SoftwareDeployment only allows attaching a config to a single server. Yeah, and that would be a tricky thing to implement well, because a resource group may not be a group of servers (but in many cases it may be a group of nested stacks that each contain one or more servers, and you'd want to be able to handle that too). Is there a way to do this that I'm missing? And if there isn't, is this something we could add to Heat? E.g. extending a SoftwareDeployment to accept ResourceGroups or adding another resource for that purpose. Thanks, Tomas ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev