Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Tue, 2018-04-24 at 16:12 -0400, Zane Bitter wrote: > On 19/04/18 08:59, Harald Jensås wrote: > > The problem is getting there using heat ... > > The real answer is to make everything explicit - create a Subnet > resource and a Port resource and don't allow Neutron/Nova to make > any > decisions for you that would have the effect of hiding data that you > need. However, since that's impractical in this particular case... > Yeah, I wish the ctlplane network in tripleo was defined in THT. But since it's created by undercloud installer we are where we are. Moving it is impractical for the same reasons migrating from server resources with implicit ports is ... Another non tripleo use case is when connecting the instance to a provider network, in this case the network and subnet resource is beyond the user control. (External resource probably, but there seem to be the issues Zane mentions below.) > > a couple of ideas: > > > > a) Use heat's ``external_resource`` to create a port resource, > > and then a external subnet resource. Then get the data > > from the external resources. We probably would have to make > > it possible for a ``external_resource`` depend on the server > > resource, and verify that these resource have the required > > attributes. > > Yeah, I don't know why we don't allow depends_on for resources with > external_id. (There's also a bug where we don't recognise > dependencies > contributed by any functions used in the external_id field, like > get_resource or get_attr, even though we allow those functions.) > Apparently somebody had a brain explosion at a design summit session > that nobody remembers attending, and here we are :D > > The difficulty is that the fix should be tied to a template version, > but > the offending check is in the template-independent part of the code > base. > > Nevertheless, a workaround is trivial: > >ext_port: > type: OS::Neutron::Port > external_id: {get_attr: [, addresses, , 0, > port]} > metadata: >do_something_to_add_a_dependency: {get_resource: } > > > b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as > > well probably) to include the data. > > > > If we do this we should probably aim to be in parity with > > what is made available to clients getting the configuration > > from dhcp. (mtu, dns_domain, dns_servers, prefixlen, > > gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode > > etc.) > > This makes sense to me. If we're allowing people to let Nova/Neutron > make implicit choices for them then we also need to allow them to > see > the result. > I like this idea best as well. I will open an rfe against Heat. > > c) Create a new heat function to read properties of any > > openstack resource, without having to make use of the > > external_resource in heat. > > I'm pretty -1 on this, because I think you want to have the same > caching > behaviour as a resource, not a function. At that point you're just > implementing syntactic sugar that makes things _less_ consistent, not > to > mention the enormous implementation hacks required. > > cheers, > Zane. > > _ > _ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs > cribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On 19/04/18 08:59, Harald Jensås wrote: The problem is getting there using heat ... The real answer is to make everything explicit - create a Subnet resource and a Port resource and don't allow Neutron/Nova to make any decisions for you that would have the effect of hiding data that you need. However, since that's impractical in this particular case... a couple of ideas: a) Use heat's ``external_resource`` to create a port resource, and then a external subnet resource. Then get the data from the external resources. We probably would have to make it possible for a ``external_resource`` depend on the server resource, and verify that these resource have the required attributes. Yeah, I don't know why we don't allow depends_on for resources with external_id. (There's also a bug where we don't recognise dependencies contributed by any functions used in the external_id field, like get_resource or get_attr, even though we allow those functions.) Apparently somebody had a brain explosion at a design summit session that nobody remembers attending, and here we are :D The difficulty is that the fix should be tied to a template version, but the offending check is in the template-independent part of the code base. Nevertheless, a workaround is trivial: ext_port: type: OS::Neutron::Port external_id: {get_attr: [, addresses, , 0, port]} metadata: do_something_to_add_a_dependency: {get_resource: } b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as well probably) to include the data. If we do this we should probably aim to be in parity with what is made available to clients getting the configuration from dhcp. (mtu, dns_domain, dns_servers, prefixlen, gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode etc.) This makes sense to me. If we're allowing people to let Nova/Neutron make implicit choices for them then we also need to allow them to see the result. c) Create a new heat function to read properties of any openstack resource, without having to make use of the external_resource in heat. I'm pretty -1 on this, because I think you want to have the same caching behaviour as a resource, not a function. At that point you're just implementing syntactic sugar that makes things _less_ consistent, not to mention the enormous implementation hacks required. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On 23/04/18 14:09, Dan Sneddon wrote: Yes, the port is currently created as part of the Ironic server resource. We would have more flexibility if this were a separate Neutron port, but we need to be able to support upgrades. This would require the ability in Heat to detach the implicit port from the Ironic resource, and attach a Neutron port resource with the same IP to a node without rebuilding the entire node. This isn't currently possible. I believe it's possible using a two-step migration. First create an OS::Neutron::Port resource with external_id as the current port ID. Then use get_resource to pass the ID of the Port explicitly to the Server in its network config. On update, Heat will recognise this as an unchanged config thanks to the fixes that Harald made in Queens. Second, do another update removing the external_id to allow Heat to manage this port (or don't, I guess, since Nova will clean up the port when the server is deleted regardless). This process is pretty horrible though, and more suited to a manual fix-up than something like TripleO. I can see another use case for this Heat functionality, which is that I would like to be able to generate a report using Heat that lists all the ports in use in the entire deployment. This would be generated post-deployment, and could be used to populate an external DNS server, or simply to report on which IPs belong to which nodes. You can get IPs from the servers already. (Also, you should use Designate resources to populate your external DNS server ;) The issue here AIUI is that you can't get info from the subnet, like the CIDR, and in fact you may not even know the subnet because of the magical way Neutron will implicitly allocate stuff for you. cheers, Zane. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Mon, Apr 23, 2018 at 8:09 PM, Dan Sneddonwrote: > We could add the ControlPlaneDefaultRoute and ControlPlaneSubnetCidr to > network_data.yaml, but this would involve some duplication of configuration > data, since those are currently defined in undercloud.conf. A more robust > solution might be to generate network_data.yaml from that info in > undercloud.conf, but currently we don't modify any files in the > tripleo-heat-templates package after it gets installed. Right, it seems getting those values from Neutron is better. > I can't speak to the roadmap of Heat/Neutron/Nova on the undercloud, for the > immediate future I don't see us moving away from Heat entirely due to > upgrade requirements. > > I can see another use case for this Heat functionality, which is that I > would like to be able to generate a report using Heat that lists all the > ports in use in the entire deployment. This would be generated > post-deployment, and could be used to populate an external DNS server, or > simply to report on which IPs belong to which nodes. Jiri wrote a small tool that does mostly that: https://gist.github.com/jistr/ad385d77db7600c18e8d52652358b616 We could make that more official, but we already have the info. -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Mon, Apr 23, 2018 at 7:16 PM, Harald Jensåswrote: > On Fri, 2018-04-20 at 14:44 +0200, Thomas Herve wrote: >> To make it clear, what you want to prevent is the need to add more >> keys in network_data.yaml? >> >> As those had to be provided at some point, I wonder if tripleo can't >> find a way to pass them again on the overcloud deploy. >> > No, the networks defined in network_data.yaml is fine, that is the data > used to create the neutron stuff so passing the data from there is > already in place to some extent. > > But, the ctlplane network is not defined in network_data.yaml. OK. >> If you can list the attributes you care about that'd be great. >> > > Guess what I envision is a client_config attribute, a map with data > useful to configure a network interface on the client. (I put * on the > ones I believe could be useful for TripleO) > > * /v2.0/networks/{network_id}/mtu > /v2.0/networks/{network_id}/dns_domain > * /v2.0/subnets/{subnet_id}/dns_nameservers > * /v2.0/subnets/{subnet_id}/host_routes > /v2.0/subnets/{subnet_id}/ip_version > * /v2.0/subnets/{subnet_id}/gateway_ip > * /v2.0/subnets/{subnet_id}/cidr > * /v2.0/subnets/{subnet_id}/ipv6_address_mode > * /v2.0/subnets/{subnet_id}/ipv6_ra_mode > /v2.0/ports/{port_id}/description - Why not? > /v2.0/ports/{port_id}/dns_assignment > /v2.0/ports/{port_id}/dns_domain > /v2.0/ports/{port_id}/dns_name > * /v2.0/ports/{port_id}/fixed_ips - We have this already > /v2.0/ports/{port_id}/name- Why not? I think we have most of those on resources already. From the required ones, I think the only ones mising are ipv6_address_mode and ipv6_ra_mode on subnets. If we make external resources work, it'll be easy to provide what you need. -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Mon, Apr 23, 2018 at 10:16 AM, Harald Jensåswrote: > On Fri, 2018-04-20 at 14:44 +0200, Thomas Herve wrote: > > On Thu, Apr 19, 2018 at 2:59 PM, Harald Jensås > > wrote: > > > Hi, > > > > Hi, thanks for sending this. Responses inline. > > > > > When configuring TripleO deployments with nodes on routed ctlplane > > > networks we need to pass some per-network properties to the > > > NetworkConfig resource[1] in THT. We get the ``ControlPlaneIp`` > > > property using get_attr, but the NIC configs need a couple of more > > > parameters[2], for example: ``ControlPlaneSubnetCidr``, > > > ``ControlPlaneDefaultRoute`` and ``DnsServers``. > > > > > > Since queens these templates are jinja templated, to generate > > > things > > > from from network_data.yaml. When using routed ctlplane networks, > > > the > > > parameters ``ControlPlaneSubnetCidr`` and > > > ``ControlPlaneDefaultRoute`` > > > will be different. So we need to use static per-role > > > Net::SoftwareConfig templates, and add parameters such as > > > ``ControlPlaneDefaultRouteLeafX``. > > > > > > The values the use need to pass in for these are already available > > > in > > > the neutron ctlplane network configuration on the undercloud. So > > > ideally we should not need to ask the user to provide them in > > > parameter_defaults, we should resolve the correct values > > > automatically. > > > > To make it clear, what you want to prevent is the need to add more > > keys in network_data.yaml? > > > > As those had to be provided at some point, I wonder if tripleo can't > > find a way to pass them again on the overcloud deploy. > > > No, the networks defined in network_data.yaml is fine, that is the data > used to create the neutron stuff so passing the data from there is > already in place to some extent. > > But, the ctlplane network is not defined in network_data.yaml. > We could add the ControlPlaneDefaultRoute and ControlPlaneSubnetCidr to network_data.yaml, but this would involve some duplication of configuration data, since those are currently defined in undercloud.conf. A more robust solution might be to generate network_data.yaml from that info in undercloud.conf, but currently we don't modify any files in the tripleo-heat-templates package after it gets installed. > > > Inspecting neutron is an elegant solution, though. > > > > > > > : We can get the port ID using get_attr: > > > > > > {get_attr: [, addresses, , 0, port]} > > > > > > : From there outside of heat we can get the subnet_id: > > > > > > openstack port show 2fb4baf9-45b0-48cb-8249-c09a535b9eda \ > > > -f yaml -c fixed_ips > > > > > > fixed_ips: ip_address='172.20.0.10', subnet_id='2b06ae2e-423f- > > > 4a73- > > > 97ad-4e9822d201e5' > > > > > > : And finally we can get the gateway_ip and cidr of the subnet: > > > > > > openstack subnet show 2b06ae2e-423f-4a73-97ad-4e9822d201e5 \ > > > -f yaml -c gateway_ip -c cidr > > > > > > cidr: 172.20.0.0/26 > > > gateway_ip: 172.20.0.62 > > > > > > > > > The problem is getting there using heat ... > > > a couple of ideas: > > > > > > a) Use heat's ``external_resource`` to create a port resource, > > >and then a external subnet resource. Then get the data > > >from the external resources. We probably would have to make > > >it possible for a ``external_resource`` depend on the server > > >resource, and verify that these resource have the required > > >attributes. > > > > I believe that's a relatively easy fix. It's unclear why we didn't > > allow that in the first place, probably because we were missing a use > > case, but it seems valuable here. > > > > > b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as > > >well probably) to include the data. > > > > > >If we do this we should probably aim to be in parity with > > >what is made available to clients getting the configuration > > >from dhcp. (mtu, dns_domain, dns_servers, prefixlen, > > >gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode > > >etc.) > > > > I'm with you on exposing more neutron data to the Port resource. It > > can be complicated because some of them are implementation specific, > > but we can look into those. > > > > I don't think adding them directly to the Server resource makes a ton > > of sense though. > > > In tripleo, the ctlplane interface is an implicit port created by the > server resource. :( (Attempts where made to change this, but upgrades > would'nt work) So the server resource is where I would find it most > useful. (Adding attributes to the port resource, and then using > external resource for the implicit server ports may be a compromise. > Nested dependencies for external_resources might be hard?) > Yes, the port is currently created as part of the Ironic server resource. We would have more flexibility if this were a separate Neutron port, but we need to be able to support upgrades. This would require the ability in Heat to detach
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Fri, 2018-04-20 at 14:44 +0200, Thomas Herve wrote: > On Thu, Apr 19, 2018 at 2:59 PM, Harald Jensås> wrote: > > Hi, > > Hi, thanks for sending this. Responses inline. > > > When configuring TripleO deployments with nodes on routed ctlplane > > networks we need to pass some per-network properties to the > > NetworkConfig resource[1] in THT. We get the ``ControlPlaneIp`` > > property using get_attr, but the NIC configs need a couple of more > > parameters[2], for example: ``ControlPlaneSubnetCidr``, > > ``ControlPlaneDefaultRoute`` and ``DnsServers``. > > > > Since queens these templates are jinja templated, to generate > > things > > from from network_data.yaml. When using routed ctlplane networks, > > the > > parameters ``ControlPlaneSubnetCidr`` and > > ``ControlPlaneDefaultRoute`` > > will be different. So we need to use static per-role > > Net::SoftwareConfig templates, and add parameters such as > > ``ControlPlaneDefaultRouteLeafX``. > > > > The values the use need to pass in for these are already available > > in > > the neutron ctlplane network configuration on the undercloud. So > > ideally we should not need to ask the user to provide them in > > parameter_defaults, we should resolve the correct values > > automatically. > > To make it clear, what you want to prevent is the need to add more > keys in network_data.yaml? > > As those had to be provided at some point, I wonder if tripleo can't > find a way to pass them again on the overcloud deploy. > No, the networks defined in network_data.yaml is fine, that is the data used to create the neutron stuff so passing the data from there is already in place to some extent. But, the ctlplane network is not defined in network_data.yaml. > Inspecting neutron is an elegant solution, though. > > > : We can get the port ID using get_attr: > > > > {get_attr: [, addresses, , 0, port]} > > > > : From there outside of heat we can get the subnet_id: > > > > openstack port show 2fb4baf9-45b0-48cb-8249-c09a535b9eda \ > > -f yaml -c fixed_ips > > > > fixed_ips: ip_address='172.20.0.10', subnet_id='2b06ae2e-423f- > > 4a73- > > 97ad-4e9822d201e5' > > > > : And finally we can get the gateway_ip and cidr of the subnet: > > > > openstack subnet show 2b06ae2e-423f-4a73-97ad-4e9822d201e5 \ > > -f yaml -c gateway_ip -c cidr > > > > cidr: 172.20.0.0/26 > > gateway_ip: 172.20.0.62 > > > > > > The problem is getting there using heat ... > > a couple of ideas: > > > > a) Use heat's ``external_resource`` to create a port resource, > >and then a external subnet resource. Then get the data > >from the external resources. We probably would have to make > >it possible for a ``external_resource`` depend on the server > >resource, and verify that these resource have the required > >attributes. > > I believe that's a relatively easy fix. It's unclear why we didn't > allow that in the first place, probably because we were missing a use > case, but it seems valuable here. > > > b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as > >well probably) to include the data. > > > >If we do this we should probably aim to be in parity with > >what is made available to clients getting the configuration > >from dhcp. (mtu, dns_domain, dns_servers, prefixlen, > >gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode > >etc.) > > I'm with you on exposing more neutron data to the Port resource. It > can be complicated because some of them are implementation specific, > but we can look into those. > > I don't think adding them directly to the Server resource makes a ton > of sense though. > In tripleo, the ctlplane interface is an implicit port created by the server resource. :( (Attempts where made to change this, but upgrades would'nt work) So the server resource is where I would find it most useful. (Adding attributes to the port resource, and then using external resource for the implicit server ports may be a compromise. Nested dependencies for external_resources might be hard?) > > c) Create a new heat function to read properties of any > >openstack resource, without having to make use of the > >external_resource in heat. > > It's an interesting idea, but I think it would look a lot like what > external resources are supposed to be. > > I see a few changes: > * Allow external resource to depend on other resources > * Expose more port attributes > * Expose more subnet attributes > > If you can list the attributes you care about that'd be great. > Guess what I envision is a client_config attribute, a map with data useful to configure a network interface on the client. (I put * on the ones I believe could be useful for TripleO) * /v2.0/networks/{network_id}/mtu /v2.0/networks/{network_id}/dns_domain * /v2.0/subnets/{subnet_id}/dns_nameservers * /v2.0/subnets/{subnet_id}/host_routes /v2.0/subnets/{subnet_id}/ip_version *
Re: [openstack-dev] [Heat][TripleO] - Getting attributes of openstack resources not created by the stack for TripleO NetworkConfig.
On Thu, Apr 19, 2018 at 2:59 PM, Harald Jensåswrote: > Hi, Hi, thanks for sending this. Responses inline. > When configuring TripleO deployments with nodes on routed ctlplane > networks we need to pass some per-network properties to the > NetworkConfig resource[1] in THT. We get the ``ControlPlaneIp`` > property using get_attr, but the NIC configs need a couple of more > parameters[2], for example: ``ControlPlaneSubnetCidr``, > ``ControlPlaneDefaultRoute`` and ``DnsServers``. > > Since queens these templates are jinja templated, to generate things > from from network_data.yaml. When using routed ctlplane networks, the > parameters ``ControlPlaneSubnetCidr`` and ``ControlPlaneDefaultRoute`` > will be different. So we need to use static per-role > Net::SoftwareConfig templates, and add parameters such as > ``ControlPlaneDefaultRouteLeafX``. > > The values the use need to pass in for these are already available in > the neutron ctlplane network configuration on the undercloud. So > ideally we should not need to ask the user to provide them in > parameter_defaults, we should resolve the correct values automatically. To make it clear, what you want to prevent is the need to add more keys in network_data.yaml? As those had to be provided at some point, I wonder if tripleo can't find a way to pass them again on the overcloud deploy. Inspecting neutron is an elegant solution, though. > : We can get the port ID using get_attr: > > {get_attr: [, addresses, , 0, port]} > > : From there outside of heat we can get the subnet_id: > > openstack port show 2fb4baf9-45b0-48cb-8249-c09a535b9eda \ > -f yaml -c fixed_ips > > fixed_ips: ip_address='172.20.0.10', subnet_id='2b06ae2e-423f-4a73- > 97ad-4e9822d201e5' > > : And finally we can get the gateway_ip and cidr of the subnet: > > openstack subnet show 2b06ae2e-423f-4a73-97ad-4e9822d201e5 \ > -f yaml -c gateway_ip -c cidr > > cidr: 172.20.0.0/26 > gateway_ip: 172.20.0.62 > > > The problem is getting there using heat ... > a couple of ideas: > > a) Use heat's ``external_resource`` to create a port resource, >and then a external subnet resource. Then get the data >from the external resources. We probably would have to make >it possible for a ``external_resource`` depend on the server >resource, and verify that these resource have the required >attributes. I believe that's a relatively easy fix. It's unclear why we didn't allow that in the first place, probably because we were missing a use case, but it seems valuable here. > b) Extend attributes of OS::Nova::Server (OS::Neutron::Port as >well probably) to include the data. > >If we do this we should probably aim to be in parity with >what is made available to clients getting the configuration >from dhcp. (mtu, dns_domain, dns_servers, prefixlen, >gateway_ip, host_routes, ipv6_address_mode, ipv6_ra_mode >etc.) I'm with you on exposing more neutron data to the Port resource. It can be complicated because some of them are implementation specific, but we can look into those. I don't think adding them directly to the Server resource makes a ton of sense though. > c) Create a new heat function to read properties of any >openstack resource, without having to make use of the >external_resource in heat. It's an interesting idea, but I think it would look a lot like what external resources are supposed to be. I see a few changes: * Allow external resource to depend on other resources * Expose more port attributes * Expose more subnet attributes If you can list the attributes you care about that'd be great. Thanks, -- Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev