Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Kevin Benton]

We chatted a bit about this on IRC. The issue here is that subnets that
belong to router:external networks are not visible unless the network is
shared as well.
So the only way the users can learn about which subnets to pick are in the
list of subnet UUIDs in the json body of the network 'subnets' field.[1]
They would basically be picking blindly because they don't know any details
about the subnets on the external network.

I think for us to allow a reasonable workflow where they pick an external
subnet instead, we would need to revisit the decision to hide external
subnets from users.


1.
https://github.com/openstack/neutron-lib/blob/ca299b8e47fdd5030dda596fd779beb3e5bea6cf/neutron_lib/api/definitions/network.py#L43

On Fri, May 19, 2017 at 3:27 PM, Armando M.  wrote:

>
>
> On 19 May 2017 at 14:54, Clark Boylan  wrote:
>
>> On Fri, May 19, 2017, at 02:03 PM, Kevin Benton wrote:
>> > I split this conversation off of the "Is the pendulum swinging on PaaS
>> > layers?" thread [1] to discuss some improvements we can make to Neutron
>> > to
>> > make orchestration easier.
>> >
>> > There are some pain points that heat has when working with the Neutron
>> > API.
>> > I would like to get them converted into requests for enhancements in
>> > Neutron so the wider community is aware of them.
>> >
>> > Starting with the port/subnet/network relationship - it's important to
>> > understand that IP addresses are not required on a port.
>> >
>> > >So knowing now that a Network is a layer-2 network segment and a Subnet
>> > is... effectively a glorified DHCP address pool
>> >
>> > Yes, a Subnet controls IP address allocation as well as setting up
>> > routing
>> > for routers, which is why routers reference subnets instead of networks
>> > (different routers can route for different subnets on the same network).
>> > It
>> > essentially dictates things related to L3 addressing and provides
>> > information for L3 reachability.
>>
>> One thing that is odd about this is when creating a router you specify
>> the gateway information using a network which is l2 not l3. Seems like
>> it would be more correct to use a subnet rather than a network there?
>>
>
> I think this is due the way external networks ended up being modeled in
> neutron. I suppose we could have allowed the user to specify a subnet, so
> long that it fell in the bucket of subnets that belong to a router:external
> network.
>
>
>>
>> Clark
>>
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib
>> e
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Zane Bitter]

On 19/05/17 19:53, Kevin Benton wrote:

So making a subnet ID mandatory for a port creation and
a RouterInterface ID mandatory for a Floating IP creation are both
possible in Heat without Neutron changes. Presumably you haven't done
that because it's backwards-incompatible, but you would need to
implement the change anyway if the Neutron API was changed to require it.

It seems like Heat has a backwards-compatibility requirement for
supporting old templates that aren't explicit. That will be the real
blocker to actually making any of these changes, no? i.e. Neutron isn't
preventing Heat from being more strict, it's the legacy Heat modeling
that is preventing it.


We have a translation mechanism for resource properties (much improved 
in Pike - thanks prazumovsky!) that could in theory help us to make such 
a change (with or without a corresponding change in the Neutron API) 
without breaking existing users (although it would probably require a 
bunch of expensive API calls at inopportune times). That would likely be 
just as much of a pain to maintain as the workarounds we have now, so 
tbh we're likely to stick with reflecting the Neutron API directly, 
whatever it does.


I've long since chalked this one up to 'lessons learned'; if I keep 
harping on it, it's because I want to make sure that everyone really 
does learn the lessons.



(a) drop the requirement that the Network has to be connected to the

external network with the FloatingIPs with a RouterInterface prior to
creating the FloatingIP. IIUC only *some* Neutron backends require this.

This can produce difficult to debug situations when multiple routers
attached to different external networks are attached to different
subnets of the same network and the user associates a floating IP to the
wrong fixed IP of the instance. Right now the interface check will
prevent that, but if we remove it the floating IP would just sit in the
DOWN state.

If a backend supports floating IPs without router interfaces entirely,
it's likely making assumptions that prevent it from supporting
multi-router scenarios. A single fixed IP on a port can have multiple
floating IPs associated with it from different external networks. So the
only way to distinguish which floating IP to translate to is which
router the traffic is being directed to by the instance, which requires
router interfaces.

Cheers

On Fri, May 19, 2017 at 3:29 PM, Zane Bitter > wrote:

On 19/05/17 17:03, Kevin Benton wrote:

I split this conversation off of the "Is the pendulum swinging
on PaaS
layers?" thread [1] to discuss some improvements we can make to
Neutron
to make orchestration easier.

There are some pain points that heat has when working with the
Neutron
API. I would like to get them converted into requests for
enhancements
in Neutron so the wider community is aware of them.

Starting with the port/subnet/network relationship - it's
important to
understand that IP addresses are not required on a port.

So knowing now that a Network is a layer-2 network segment
and a Subnet

is... effectively a glorified DHCP address pool

Yes, a Subnet controls IP address allocation as well as setting up
routing for routers, which is why routers reference subnets
instead of
networks (different routers can route for different subnets on
the same
network). It essentially dictates things related to L3
addressing and
provides information for L3 reachability.

But at the end of the day, I still can't create a Port until
a Subnet exists


This is only true if you want an IP address on the port. This sounds
silly for most use cases, but there are a non-trivial portion of NFV
workloads that do not want IP addresses at all so they create a
network
and just attach ports without creating any subnets.


Fair. A more precise statement of the problem would be that given a
template containing both a Port and a Subnet that it will be
attached to, there is a specific order in which those need to be
created that is _not_ reflected in the data flow between them.

I still don't know what Subnet a Port will be attached to
(unless the

user specifies it explicitly using the --fixed-ip option...
regardless
of whether they actually specify a fixed IP),

So what would you like Neutron to do differently here? Always
force a
user to pick which subnet they want an allocation from


That would work.

if there are
multiple?


Ideally even if there aren't.

If so, can't you just force that explicitness in Heat?


I think the answer here is exactly the same as for Neutron: yes, we
totally could have if 

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Kevin Benton]

So making a subnet ID mandatory for a port creation and a RouterInterface
ID mandatory for a Floating IP creation are both possible in Heat without
Neutron changes. Presumably you haven't done that because it's
backwards-incompatible, but you would need to implement the change anyway
if the Neutron API was changed to require it.

It seems like Heat has a backwards-compatibility requirement for supporting
old templates that aren't explicit. That will be the real blocker to
actually making any of these changes, no? i.e. Neutron isn't preventing
Heat from being more strict, it's the legacy Heat modeling that is
preventing it.

>(a) drop the requirement that the Network has to be connected to the
external network with the FloatingIPs with a RouterInterface prior to
creating the FloatingIP. IIUC only *some* Neutron backends require this.

This can produce difficult to debug situations when multiple routers
attached to different external networks are attached to different subnets
of the same network and the user associates a floating IP to the wrong
fixed IP of the instance. Right now the interface check will prevent that,
but if we remove it the floating IP would just sit in the DOWN state.

If a backend supports floating IPs without router interfaces entirely, it's
likely making assumptions that prevent it from supporting multi-router
scenarios. A single fixed IP on a port can have multiple floating IPs
associated with it from different external networks. So the only way to
distinguish which floating IP to translate to is which router the traffic
is being directed to by the instance, which requires router interfaces.

Cheers

On Fri, May 19, 2017 at 3:29 PM, Zane Bitter  wrote:

> On 19/05/17 17:03, Kevin Benton wrote:
>
>> I split this conversation off of the "Is the pendulum swinging on PaaS
>> layers?" thread [1] to discuss some improvements we can make to Neutron
>> to make orchestration easier.
>>
>> There are some pain points that heat has when working with the Neutron
>> API. I would like to get them converted into requests for enhancements
>> in Neutron so the wider community is aware of them.
>>
>> Starting with the port/subnet/network relationship - it's important to
>> understand that IP addresses are not required on a port.
>>
>> So knowing now that a Network is a layer-2 network segment and a Subnet
>>>
>> is... effectively a glorified DHCP address pool
>>
>> Yes, a Subnet controls IP address allocation as well as setting up
>> routing for routers, which is why routers reference subnets instead of
>> networks (different routers can route for different subnets on the same
>> network). It essentially dictates things related to L3 addressing and
>> provides information for L3 reachability.
>>
>> But at the end of the day, I still can't create a Port until a Subnet
>>> exists
>>>
>>
>> This is only true if you want an IP address on the port. This sounds
>> silly for most use cases, but there are a non-trivial portion of NFV
>> workloads that do not want IP addresses at all so they create a network
>> and just attach ports without creating any subnets.
>>
>
> Fair. A more precise statement of the problem would be that given a
> template containing both a Port and a Subnet that it will be attached to,
> there is a specific order in which those need to be created that is _not_
> reflected in the data flow between them.
>
> I still don't know what Subnet a Port will be attached to (unless the
>>>
>> user specifies it explicitly using the --fixed-ip option... regardless
>> of whether they actually specify a fixed IP),
>>
>> So what would you like Neutron to do differently here? Always force a
>> user to pick which subnet they want an allocation from
>>
>
> That would work.
>
> if there are
>> multiple?
>>
>
> Ideally even if there aren't.
>
> If so, can't you just force that explicitness in Heat?
>>
>
> I think the answer here is exactly the same as for Neutron: yes, we
> totally could have if we'd realised that it was a problem at the time.
>
> and I have no way in general of telling which Subnets can be deleted
>>> before a given Port is and which will fail to delete until the Port
>>> disappears.
>>>
>>
>> A given port will only block subnet deletions from subnets it is
>> attached to. Conversely, you can see all ports with allocations from a
>> subnet with 'neutron port-list --fixed-ips subnet_id='.  So
>> is the issue here that the dependency wasn't made explicit in the heat
>> modeling (leading to the problem above and this one)?
>>
>
> Yes, that's exactly the issue. The Heat modelling was based on 1:1 with
> the Neutron API to minimise user confusion.
>
> For the individual bugs you highlighted, it would be good if you can
>> provide some details about what changes we could make to help.
>>
>>
>> https://bugs.launchpad.net/heat/+bug/1442121 - This looks like a result
>> of partially specified floating IPs (no fixed_ip). What can we
>> add/change here to help? Or can heat just 

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Zane Bitter]

On 19/05/17 17:03, Kevin Benton wrote:

I split this conversation off of the "Is the pendulum swinging on PaaS
layers?" thread [1] to discuss some improvements we can make to Neutron
to make orchestration easier.

There are some pain points that heat has when working with the Neutron
API. I would like to get them converted into requests for enhancements
in Neutron so the wider community is aware of them.

Starting with the port/subnet/network relationship - it's important to
understand that IP addresses are not required on a port.


So knowing now that a Network is a layer-2 network segment and a Subnet

is... effectively a glorified DHCP address pool

Yes, a Subnet controls IP address allocation as well as setting up
routing for routers, which is why routers reference subnets instead of
networks (different routers can route for different subnets on the same
network). It essentially dictates things related to L3 addressing and
provides information for L3 reachability.


But at the end of the day, I still can't create a Port until a Subnet exists


This is only true if you want an IP address on the port. This sounds
silly for most use cases, but there are a non-trivial portion of NFV
workloads that do not want IP addresses at all so they create a network
and just attach ports without creating any subnets.


Fair. A more precise statement of the problem would be that given a 
template containing both a Port and a Subnet that it will be attached 
to, there is a specific order in which those need to be created that is 
_not_ reflected in the data flow between them.



I still don't know what Subnet a Port will be attached to (unless the

user specifies it explicitly using the --fixed-ip option... regardless
of whether they actually specify a fixed IP),

So what would you like Neutron to do differently here? Always force a
user to pick which subnet they want an allocation from


That would work.


if there are
multiple?


Ideally even if there aren't.


If so, can't you just force that explicitness in Heat?


I think the answer here is exactly the same as for Neutron: yes, we 
totally could have if we'd realised that it was a problem at the time.



and I have no way in general of telling which Subnets can be deleted before a 
given Port is and which will fail to delete until the Port disappears.


A given port will only block subnet deletions from subnets it is
attached to. Conversely, you can see all ports with allocations from a
subnet with 'neutron port-list --fixed-ips subnet_id='.  So
is the issue here that the dependency wasn't made explicit in the heat
modeling (leading to the problem above and this one)?


Yes, that's exactly the issue. The Heat modelling was based on 1:1 with 
the Neutron API to minimise user confusion.



For the individual bugs you highlighted, it would be good if you can
provide some details about what changes we could make to help.


https://bugs.launchpad.net/heat/+bug/1442121 - This looks like a result
of partially specified floating IPs (no fixed_ip). What can we
add/change here to help? Or can heat just always force the user to
specify a fixed IP for the case where disambiguation on multiple
fixed_ip ports is needed?


This is the issue from which all the others on that list were spawned 
(see https://bugs.launchpad.net/heat/+bug/1442121/comments/10), so the 
only thing we're planning to actually do for this one is to catch any 
exceptions closer to where they occur than we're doing in the fix for 
https://bugs.launchpad.net/heat/+bug/1554625



https://launchpad.net/bugs/1626607


Note that this one is fixed.


- I see this is about a dependency
between RouterGateways and RouterInterfaces, but it's not clear to me
why that dependency exists. Is it to solve a lack of visibility into the
interfaces required for a floating IP?


Yes, exactly.

We essentially solved the RouterGateway/RouterInterface half of the 
problem in Heat back in Juno, by deprecating the 
OS::Neutron::RouterGateway resource and replacing it with an 
"external_gateway_info" property in OS::Neutron::Router. Old templates 
never die though.



https://bugs.launchpad.net/heat/+bug/1626619,
https://bugs.launchpad.net/heat/+bug/1626630, and
https://bugs.launchpad.net/heat/+bug/1626634 - These seems similar to
1626607.


The first and third are the RouterInterface/FloatingIP half of the 
problem. And to work around that we also have to work around the 
Subnet/Port problem (that's the third bug). The second bug is the 
RouterGateway/RouterInterface equivalent of the third.



Can we just expose the interfaces/router a floating IP is
depending on explicitly in the API for you to fix these?


Not really. We need to know before any of them are actually created. 
Preferably without making any REST calls, because REST calls are slow 
and tend to raise exceptions at unfortunate times.



If not, what
can we do to help here?


In principle, either:

(a) drop the requirement that the Network has to be connected to the 

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Armando M.]

On 19 May 2017 at 14:54, Clark Boylan  wrote:

> On Fri, May 19, 2017, at 02:03 PM, Kevin Benton wrote:
> > I split this conversation off of the "Is the pendulum swinging on PaaS
> > layers?" thread [1] to discuss some improvements we can make to Neutron
> > to
> > make orchestration easier.
> >
> > There are some pain points that heat has when working with the Neutron
> > API.
> > I would like to get them converted into requests for enhancements in
> > Neutron so the wider community is aware of them.
> >
> > Starting with the port/subnet/network relationship - it's important to
> > understand that IP addresses are not required on a port.
> >
> > >So knowing now that a Network is a layer-2 network segment and a Subnet
> > is... effectively a glorified DHCP address pool
> >
> > Yes, a Subnet controls IP address allocation as well as setting up
> > routing
> > for routers, which is why routers reference subnets instead of networks
> > (different routers can route for different subnets on the same network).
> > It
> > essentially dictates things related to L3 addressing and provides
> > information for L3 reachability.
>
> One thing that is odd about this is when creating a router you specify
> the gateway information using a network which is l2 not l3. Seems like
> it would be more correct to use a subnet rather than a network there?
>

I think this is due the way external networks ended up being modeled in
neutron. I suppose we could have allowed the user to specify a subnet, so
long that it fell in the bucket of subnets that belong to a router:external
network.


>
> Clark
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Clark Boylan]

On Fri, May 19, 2017, at 02:03 PM, Kevin Benton wrote:
> I split this conversation off of the "Is the pendulum swinging on PaaS
> layers?" thread [1] to discuss some improvements we can make to Neutron
> to
> make orchestration easier.
> 
> There are some pain points that heat has when working with the Neutron
> API.
> I would like to get them converted into requests for enhancements in
> Neutron so the wider community is aware of them.
> 
> Starting with the port/subnet/network relationship - it's important to
> understand that IP addresses are not required on a port.
> 
> >So knowing now that a Network is a layer-2 network segment and a Subnet
> is... effectively a glorified DHCP address pool
> 
> Yes, a Subnet controls IP address allocation as well as setting up
> routing
> for routers, which is why routers reference subnets instead of networks
> (different routers can route for different subnets on the same network).
> It
> essentially dictates things related to L3 addressing and provides
> information for L3 reachability.

One thing that is odd about this is when creating a router you specify
the gateway information using a network which is l2 not l3. Seems like
it would be more correct to use a subnet rather than a network there?

Clark

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Kevin Benton]

We could potentially alter Neutron to inspect each of the fixed IPs to see
if it's eligible to be associated to that floating IP to find the one that
works. Unfortunately that still won't solve multiple eligible fixed IPs. So
is the right thing to do to just always force the user to specify port+IP?

On Fri, May 19, 2017 at 2:13 PM, Monty Taylor  wrote:

> On 05/19/2017 04:03 PM, Kevin Benton wrote:
>
>> I split this conversation off of the "Is the pendulum swinging on PaaS
>> layers?" thread [1] to discuss some improvements we can make to Neutron
>> to make orchestration easier.
>>
>> There are some pain points that heat has when working with the Neutron
>> API. I would like to get them converted into requests for enhancements
>> in Neutron so the wider community is aware of them.
>>
>> Starting with the port/subnet/network relationship - it's important to
>> understand that IP addresses are not required on a port.
>>
>> So knowing now that a Network is a layer-2 network segment and a Subnet
>>>
>> is... effectively a glorified DHCP address pool
>>
>> Yes, a Subnet controls IP address allocation as well as setting up
>> routing for routers, which is why routers reference subnets instead of
>> networks (different routers can route for different subnets on the same
>> network). It essentially dictates things related to L3 addressing and
>> provides information for L3 reachability.
>>
>> But at the end of the day, I still can't create a Port until a Subnet
>>> exists
>>>
>>
>> This is only true if you want an IP address on the port. This sounds
>> silly for most use cases, but there are a non-trivial portion of NFV
>> workloads that do not want IP addresses at all so they create a network
>> and just attach ports without creating any subnets.
>>
>> I still don't know what Subnet a Port will be attached to (unless the
>>>
>> user specifies it explicitly using the --fixed-ip option... regardless
>> of whether they actually specify a fixed IP),
>>
>> So what would you like Neutron to do differently here? Always force a
>> user to pick which subnet they want an allocation from if there are
>> multiple? If so, can't you just force that explicitness in Heat?
>>
>> and I have no way in general of telling which Subnets can be deleted
>>> before a given Port is and which will fail to delete until the Port
>>> disappears.
>>>
>>
>> A given port will only block subnet deletions from subnets it is
>> attached to. Conversely, you can see all ports with allocations from a
>> subnet with 'neutron port-list --fixed-ips subnet_id='.  So
>> is the issue here that the dependency wasn't made explicit in the heat
>> modeling (leading to the problem above and this one)?
>>
>>
>> For the individual bugs you highlighted, it would be good if you can
>> provide some details about what changes we could make to help.
>>
>>
>> https://bugs.launchpad.net/heat/+bug/1442121 - This looks like a result
>> of partially specified floating IPs (no fixed_ip). What can we
>> add/change here to help? Or can heat just always force the user to
>> specify a fixed IP for the case where disambiguation on multiple
>> fixed_ip ports is needed?
>>
>
> If the server has more than one fixed_ip ports, it's possible that only
> one of them will be able to receive a floating ip. The subnet a port comes
> from must have gateway_ip set for a floating_ip to attach to it. So if you
> have a server, you can poke and find the right fixed_ip in all cases except
> when the server has more than one fixed_ip and each of them are from a
> subnet with a gateway_ip. In that case, a user _must_ provide a fixed_ip,
> because there is no way to know what they intend.
>
> https://launchpad.net/bugs/1626607 - I see this is about a dependency
>> between RouterGateways and RouterInterfaces, but it's not clear to me
>> why that dependency exists. Is it to solve a lack of visibility into the
>> interfaces required for a floating IP?
>>
>> https://bugs.launchpad.net/heat/+bug/1626619,
>> https://bugs.launchpad.net/heat/+bug/1626630, and
>> https://bugs.launchpad.net/heat/+bug/1626634 - These seems similar to
>> 1626607. Can we just expose the interfaces/router a floating IP is
>> depending on explicitly in the API for you to fix these? If not, what
>> can we do to help here?
>>
>>
>> 1. http://lists.openstack.org/pipermail/openstack-dev/2017-May/
>> 117106.html
>>
>> Cheers,
>> Kevin Benton
>>
>> On Fri, May 19, 2017 at 1:05 PM, Zane Bitter > > wrote:
>>
>> On 19/05/17 15:06, Kevin Benton wrote:
>>
>> Don't even get me started on Neutron.[2]
>>
>>
>> It seems to me the conclusion to that thread was that the
>> majority of
>> your issues stemmed from the fact that we had poor documentation
>> at the
>> time.  A major component of the complaints resulted from you
>> misunderstanding the difference between networks/subnets in
>> Neutron.
>>
>>
>> 

Re: [openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Monty Taylor]

On 05/19/2017 04:03 PM, Kevin Benton wrote:

I split this conversation off of the "Is the pendulum swinging on PaaS
layers?" thread [1] to discuss some improvements we can make to Neutron
to make orchestration easier.

There are some pain points that heat has when working with the Neutron
API. I would like to get them converted into requests for enhancements
in Neutron so the wider community is aware of them.

Starting with the port/subnet/network relationship - it's important to
understand that IP addresses are not required on a port.


So knowing now that a Network is a layer-2 network segment and a Subnet

is... effectively a glorified DHCP address pool

Yes, a Subnet controls IP address allocation as well as setting up
routing for routers, which is why routers reference subnets instead of
networks (different routers can route for different subnets on the same
network). It essentially dictates things related to L3 addressing and
provides information for L3 reachability.


But at the end of the day, I still can't create a Port until a Subnet exists


This is only true if you want an IP address on the port. This sounds
silly for most use cases, but there are a non-trivial portion of NFV
workloads that do not want IP addresses at all so they create a network
and just attach ports without creating any subnets.


I still don't know what Subnet a Port will be attached to (unless the

user specifies it explicitly using the --fixed-ip option... regardless
of whether they actually specify a fixed IP),

So what would you like Neutron to do differently here? Always force a
user to pick which subnet they want an allocation from if there are
multiple? If so, can't you just force that explicitness in Heat?


and I have no way in general of telling which Subnets can be deleted before a 
given Port is and which will fail to delete until the Port disappears.


A given port will only block subnet deletions from subnets it is
attached to. Conversely, you can see all ports with allocations from a
subnet with 'neutron port-list --fixed-ips subnet_id='.  So
is the issue here that the dependency wasn't made explicit in the heat
modeling (leading to the problem above and this one)?


For the individual bugs you highlighted, it would be good if you can
provide some details about what changes we could make to help.


https://bugs.launchpad.net/heat/+bug/1442121 - This looks like a result
of partially specified floating IPs (no fixed_ip). What can we
add/change here to help? Or can heat just always force the user to
specify a fixed IP for the case where disambiguation on multiple
fixed_ip ports is needed?


If the server has more than one fixed_ip ports, it's possible that only 
one of them will be able to receive a floating ip. The subnet a port 
comes from must have gateway_ip set for a floating_ip to attach to it. 
So if you have a server, you can poke and find the right fixed_ip in all 
cases except when the server has more than one fixed_ip and each of them 
are from a subnet with a gateway_ip. In that case, a user _must_ provide 
a fixed_ip, because there is no way to know what they intend.



https://launchpad.net/bugs/1626607 - I see this is about a dependency
between RouterGateways and RouterInterfaces, but it's not clear to me
why that dependency exists. Is it to solve a lack of visibility into the
interfaces required for a floating IP?

https://bugs.launchpad.net/heat/+bug/1626619,
https://bugs.launchpad.net/heat/+bug/1626630, and
https://bugs.launchpad.net/heat/+bug/1626634 - These seems similar to
1626607. Can we just expose the interfaces/router a floating IP is
depending on explicitly in the API for you to fix these? If not, what
can we do to help here?


1. http://lists.openstack.org/pipermail/openstack-dev/2017-May/117106.html

Cheers,
Kevin Benton

On Fri, May 19, 2017 at 1:05 PM, Zane Bitter > wrote:

On 19/05/17 15:06, Kevin Benton wrote:

Don't even get me started on Neutron.[2]


It seems to me the conclusion to that thread was that the
majority of
your issues stemmed from the fact that we had poor documentation
at the
time.  A major component of the complaints resulted from you
misunderstanding the difference between networks/subnets in Neutron.


It's true that I was completely off base as to what the various
primitives in Neutron actually do. (Thanks for educating me!) The
implications for orchestration are largely unchanged though. It's a
giant pain that we have to infer implicit dependencies between stuff
to get them to create/delete in the right order, pretty much
independently of what that stuff does.

So knowing now that a Network is a layer-2 network segment and a
Subnet is... effectively a glorified DHCP address pool, I understand
better why it probably seemed like a good idea to hook stuff up
magically. But at the end of the day, I still can't 

[openstack-dev] [neutron][heat] - making Neutron more friendly for orchestration

[Kevin Benton]

I split this conversation off of the "Is the pendulum swinging on PaaS
layers?" thread [1] to discuss some improvements we can make to Neutron to
make orchestration easier.

There are some pain points that heat has when working with the Neutron API.
I would like to get them converted into requests for enhancements in
Neutron so the wider community is aware of them.

Starting with the port/subnet/network relationship - it's important to
understand that IP addresses are not required on a port.

>So knowing now that a Network is a layer-2 network segment and a Subnet
is... effectively a glorified DHCP address pool

Yes, a Subnet controls IP address allocation as well as setting up routing
for routers, which is why routers reference subnets instead of networks
(different routers can route for different subnets on the same network). It
essentially dictates things related to L3 addressing and provides
information for L3 reachability.

>But at the end of the day, I still can't create a Port until a Subnet
exists

This is only true if you want an IP address on the port. This sounds silly
for most use cases, but there are a non-trivial portion of NFV workloads
that do not want IP addresses at all so they create a network and just
attach ports without creating any subnets.

>I still don't know what Subnet a Port will be attached to (unless the user
specifies it explicitly using the --fixed-ip option... regardless of
whether they actually specify a fixed IP),

So what would you like Neutron to do differently here? Always force a user
to pick which subnet they want an allocation from if there are multiple? If
so, can't you just force that explicitness in Heat?

> and I have no way in general of telling which Subnets can be deleted
before a given Port is and which will fail to delete until the Port
disappears.

A given port will only block subnet deletions from subnets it is attached
to. Conversely, you can see all ports with allocations from a subnet with
'neutron port-list --fixed-ips subnet_id='.  So is the issue
here that the dependency wasn't made explicit in the heat modeling (leading
to the problem above and this one)?


For the individual bugs you highlighted, it would be good if you can
provide some details about what changes we could make to help.


https://bugs.launchpad.net/heat/+bug/1442121 - This looks like a result of
partially specified floating IPs (no fixed_ip). What can we add/change here
to help? Or can heat just always force the user to specify a fixed IP for
the case where disambiguation on multiple fixed_ip ports is needed?

https://launchpad.net/bugs/1626607 - I see this is about a dependency
between RouterGateways and RouterInterfaces, but it's not clear to me why
that dependency exists. Is it to solve a lack of visibility into the
interfaces required for a floating IP?

https://bugs.launchpad.net/heat/+bug/1626619,
https://bugs.launchpad.net/heat/+bug/1626630, and
https://bugs.launchpad.net/heat/+bug/1626634 - These seems similar to
1626607. Can we just expose the interfaces/router a floating IP is
depending on explicitly in the API for you to fix these? If not, what can
we do to help here?


1. http://lists.openstack.org/pipermail/openstack-dev/2017-May/117106.html

Cheers,
Kevin Benton

On Fri, May 19, 2017 at 1:05 PM, Zane Bitter  wrote:

> On 19/05/17 15:06, Kevin Benton wrote:
>
>> Don't even get me started on Neutron.[2]
>>>
>>
>> It seems to me the conclusion to that thread was that the majority of
>> your issues stemmed from the fact that we had poor documentation at the
>> time.  A major component of the complaints resulted from you
>> misunderstanding the difference between networks/subnets in Neutron.
>>
>
> It's true that I was completely off base as to what the various primitives
> in Neutron actually do. (Thanks for educating me!) The implications for
> orchestration are largely unchanged though. It's a giant pain that we have
> to infer implicit dependencies between stuff to get them to create/delete
> in the right order, pretty much independently of what that stuff does.
>
> So knowing now that a Network is a layer-2 network segment and a Subnet
> is... effectively a glorified DHCP address pool, I understand better why it
> probably seemed like a good idea to hook stuff up magically. But at the end
> of the day, I still can't create a Port until a Subnet exists, I still
> don't know what Subnet a Port will be attached to (unless the user
> specifies it explicitly using the --fixed-ip option... regardless of
> whether they actually specify a fixed IP), and I have no way in general of
> telling which Subnets can be deleted before a given Port is and which will
> fail to delete until the Port disappears.
>
> There are some legitimate issues in there about the extra routes
>> extension being replace-only and the routers API not accepting a list of
>> interfaces in POST.  However, it hardly seems that those are worthy of
>> "Don't even get me started on