Responses in-line:

From: Vijay B <<>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
Date: Tuesday, October 14, 2014 7:08 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
Subject: Re: [openstack-dev] [Neutron][LBaaS][Octavia] Floating IP management

Hi Phillip,

Adding my thoughts below. I’ll first answer the questions you raised with what 
I think should be done, and then give my explanations to reason through with 
those views.

1. Do we want to add logic in the plugin to call the FLIP association API?

>> We should implement the logic in the new v2 extension and the plugin layer 
>> as a single API call. We would need to add to the existing v2 API to be able 
>> to do this. The best place to add this option of passing the FLIP 
>> info/request to the VIP is in the VIP create and update API calls via new 
>> parameters.

>>>>Agreed, we would need to have another field for the FLIP itself. Although, 
>>>>I was under the impression we would use the associate_floating_ip and not 
>>>>have to modify the VIP call at all. I may be misunderstanding where you 
>>>>think this should happen.

2. If we have logic in the plugin should we have configuration that identifies 
whether to use/return the FLIP instead of the port VIP?

>> Yes and no, in that we should return the complete result of the VIP 
>> create/update/list/show API calls, in which we show the VIP internal IP, but 
>> we also show the FLIP either as empty or having a FLIP uuid. External users 
>> will anyway use only the FLIP, else they wouldn’t be able to reach the LB 
>> and the VIP IP, but the APIs need to show both fields.

>>>> By this, do you mean we would be altering the vip_port API calls? Again, I 
>>>> was under the impression that we would utilize neutrons FLIP calls. With 
>>>> the question #2 I was referring to 'replacing' the VIP on the LB object 
>>>> with a configuration value rather than showing both or just the VIP. If by 
>>>> VIP CRUD you are talking about LBaaS API then this makes sense to me and 
>>>> we should indeed show them both.

3. Would we rather have logic for FLIP association in the drivers?

>> This is the hardest part to decide. To understand this, we need to look at 
>> two important drivers of LBaaS design:

I)  The Neutron core plugin we’re using.

II) The different types of LB devices - physical, virtual standalone, and 
virtual controlled by a management plane. This leads to different kinds of 
LBaaS drivers and different kinds of interaction or the lack of it between them 
and the core neutron plugin.

The reason we need to take into account both these is that port provisioning as 
well as NATing for the FLIP to internal VIP IP will be configured differently 
by the different network management/backend planes that the plugins use, and 
the way drivers configure LBs can be highly impacted by this.

For example, we can have an NSX infrastructure that will implement the FLIP to 
internal IP conversion in the logical router module which sits pretty much 
outside of Openstack’s realm, using openflow. Or we can use lighter solutions 
directly on the hypervisor that still employ open flow entries without actually 
having a dedicated logical routing module. Neither will matter much if we are 
in a position to have them deploy our networking for us, i.e., in the cases of 
us using virtual LBs sitting on compute nodes. But if we have a physical LB, 
the neutron plugins cannot do much of the network provisioning work for us, 
typically because physical LBs usually sit outside of the cloud, and are among 
the earliest points of contact from the external world.

This already nudges us to consider putting the FLIP provisioning functionality 
in the driver. However, consider again more closely the major ways in which 
LBaaS drivers talk to LB solutions today depending on II) :

a) LBaaS drivers that talk to a virtual LB device on a compute node, directly.

b) LBaaS drivers that talk to a physical LB device (or a virtual LB sitting 
outside the cloud) directly.

c) LBaaS drivers that talk to a management plane like F5’s BigIQ, or 
Netscaler’s NCC, or as in our case, Octavia, that try to provide tenant based 
provisioning of virtual LBs.

d) The HAProxy reference namespace driver.

d) is really a PoC use case, and we can forget it. Let’s consider a), b) and c).

If we use a) or b), we must assume that the required routing for the virtual LB 
has been setup correctly, either already through nova or manually. So we can 
afford to do our FLIP plumbing in the neutron plugin layer, but, driven by the 
driver - how? - typically, after the VIP is successfully created on the LB, and 
just before the driver updates the VIP’s status as ACTIVE, it can create the 
FLIP. Of course, if the FLIP provisioning fails for any reason, the VIP still 
stands. It’ll be empty in the result, and the API will error out saying “VIP 
created but FLIP creation failed”. It must be manually deleted by another 
delete VIP call. We can’t afford to provision a FLIP before a VIP is active, 
for external traffic shouldn’t be taken while the VIP isn’t up yet. If the 
lines are getting hazy right now because of this callback model, let’s just 
focus on the point that we’re initiating FLIP creation in the driver layer 
while the code sits in the plugin layer because it will need to update the 
database. But in absolute terms, we’re doing it in the driver.

It is use case c) that is interesting. In this case, we should do all neutron 
based provisioning neither in the driver, nor in the plugin in neutron, rather, 
we should do this in Octavia, and in the Octavia controller to be specific. 
This is very important to note, because if customers are using this deployment 
(which today has the potential to be way greater in the near future than any 
other model simply because of the sheer existing customer base), we can’t be 
creating the FLIP in the plugin layer and have the controller reattempt it. 
Indeed, the controllers can change their code to not attempt this, but like 
just noted, it isn’t logically orderly to create the FLIP before creating the 
VIP successfully. So let’s leave this op to the driver.

I am guilty of not following the proceedings on the Octavia controller design 
due to internal work deadlines, but if I remember right, the design specified a 
controller being built to spawn and configure virtual HAProxy LB instances. If 
Octavia is not taking that route today, I think that the best place to do this 
provisioning would be as described above for a) and b).

>>>>These are good points! Having the FLIP creation in the driver or in 
>>>>Octavias case the controller code that manages the HA of the VLBs is a 
>>>>great idea. Would this mean though that we are bypassing Neutrons calls 
>>>>completely if we do this? Or would this be API calls made from the 
>>>>controller/driver to Neutron? If were bypassing neutron we should assume 
>>>>that the deployers environment has the plumbing set up for the FLIP pool 
>>>>and some kind of service that can be queried for the FLIP?

4) If logic is in the drivers would we still return the port VIP to the user 
then later overwrite it with the FLIP?

>> As described in 2), we always return the VIP port along with the FLIP info 
>> in API calls. Users like PaaS that sit on top of IaaS (Openstack) will only 
>> use the FLIP when they commission endpoints for their users, who will only 
>> be concerned with the FLIP.

5) Or would we have configuration to not return the port VIP initially, but an 
additional query would show the associated FLIP.

>> Not inclined to doing this :) It’s just another field associated with the 
>> VIP - an API call for it would be overkill.

>>>>Makes sense to me

6) Is there an internal service call for this, and if so would we use it 
instead of API calls?

>> Really depends on the core plugin you’re using - each plugin does this 
>> differently, but all you need to do is to simply call 
>> core_plugin.create_floatingip() and it should do everything for you - it 
>> should correctly create the FLIP entry in the db, then go to its backend (a 
>> controller or ovs or the L3 agent etc) which will correctly provision the 
>> necessary interfaces and/or program the required flow rules or iptables 
>> rules or equivalent and return success or failure.

>>>> I may need to dig into the docs a bit more, but if theres a 
>>>> core_plugin.associate_flip call as there is in the API then this would 
>>>> save us from writing API communications from the controller/driver if we 
>>>> go that route.

All said and done, the fact is that in LBaaS, as with many other components, we 
can always override the default plugin and driver layers and write our own and 
do whatever we want in them :) This allows vendors to rewrite implementations 
if they need to. Hopefully we’ll get the design well enough that they don’t 
have to do that in more than a few modules.

Hope this helps! Please do let me know if you have any questions or need more 
pointers with the extensions/neutron/etc.

>>>> Thanks for the response and information! I may have a few points 
>>>> fundamentally incorrect, which is reasoning for this discussion, so I do 
>>>> appreciate the time you've taken to discuss with me :)




On Mon, Oct 13, 2014 at 1:18 PM, Phillip Toohill 
<<>> wrote:
Diagrams in jpeg format..

On 10/12/14 10:06 PM, "Phillip Toohill" 

>Hello all,
>Heres some additional diagrams and docs. Not incredibly detailed, but
>should get the point across.
>Feel free to edit if needed.
>Once we come to some kind of agreement and understanding I can rewrite
>these more to be thorough and get them in a more official place. Also, I
>understand theres other use cases not shown in the initial docs, so this
>is a good time to collaborate to make this more thought out.
>Please feel free to ping me with any questions,
>Thank you
>Google DOCS link for FLIP folder:
>-diagrams are<> based and can be opened from within 
>Drive by
>selecting the appropriate application.
>On 10/7/14 2:25 PM, "Brandon Logan" 
><<>> wrote:
>>I'll add some more info to this as well:
>>Neutron LBaaS creates the neutron port for the VIP in the plugin layer
>>before drivers ever have any control.  In the case of an async driver,
>>it will then call the driver's create method, and then return to the
>>user the vip info.  This means the user will know the VIP before the
>>driver even finishes creating the load balancer.
>>So if Octavia is just going to create a floating IP and then associate
>>that floating IP to the neutron port, there is the problem of the user
>>not ever seeing the correct VIP (which would be the floating iP).
>>So really, we need to have a very detailed discussion on what the
>>options are for us to get this to work for those of us intending to use
>>floating ips as VIPs while also working for those only requiring a
>>neutron port.  I'm pretty sure this will require changing the way V2
>>behaves, but there's more discussion points needed on that.  Luckily, V2
>>is in a feature branch and not merged into Neutron master, so we can
>>change it pretty easily.  Phil and I will bring this up in the meeting
>>tomorrow, which may lead to a meeting topic in the neutron lbaas
>>On Mon, 2014-10-06 at 17:40 +0000, Phillip Toohill wrote:
>>> Hello All,
>>> I wanted to start a discussion on floating IP management and ultimately
>>> decide how the LBaaS group wants to handle the association.
>>> There is a need to utilize floating IPs(FLIP) and its API calls to
>>> associate a FLIP to the neutron port that we currently spin up.
>>> See DOCS here:
>>> >
>>> Currently, LBaaS will make internal service calls (clean interface :/)
>>>to create and attach a Neutron port.
>>> The VIP from this port is added to the Loadbalancer object of the Load
>>>balancer configuration and returned to the user.
>>> This creates a bit of a problem if we want to associate a FLIP with the
>>>port and display the FLIP to the user instead of
>>> the ports VIP because the port is currently created and attached in the
>>>plugin and there is no code anywhere to handle the FLIP
>>> association.
>>> To keep this short and to the point:
>>> We need to discuss where and how we want to handle this association. I
>>>have a few questions to start it off.
>>> Do we want to add logic in the plugin to call the FLIP association API?
>>> If we have logic in the plugin should we have configuration that
>>>identifies weather to use/return the FLIP instead the port VIP?
>>> Would we rather have logic for FLIP association in the drivers?
>>> If logic is in the drivers would we still return the port VIP to the
>>>user then later overwrite it with the FLIP?
>>> Or would we have configuration to not return the port VIP initially,
>>>but an additional query would show the associated FLIP.
>>> Is there an internal service call for this, and if so would we use it
>>>instead of API calls?
>>> Theres plenty of other thoughts and questions to be asked and discussed
>>>in regards to FLIP handling,
>>> hopefully this will get us going. I'm certain I may not be completely
>>>understanding this and
>>> is the hopes of this email to clarify any uncertainties.
>>> _______________________________________________
>>> OpenStack-dev mailing list
>>OpenStack-dev mailing list
>OpenStack-dev mailing list

OpenStack-dev mailing list<>

OpenStack-dev mailing list

Reply via email to