Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-04-08 Thread Pavel Bondar 
I believe during next week I will upload code for the one time switch to
pluggable ipam using pure alembic migration.

Thanks,
Pavel

On 05.04.2016 22:05, Carl Baldwin wrote:
> I think the only thing standing in our way is this bug [1].  Ryan
> Tidwell and I are working on this.
>
> Carl
>
> [1] https://bugs.launchpad.net/neutron/+bug/1543094
>
> On Mon, Apr 4, 2016 at 3:48 PM, John Belamaric  
> wrote:
>> I was on vacation last week so I am just seeing this now. I agree that now 
>> that we are in Newton we should just remove the non-pluggable code and move 
>> forward with the migration.
>>
>> John
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-04-05 Thread Carl Baldwin 
I think the only thing standing in our way is this bug [1].  Ryan
Tidwell and I are working on this.

Carl

[1] https://bugs.launchpad.net/neutron/+bug/1543094

On Mon, Apr 4, 2016 at 3:48 PM, John Belamaric  wrote:
> I was on vacation last week so I am just seeing this now. I agree that now 
> that we are in Newton we should just remove the non-pluggable code and move 
> forward with the migration.
>
> John
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-04-04 Thread John Belamaric 
I was on vacation last week so I am just seeing this now. I agree that now that 
we are in Newton we should just remove the non-pluggable code and move forward 
with the migration.

John
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-03-30 Thread Carl Baldwin 
On Wed, Mar 30, 2016 at 9:03 AM, Pavel Bondar  wrote:
> Kevin Benton commented on review page for current migration to pluggable
> approach [1]:
>
> IMO this cannot be optional. It's going to be a nightmare to try to support
> two IPAM systems that people may have switched between at various points in
> time. I would much rather go all-in on the upgrade by making it automatic
> with alembic and removing the option to use the legacy IPAM code completely.
>
> I've already been bitten by testing the new IPAM code with the config option
> and switching back which resulted in undeletable subnets. Now we can always
> yell at anyone that changes the config option like I did, but it takes a lot
> of energy to yell at users and they don't care for it much. :)
>
> Even ignoring the support issue, consider schema changes. This migration
> script will have to be constantly updated to work with whatever the current
> state of the schema is on both sets of ipam tables. Without constant in-tree
> testing enforcing that, we are one schema change away from this script
> breaking.
>
> So let's bite the bullet and make this a normal contract migration. Either
> the new ipam system is stable enough for us to commit to supporting it and
> fix whatever bugs it may have, or we need to remove it from the tree.
> Supporting both systems is unsustainable.
>
> This sound reasonable to me. It simplify support and testing (testing both
> implementations in gate with full coverage is not easy).
> From user prospective there should be no visible changes between pluggable
> ipam and non-pluggable.
> And making switch early in release cycle gives us enough time to fix any bug
> we will find in pluggable implementation.

This is what I want too but some people wanted to allow choice.

> Right now we have some open bugs for pluggable code [2], but they are still
> possible to fix.

Yes, we've got to fix this one but I think we have a way forward.  I'm
actually going to be working in IPAM for the next week or two on work
related to the thread I posted to yesterday [3].  Maybe I could help
out with this.  Could you get this migration lined up in review and
then we'll tackle the bugs as a joint effort?  Hopefully we can make
the switch before summit.

Carl

> Does it make sense to you?
>
> [1] https://review.openstack.org/#/c/277767/
> [2] https://bugs.launchpad.net/neutron/+bug/1543094

[3] http://lists.openstack.org/pipermail/openstack-dev/2016-March/090748.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-03-30 Thread Neil Jerram 
On 30/03/16 16:08, Pavel Bondar wrote:

> We are now in early Newton, so it is good time to discuss plan for
> pluggable ipam for this release cycle.
>
> Kevin Benton commented on review page for current migration to pluggable
> approach [1]:
>>
>> IMO this cannot be optional. It's going to be a nightmare to try to
>> support two IPAM systems that people may have switched between at
>> various points in time. I would much rather go all-in on the upgrade
>> by making it automatic with alembic and removing the option to use the
>> legacy IPAM code completely.
>>
>> I've already been bitten by testing the new IPAM code with the config
>> option and switching back which resulted in undeletable subnets. Now
>> we can always yell at anyone that changes the config option like I
>> did, but it takes a lot of energy to yell at users and they don't care
>> for it much. :)
>>
>> Even ignoring the support issue, consider schema changes. This
>> migration script will have to be constantly updated to work with
>> whatever the current state of the schema is on both sets of ipam
>> tables. Without constant in-tree testing enforcing that, we are one
>> schema change away from this script breaking.
>>
>> So let's bite the bullet and make this a normal contract migration.
>> Either the new ipam system is stable enough for us to commit to
>> supporting it and fix whatever bugs it may have, or we need to remove
>> it from the tree. Supporting both systems is unsustainable.
>>
> This sound reasonable to me. It simplify support and testing (testing
> both implementations in gate with full coverage is not easy).
>  From user prospective there should be no visible changes between
> pluggable ipam and non-pluggable.
> And making switch early in release cycle gives us enough time to fix any
> bug we will find in pluggable implementation.
>
> Right now we have some open bugs for pluggable code [2], but they are
> still possible to fix.
>
> Does it make sense to you?

Yes!  Kill the non-pluggable code already.  Neutron desperately needs to 
have less and simpler code in any area where it can possibly get it.

Neil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-03-30 Thread Pavel Bondar 
On 12.02.2016 15:01, Ihar Hrachyshka wrote:
> Salvatore Orlando  wrote:
>
>> On 11 February 2016 at 20:17, John Belamaric
>>  wrote:
>>
>>> On Feb 11, 2016, at 12:04 PM, Armando M.  wrote:
>>>
>>>
>>>
>>> On 11 February 2016 at 07:01, John Belamaric
>>>  wrote:
>>>
>>>
>>>
>>> It is only internal implementation changes.
>>>
>>> That's not entirely true, is it? There are config variables to
>>> change and it opens up the possibility of a scenario that the
>>> operator may not care about.
>>>
>>
>> If we were to remove the non-pluggable version altogether, then the
>> default for ipam_driver would switch from None to internal.
>> Therefore, there would be no config file changes needed.
>>
>> I think this is correct.
>> Assuming the migration path to Neutron will include the data
>> transformation from built-in to pluggable IPAM, do we just remove the
>> old code and models?
>> On the other hand do you think it might make sense to give operators
>> a chance to rollback - perhaps just in case some nasty bug pops up?
>
> They can always revert to a previous release. And if we enable the new
> implementation start of Newton, we’ll have enough time to fix bugs
> that will pop up in gate.
>
We are now in early Newton, so it is good time to discuss plan for
pluggable ipam for this release cycle.

Kevin Benton commented on review page for current migration to pluggable
approach [1]:
>
> IMO this cannot be optional. It's going to be a nightmare to try to
> support two IPAM systems that people may have switched between at
> various points in time. I would much rather go all-in on the upgrade
> by making it automatic with alembic and removing the option to use the
> legacy IPAM code completely.
>
> I've already been bitten by testing the new IPAM code with the config
> option and switching back which resulted in undeletable subnets. Now
> we can always yell at anyone that changes the config option like I
> did, but it takes a lot of energy to yell at users and they don't care
> for it much. :)
>
> Even ignoring the support issue, consider schema changes. This
> migration script will have to be constantly updated to work with
> whatever the current state of the schema is on both sets of ipam
> tables. Without constant in-tree testing enforcing that, we are one
> schema change away from this script breaking.
>
> So let's bite the bullet and make this a normal contract migration.
> Either the new ipam system is stable enough for us to commit to
> supporting it and fix whatever bugs it may have, or we need to remove
> it from the tree. Supporting both systems is unsustainable.
>
This sound reasonable to me. It simplify support and testing (testing
both implementations in gate with full coverage is not easy).
>From user prospective there should be no visible changes between
pluggable ipam and non-pluggable.
And making switch early in release cycle gives us enough time to fix any
bug we will find in pluggable implementation.

Right now we have some open bugs for pluggable code [2], but they are
still possible to fix.

Does it make sense to you?

[1] https://review.openstack.org/#/c/277767/
[2] https://bugs.launchpad.net/neutron/+bug/1543094
>> What's the team level of confidence in the robustness of the
>> reference IPAM driver?
>>
>> Salvatore
>>
>>
>>
>>
>> John
>>
>>
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>>
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
>
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-16 Thread Carl Baldwin 
On Mon, Feb 15, 2016 at 9:26 AM, Pavel Bondar  wrote:
> Your idea sounds workable to me. However I think a simpler way exists.

I'll be happy to review your migration for Mitaka which should be
totally out-of-band and leave both implementations intact.  As for the
details of the switch-over in Newton, I'll be happy to leave it up to
you.

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-15 Thread Pavel Bondar 
On 13.02.2016 02:42, Carl Baldwin wrote:
> On Fri, Feb 12, 2016 at 5:01 AM, Ihar Hrachyshka  wrote:
 It is only internal implementation changes.

 That's not entirely true, is it? There are config variables to change and
 it opens up the possibility of a scenario that the operator may not care
 about.

>>> If we were to remove the non-pluggable version altogether, then the
>>> default for ipam_driver would switch from None to internal. Therefore, there
>>> would be no config file changes needed.
>>>
>>> I think this is correct.
>>> Assuming the migration path to Neutron will include the data
>>> transformation from built-in to pluggable IPAM, do we just remove the old
>>> code and models?
>>> On the other hand do you think it might make sense to give operators a
>>> chance to rollback - perhaps just in case some nasty bug pops up?
>>
>> They can always revert to a previous release. And if we enable the new
>> implementation start of Newton, we’ll have enough time to fix bugs that will
>> pop up in gate.
> So, to do this, we have to consider two classes of current users.
> Since the pluggable implementation has been available, I think that we
> have to assume someone might be using it.  Someone could easily have
> turned it on in a green-field deployment.  If we push the offline
> migration in to Mitaka as per my last email then we'll likely get a
> few more of these but it doesn't really matter, the point is that I
> think we need t assume that they exist.
>
> 1) Users of the old baked-in implementation
>   - Their current data is stored in the old tables.
>
> 2) User of the new pluggable implementation
>  - Their current data is stored in the new tables.
>
> So, how does an unconditional migration work?  We can't just copy the
> old tables to the new tables because we might clobber data for the
> users in #2.  I've already heard that conditional migrations are a
> pain and shouldn't be considered.  This seems like a problem.
>
> I had an idea that I wanted to share but I'm warning you, it sounds a
> little crazy even to me.  But, maybe it could work.  Read through it
> for entertainment purposes if nothing else.
>
> Instead of migrating data from the old tables to the new.  What if we
> migrated the old tables in place in a patch set that removed all of
> the old code?  The table structure is nearly identical, right?  The
> differences, I imagine, could be easily handled by an alembic
> migration.  Correct me if I'm wrong.
>
> Now, we still have a difference between users in groups #1 and #2
> above.  To keep them separate, we would call the new built-in
> pluggable driver "built-in", "neutron", or whatever.  The name isn't
> important except that it can't be "internal".
>
> 1) Users who were migrated to the new baked-in implementation.
>   - Their current data is still in the old tables but they have been
> migrated to look just like the new tables.
>   - They have still not set "ipam_driver" in their config so they get
> the new default of "built-in".
>
> 2) Early adopters of built-in pluggable ipam
>   - Their current data is still in the new tables
>   - They have their config set to "internal" already
>
> So, now we have to deal with two identical pluggable implementations:
> one called "built-in" and the other called "internal" but otherwise
> they're identical in every way.  So, to handle this, could we
> parameterize the plugin so that they share exactly the same code while
> "internal" is deprecated?  Just the table names would be
> parameterized.
>
> We have to eventually reconcile users in group #1 with #2.  But, now
> that the tables are identical we could provide an offline migration
> that might be as simple as deleting the "old" tables and renaming the
> "new" tables.  Now, only users in group #2 are required to perform an
> offline migration.
>
> Carl
Hi Carl,

Your idea sounds workable to me. However I think a simpler way exists.

Significant part of 'built-in' ipam tables are continued to be updated even
if reference ipam driver is used (or any another driver).
It happens because these tables are API exposed, so they still have to
receive updates.

To be specific, next models from 'built-in' are related to ipam in some way:
- Subnet;
- IPAllocationPool;
- IPAllocation;
- IPAvailabilityRange;
Only IPAvailabilityRange stops to receive updates when switch to pluggable
ipam backend occurs.  And IPAvailabilityRange can be rebuilt based on
information
from IPAllocationPool and IPAllocation models [1].
It gives us ability to rebuild all the needed ipam information in
'built-in' tables even
if ipam driver is used.

I am trying to implement this approach in current version of migration
to pluggable ipam [2]
to allow safe switch back to 'built-in' implementation from pluggable.
'--rebuild' flag forces ip availability ranges recalculation relying
only on tables that are up to date
independently of backend currently used.

So to remove built-in ipam

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-12 Thread Ihar Hrachyshka 

Salvatore Orlando  wrote:

On 11 February 2016 at 20:17, John Belamaric   
wrote:



On Feb 11, 2016, at 12:04 PM, Armando M.  wrote:



On 11 February 2016 at 07:01, John Belamaric   
wrote:




It is only internal implementation changes.

That's not entirely true, is it? There are config variables to change  
and it opens up the possibility of a scenario that the operator may not  
care about.




If we were to remove the non-pluggable version altogether, then the  
default for ipam_driver would switch from None to internal. Therefore,  
there would be no config file changes needed.


I think this is correct.
Assuming the migration path to Neutron will include the data  
transformation from built-in to pluggable IPAM, do we just remove the old  
code and models?
On the other hand do you think it might make sense to give operators a  
chance to rollback - perhaps just in case some nasty bug pops up?


They can always revert to a previous release. And if we enable the new  
implementation start of Newton, we’ll have enough time to fix bugs that  
will pop up in gate.


What's the team level of confidence in the robustness of the reference  
IPAM driver?


Salvatore




John



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-12 Thread Salvatore Orlando 
On 11 February 2016 at 20:17, John Belamaric 
wrote:

>
> On Feb 11, 2016, at 12:04 PM, Armando M.  wrote:
>
>
>
> On 11 February 2016 at 07:01, John Belamaric 
> wrote:
>
>>
>>
>>
>> It is only internal implementation changes.
>>
>
> That's not entirely true, is it? There are config variables to change and
> it opens up the possibility of a scenario that the operator may not care
> about.
>
>
>
> If we were to remove the non-pluggable version altogether, then the
> default for ipam_driver would switch from None to internal. Therefore,
> there would be no config file changes needed.
>

I think this is correct.
Assuming the migration path to Neutron will include the data transformation
from built-in to pluggable IPAM, do we just remove the old code and models?
On the other hand do you think it might make sense to give operators a
chance to rollback - perhaps just in case some nasty bug pops up?
What's the team level of confidence in the robustness of the reference IPAM
driver?

Salvatore



>
>
> John
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-12 Thread Carl Baldwin 
On Thu, Feb 11, 2016 at 10:04 AM, Armando M.  wrote:
> I believe we have more recovery options out a potentially fatal situation.
> In fact the offline script can provide a dry-run option that can just
> validate that the migration will succeed before it is even actually
> performed; I think that the size and the amount of tables involved in the
> data migration justifies this course of action rather than the other. Think
> about what Sean said, bugs are always lurking in the dark and as much as we
> can strive for correctness, things might go bad. This is not a routine
> migration and some operators may not be in a rush to embrace pluggable IPAM,
> hence I don't think we are in the position to make the decision on their
> behalf and go through the usual fast-path deprecation process.

I had a long discussion with Armando about this.  I was pretty
stubborn but there was one point that came up that got through and got
me thinking.  Basically, it is that having the ability to migrate to
pluggable IPAM in Mitaka could be a key component to giving users a
path to migrate to a 3rd party pluggable implementation.  Without it,
3rd parties will have to support two kinds of migration:  one for each
of the drivers currently available.

The only way to get something in to Mitaka is do to an offline
migration.  I agree that we shouldn't do the full automatic switch
this late in the cycle.

So, is this worth getting in to Mitaka to help this use case?  If it
is a significantly important component of migrating to 3rd party IPAM
then maybe the answer should be yes.  If it is just to help get people
to the pluggable internal implementation in Mitaka then I'd say no
because it it doesn't provide any user visible advantage and it
doesn't yet have the gate testing miles on it that the old
implementation has.

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread John Belamaric 


---
John Belamaric
(240) 383-6963

> On Feb 11, 2016, at 5:37 AM, Ihar Hrachyshka  wrote:
> 
> What’s the user visible change in behaviour after the switch? If it’s only 
> internal implementation change, I don’t see why we want to leave the choice 
> to operators.
> 

It is only internal implementation changes. 


>> The other aspect is the deprecation process. If you add the switch into the 
>> DB migration path then the whole deprecation becomes superseded as the old 
>> IPAM logic should be abandoned immediately after that. But perhaps the other 
>> way of looking at it is that we should make an exception in the deprecation 
>> process.
>> 
>> Salvatore
>> 
>> On 11 February 2016 at 00:19, Carl Baldwin  wrote:
>> On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
>> > Technically we can make this as sophisticated and seamless as we want, but
>> > this is a one-off, once it's done the pain goes away, and we won't be doing
>> > another migration like this ever again. So I wouldn't over engineer it.
>> 
>> Frankly, I was worried that going the other way was over-engineering
>> it.  It will be more difficult for us to manage this transition.
>> 
>> I'm still struggling to see what makes this particular migration
>> different than other cases where we change the database schema and the
>> code a bit and we automatically migrate everyone to it as part of the
>> routine migration.  What is it about this case that necessitates
>> giving the operator the option?
>> 
>> Carl
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Carl Baldwin 
On Thu, Feb 11, 2016 at 3:32 AM, Ihar Hrachyshka  wrote:
> Salvatore Orlando  wrote:
>
>> The difference lies in the process in my opinion.
>> If the switch is added into the migration path then we will tell operators
>> when to switch.
>> I was suggesting doing it manual because we just don't know if every
>> operator is happy about doing the switch when upgrading to Newton, but
>> perhaps it is just me over-worrying about operator behaviour.
>
> What’s the user visible change in behaviour after the switch? If it’s only
> internal implementation change, I don’t see why we want to leave the choice
> to operators.

This was my thinking exactly.  However, I did not do the
implementation, Salvatore did.  So, ultimately, I think that he should
be convinced that we're doing the right thing.  I value his input
highly.

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread John Belamaric 

On Feb 11, 2016, at 12:04 PM, Armando M. 
> wrote:



On 11 February 2016 at 07:01, John Belamaric 
> wrote:



It is only internal implementation changes.

That's not entirely true, is it? There are config variables to change and it 
opens up the possibility of a scenario that the operator may not care about.


If we were to remove the non-pluggable version altogether, then the default for 
ipam_driver would switch from None to internal. Therefore, there would be no 
config file changes needed.


John


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Carl Baldwin 
On Thu, Feb 11, 2016 at 3:20 AM, Salvatore Orlando
 wrote:
> The difference lies in the process in my opinion.
> If the switch is added into the migration path then we will tell operators
> when to switch.
> I was suggesting doing it manual because we just don't know if every
> operator is happy about doing the switch when upgrading to Newton, but
> perhaps it is just me over-worrying about operator behaviour.

I think this is the point that makes this discussion worth having.  It
does help me to hear you state this concern in this way.  I'd like to
hear/read some other opinions.

> The other aspect is the deprecation process. If you add the switch into the
> DB migration path then the whole deprecation becomes superseded as the old
> IPAM logic should be abandoned immediately after that. But perhaps the other
> way of looking at it is that we should make an exception in the deprecation
> process.

I agree.  If we do decide that we force the switch then we should
immediately abandon the old code.  However, I don't this should drive
the decision to do the switch with the migration.

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Carl Baldwin 
On Thu, Feb 11, 2016 at 10:04 AM, Armando M.  wrote:
> On 11 February 2016 at 07:01, John Belamaric 
> wrote:
>> It is only internal implementation changes.
>
> That's not entirely true, is it? There are config variables to change and it
> opens up the possibility of a scenario that the operator may not care about.

You're right.  I was thinking that if we handled the switch then we
could obsolete the config variable.  Maybe we shot ourselves in the
foot already by having the config option in the first place.  Is that
what you're thinking?

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Ihar Hrachyshka 

Salvatore Orlando  wrote:


The difference lies in the process in my opinion.
If the switch is added into the migration path then we will tell  
operators when to switch.
I was suggesting doing it manual because we just don't know if every  
operator is happy about doing the switch when upgrading to Newton, but  
perhaps it is just me over-worrying about operator behaviour.




What’s the user visible change in behaviour after the switch? If it’s only  
internal implementation change, I don’t see why we want to leave the choice  
to operators.


The other aspect is the deprecation process. If you add the switch into  
the DB migration path then the whole deprecation becomes superseded as  
the old IPAM logic should be abandoned immediately after that. But  
perhaps the other way of looking at it is that we should make an  
exception in the deprecation process.


Salvatore

On 11 February 2016 at 00:19, Carl Baldwin  wrote:
On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> Technically we can make this as sophisticated and seamless as we want,  
but
> this is a one-off, once it's done the pain goes away, and we won't be  
doing

> another migration like this ever again. So I wouldn't over engineer it.

Frankly, I was worried that going the other way was over-engineering
it.  It will be more difficult for us to manage this transition.

I'm still struggling to see what makes this particular migration
different than other cases where we change the database schema and the
code a bit and we automatically migrate everyone to it as part of the
routine migration.  What is it about this case that necessitates
giving the operator the option?

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Salvatore Orlando 
The difference lies in the process in my opinion.
If the switch is added into the migration path then we will tell operators
when to switch.
I was suggesting doing it manual because we just don't know if every
operator is happy about doing the switch when upgrading to Newton, but
perhaps it is just me over-worrying about operator behaviour.

The other aspect is the deprecation process. If you add the switch into the
DB migration path then the whole deprecation becomes superseded as the old
IPAM logic should be abandoned immediately after that. But perhaps the
other way of looking at it is that we should make an exception in the
deprecation process.

Salvatore

On 11 February 2016 at 00:19, Carl Baldwin  wrote:

> On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> > Technically we can make this as sophisticated and seamless as we want,
> but
> > this is a one-off, once it's done the pain goes away, and we won't be
> doing
> > another migration like this ever again. So I wouldn't over engineer it.
>
> Frankly, I was worried that going the other way was over-engineering
> it.  It will be more difficult for us to manage this transition.
>
> I'm still struggling to see what makes this particular migration
> different than other cases where we change the database schema and the
> code a bit and we automatically migrate everyone to it as part of the
> routine migration.  What is it about this case that necessitates
> giving the operator the option?
>
> Carl
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Armando M. 
On 10 February 2016 at 15:19, Carl Baldwin  wrote:

> On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> > Technically we can make this as sophisticated and seamless as we want,
> but
> > this is a one-off, once it's done the pain goes away, and we won't be
> doing
> > another migration like this ever again. So I wouldn't over engineer it.
>
> Frankly, I was worried that going the other way was over-engineering
> it.  It will be more difficult for us to manage this transition.
>
> I'm still struggling to see what makes this particular migration
> different than other cases where we change the database schema and the
> code a bit and we automatically migrate everyone to it as part of the
> routine migration.  What is it about this case that necessitates
> giving the operator the option?
>

I believe we have more recovery options out a potentially fatal situation.
In fact the offline script can provide a dry-run option that can just
validate that the migration will succeed before it is even actually
performed; I think that the size and the amount of tables involved in the
data migration justifies this course of action rather than the other. Think
about what Sean said, bugs are always lurking in the dark and as much as we
can strive for correctness, things might go bad. This is not a routine
migration and some operators may not be in a rush to embrace pluggable
IPAM, hence I don't think we are in the position to make the decision on
their behalf and go through the usual fast-path deprecation process.


>
> Carl
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-11 Thread Armando M. 
On 11 February 2016 at 07:01, John Belamaric 
wrote:

>
>
> ---
> John Belamaric
> (240) 383-6963
>
> > On Feb 11, 2016, at 5:37 AM, Ihar Hrachyshka 
> wrote:
> >
> > What’s the user visible change in behaviour after the switch? If it’s
> only internal implementation change, I don’t see why we want to leave the
> choice to operators.
> >
>
> It is only internal implementation changes.
>

That's not entirely true, is it? There are config variables to change and
it opens up the possibility of a scenario that the operator may not care
about.


>
>
> >> The other aspect is the deprecation process. If you add the switch into
> the DB migration path then the whole deprecation becomes superseded as the
> old IPAM logic should be abandoned immediately after that. But perhaps the
> other way of looking at it is that we should make an exception in the
> deprecation process.
> >>
> >> Salvatore
> >>
> >> On 11 February 2016 at 00:19, Carl Baldwin  wrote:
> >> On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> >> > Technically we can make this as sophisticated and seamless as we
> want, but
> >> > this is a one-off, once it's done the pain goes away, and we won't be
> doing
> >> > another migration like this ever again. So I wouldn't over engineer
> it.
> >>
> >> Frankly, I was worried that going the other way was over-engineering
> >> it.  It will be more difficult for us to manage this transition.
> >>
> >> I'm still struggling to see what makes this particular migration
> >> different than other cases where we change the database schema and the
> >> code a bit and we automatically migrate everyone to it as part of the
> >> routine migration.  What is it about this case that necessitates
> >> giving the operator the option?
> >>
> >> Carl
> >>
> >>
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >>
> >>
> __
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-10 Thread Carl Baldwin 
On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> as for c) I think it's a little late to make pluggable ipam default in
> Mitaka; I'd rather switch defaults early in the cycle (depending on the
> entity of the config) and this one seems serious enough that I'd rather have
> enough exercising in the gate to prove it solid. In a nutshell: let's defer
> the driver switch to N. When we do, we'll have to worry about grenade, but
> Grenade can run scripts and we can 'emulate' the operator hand.

Yes, it is too late.  It was wishful thinking to think that we could
do it this late.  You're good for calling us on that.  I hope that we
can have something teed up to be merged as soon as Newton development
is open so that we can put this to rest.

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-10 Thread Carl Baldwin 
On Thu, Feb 4, 2016 at 8:12 PM, Armando M.  wrote:
> Technically we can make this as sophisticated and seamless as we want, but
> this is a one-off, once it's done the pain goes away, and we won't be doing
> another migration like this ever again. So I wouldn't over engineer it.

Frankly, I was worried that going the other way was over-engineering
it.  It will be more difficult for us to manage this transition.

I'm still struggling to see what makes this particular migration
different than other cases where we change the database schema and the
code a bit and we automatically migrate everyone to it as part of the
routine migration.  What is it about this case that necessitates
giving the operator the option?

Carl

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-08 Thread Pavel Bondar 
On 06.02.2016 00:03, Salvatore Orlando wrote:
>
>
> On 5 February 2016 at 17:58, Neil Jerram  > wrote:
>
> On 05/02/16 16:31, Pavel Bondar wrote:
> > On 05.02.2016 12:28, Salvatore Orlando wrote:
> >>
> >>
> >> On 5 February 2016 at 04:12, Armando M.  
> >> >> wrote:
> >>
> >>
> >>
> >> On 4 February 2016 at 08:22, John Belamaric
> >> < >jbelama...@infoblox.com
> > wrote:
> >>
> >>
> >> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin
> 
> >> wrote:
> >> >
> >> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar
> 
> >> wrote:
> >> >> I am trying to bring more attention to [1] to make
> final decision on
> >> >> approach to use.
> >> >> There are a few point that are not 100% clear for me
> at this point.
> >> >>
> >> >> 1) Do we plan to switch all current clouds to
> pluggable ipam
> >> >> implementation in Mitaka?
>
> I possibly shouldn't comment at all, as I don't know the history, and
> wasn't around when the fundamental design decisions here were
> being made.
>
> However, it seems a shame to me that this was done in a way that
> needs a
> DB migration at all.  (And I would have thought it possible for the
> default pluggable IPAM driver to use the same DB state as the
> non-pluggable IPAM backend, given that it is delivering the same
> semantics.)  Without that, I believe it should be a no-brainer to
> switch
> unconditionally to the pluggable IPAM backend.
>
>
> This was indeed the first implementation attempt that we made, but it
> failed spectacularly as we wrestled with different foreign key
> relationships in the original and new model.
> Pavel has all the details, but after careful considerations we decided
> to adopt a specular model with different db tables.
Yeah, we had this chicken and egg problem on subnet creation.

On the one hand, ipam driver create_subnet has to be called *before*
creating neutron subnet.
Because for AnySubnetRequest ipam driver is responsible for selecting
cidr for subnet.

On the other hand, during ipam driver create_subnet call availability
ranges has to be created,
but they are linked with neutron subnet using foreign key (with
allocation pools in the middle).
So availability ranges could not be created before neutron subnet due to
FK constraint in old tables.

To solve this chicken and egg problem it was decided to use tables for
reference driver that have no FK to neutron subnet.
And it allowed to call ipam driver create_subnet (and create
availability ranges) before creating neutron subnet.
>  
>
>
> Sorry if that's unhelpful...
>
> Neil
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-08 Thread Sean M. Collins 
Salvatore Orlando wrote:
> Agreed. Operators love to automate things, but they generally don't like
> when components automatically do things they maybe do not expect to do (I
> don't think we should assume all operators fully read release notes). So
> the manual step is preferable, and not that painful after all. From an
> historical perspective, a manual switch was the same approach adopted for
> migration from OVS/LB plugins to ML2.

Honestly the migration from OVS/LB was  not very well done. 

https://bugs.launchpad.net/neutron/+bug/1424378
https://bugs.launchpad.net/neutron/+bug/1378732
https://bugs.launchpad.net/neutron/+bug/1332564 (I hit this one
personally)

Please please please please please let's put a lot of effort into making
sure this works. I beg you.

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-05 Thread Pavel Bondar 
On 05.02.2016 12:28, Salvatore Orlando wrote:
>
>
> On 5 February 2016 at 04:12, Armando M.  > wrote:
>
>
>
> On 4 February 2016 at 08:22, John Belamaric
> > wrote:
>
>
> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin
> > wrote:
> >
> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar
> > wrote:
> >> I am trying to bring more attention to [1] to make final
> decision on
> >> approach to use.
> >> There are a few point that are not 100% clear for me at
> this point.
> >>
> >> 1) Do we plan to switch all current clouds to pluggable ipam
> >> implementation in Mitaka?
> >
> > I think our plan originally was only to deprecate the
> non-pluggable
> > implementation in Mitaka and remove it in Newton.  However,
> this is
> > worth some more consideration.  The pluggable version of the
> reference
> > implementation should, in theory, be at parity with the current
> > non-pluggable implementation.  We've tested it before and shown
> > parity.  What we're missing is regular testing in the gate
> to ensure
> > it continues this way.
> >
>
> Yes, it certainly should be at parity, and gate testing to
> ensure it would be best.
>
> >> yes -->
> >> Then data migration can be done as alembic_migration and it
> is what
> >> currently implemented in [2] PS54.
> >> In this case during upgrade from Liberty to Mitaka all
> users are
> >> unconditionally switched to reference ipam driver
> >> from built-in ipam implementation.
> >> If operator wants to continue using build-in ipam
> implementation it can
> >> manually turn off ipam_driver in neutron.conf
> >> immediately after upgrade (data is not deleted from old
> tables).
> >
> > This has a certain appeal to it.  I think the migration will be
> > straight-forward since the table structure doesn't really
> change much.
> > Doing this as an alembic migration would be the easiest from an
> > upgrade point of view because it fits seamlessly in to our
> current
> > upgrade strategy.
> >
> > If we go this way, we should get this in soon so that we can
> get the
> > gate and others running with this code for the remainder of
> the cycle.
> >
>
> If we do this, and the operator reverts back to the
> non-pluggable version,
> then we will leave stale records in the new IPAM tables. At
> the very least,
> we would need a way to clean those up and to migrate at a
> later time.
>
> >> no -->
> >> Operator is free to choose whether it will switch to
> pluggable ipam
> >> implementation
> >> and when. And it leads to no automatic data migration.
> >> In this case operator is supplied with script for migration
> to pluggable
> >> ipam (and probably from pluggable ipam),
> >> which can be executed by operator during upgrade or at any
> point after
> >> upgrade is done.
> >> I was testing this approach in [2] PS53 (have unresolved
> issues in it
> >> for now).
> >
> > If there is some risk in changing over then this should still be
> > considered.  But, the more I think about it, the more I
> think that we
> > should just make the switch seamlessly for the operator and
> be done
> > with it.  This approach puts a certain burden on the operator to
> > choose when to do the migration and go through the steps
> manually to
> > do it.  And, since our intention is to deprecate and remove the
> > non-pluggable implementation, it is inevitable that they
> will have to
> > eventually switch anyway.
> >
> > This also makes testing much more difficult.  If we go this
> route, we
> > really should be testing both equally.  Does this mean that
> we need to
> > set up a whole new job to run the pluggable implementation
> along side
> > the old implementation?  This kind of feels like a nightmare
> to me.
> > What do you think?
> >
>
> Originally (as I mentioned in the meeting), I was thinking
> that we should not automatically migrate. However, I see the
> appeal of your arguments. Seamless is best, of course. But if
> we offer going back to non-pluggable, (which I think we need
> to at

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-05 Thread Neil Jerram 
On 05/02/16 16:31, Pavel Bondar wrote:
> On 05.02.2016 12:28, Salvatore Orlando wrote:
>>
>>
>> On 5 February 2016 at 04:12, Armando M. > > wrote:
>>
>>
>>
>> On 4 February 2016 at 08:22, John Belamaric
>> <jbelama...@infoblox.com> wrote:
>>
>>
>> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin > > wrote:
>> >
>> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar > > wrote:
>> >> I am trying to bring more attention to [1] to make final decision 
>> on
>> >> approach to use.
>> >> There are a few point that are not 100% clear for me at this 
>> point.
>> >>
>> >> 1) Do we plan to switch all current clouds to pluggable ipam
>> >> implementation in Mitaka?

I possibly shouldn't comment at all, as I don't know the history, and 
wasn't around when the fundamental design decisions here were being made.

However, it seems a shame to me that this was done in a way that needs a 
DB migration at all.  (And I would have thought it possible for the 
default pluggable IPAM driver to use the same DB state as the 
non-pluggable IPAM backend, given that it is delivering the same 
semantics.)  Without that, I believe it should be a no-brainer to switch 
unconditionally to the pluggable IPAM backend.

Sorry if that's unhelpful...

Neil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-05 Thread Salvatore Orlando 
On 5 February 2016 at 17:58, Neil Jerram  wrote:

> On 05/02/16 16:31, Pavel Bondar wrote:
> > On 05.02.2016 12:28, Salvatore Orlando wrote:
> >>
> >>
> >> On 5 February 2016 at 04:12, Armando M.  >> > wrote:
> >>
> >>
> >>
> >> On 4 February 2016 at 08:22, John Belamaric
> >> <jbelama...@infoblox.com> wrote:
> >>
> >>
> >> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin  > wrote:
> >> >
> >> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar <
> pbon...@infoblox.com > wrote:
> >> >> I am trying to bring more attention to [1] to make final
> decision on
> >> >> approach to use.
> >> >> There are a few point that are not 100% clear for me at this
> point.
> >> >>
> >> >> 1) Do we plan to switch all current clouds to pluggable ipam
> >> >> implementation in Mitaka?
>
> I possibly shouldn't comment at all, as I don't know the history, and
> wasn't around when the fundamental design decisions here were being made.
>
> However, it seems a shame to me that this was done in a way that needs a
> DB migration at all.  (And I would have thought it possible for the
> default pluggable IPAM driver to use the same DB state as the
> non-pluggable IPAM backend, given that it is delivering the same
> semantics.)  Without that, I believe it should be a no-brainer to switch
> unconditionally to the pluggable IPAM backend.
>

This was indeed the first implementation attempt that we made, but it
failed spectacularly as we wrestled with different foreign key
relationships in the original and new model.
Pavel has all the details, but after careful considerations we decided to
adopt a specular model with different db tables.


>
> Sorry if that's unhelpful...
>
> Neil
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-05 Thread Salvatore Orlando 
On 5 February 2016 at 04:12, Armando M.  wrote:

>
>
> On 4 February 2016 at 08:22, John Belamaric 
> wrote:
>
>>
>> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin  wrote:
>> >
>> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar 
>> wrote:
>> >> I am trying to bring more attention to [1] to make final decision on
>> >> approach to use.
>> >> There are a few point that are not 100% clear for me at this point.
>> >>
>> >> 1) Do we plan to switch all current clouds to pluggable ipam
>> >> implementation in Mitaka?
>> >
>> > I think our plan originally was only to deprecate the non-pluggable
>> > implementation in Mitaka and remove it in Newton.  However, this is
>> > worth some more consideration.  The pluggable version of the reference
>> > implementation should, in theory, be at parity with the current
>> > non-pluggable implementation.  We've tested it before and shown
>> > parity.  What we're missing is regular testing in the gate to ensure
>> > it continues this way.
>> >
>>
>> Yes, it certainly should be at parity, and gate testing to ensure it
>> would be best.
>>
>> >> yes -->
>> >> Then data migration can be done as alembic_migration and it is what
>> >> currently implemented in [2] PS54.
>> >> In this case during upgrade from Liberty to Mitaka all users are
>> >> unconditionally switched to reference ipam driver
>> >> from built-in ipam implementation.
>> >> If operator wants to continue using build-in ipam implementation it can
>> >> manually turn off ipam_driver in neutron.conf
>> >> immediately after upgrade (data is not deleted from old tables).
>> >
>> > This has a certain appeal to it.  I think the migration will be
>> > straight-forward since the table structure doesn't really change much.
>> > Doing this as an alembic migration would be the easiest from an
>> > upgrade point of view because it fits seamlessly in to our current
>> > upgrade strategy.
>> >
>> > If we go this way, we should get this in soon so that we can get the
>> > gate and others running with this code for the remainder of the cycle.
>> >
>>
>> If we do this, and the operator reverts back to the non-pluggable version,
>> then we will leave stale records in the new IPAM tables. At the very
>> least,
>> we would need a way to clean those up and to migrate at a later time.
>>
>> >> no -->
>> >> Operator is free to choose whether it will switch to pluggable ipam
>> >> implementation
>> >> and when. And it leads to no automatic data migration.
>> >> In this case operator is supplied with script for migration to
>> pluggable
>> >> ipam (and probably from pluggable ipam),
>> >> which can be executed by operator during upgrade or at any point after
>> >> upgrade is done.
>> >> I was testing this approach in [2] PS53 (have unresolved issues in it
>> >> for now).
>> >
>> > If there is some risk in changing over then this should still be
>> > considered.  But, the more I think about it, the more I think that we
>> > should just make the switch seamlessly for the operator and be done
>> > with it.  This approach puts a certain burden on the operator to
>> > choose when to do the migration and go through the steps manually to
>> > do it.  And, since our intention is to deprecate and remove the
>> > non-pluggable implementation, it is inevitable that they will have to
>> > eventually switch anyway.
>> >
>> > This also makes testing much more difficult.  If we go this route, we
>> > really should be testing both equally.  Does this mean that we need to
>> > set up a whole new job to run the pluggable implementation along side
>> > the old implementation?  This kind of feels like a nightmare to me.
>> > What do you think?
>> >
>>
>> Originally (as I mentioned in the meeting), I was thinking that we should
>> not automatically migrate. However, I see the appeal of your arguments.
>> Seamless is best, of course. But if we offer going back to non-pluggable,
>> (which I think we need to at this point in the Mitaka cycle), we probably
>> need to provide a script as mentioned above. Seems feasible, though.
>>
>>
>>
>>
> We're tackling more than one issue in this thread and I am having a hard
> time wrapping my head around it. Let me try to sum it all up.
>
> a) switching from non-pluggable to pluggable it's a matter of running a
> data migration + a config change
> b) We can either switch automatically on restart (option b1) or manually
> on operator command (b2)
> c) Do we make pluggable ipam default and when
> d) Testing the migration
> e) Deprecating the non-pluggable one.
>
> I hope we are all in agreement on bullet point a), because knowing the
> complexity of your problem is halfway to our solution.
>
> as for b) I think that manual migration is best for two reasons: 1) In HA
> scenarios, seamless upgrade (ie. on server restart) can be a challenge; 2)
> the operator must 'manually' change the driver, so he/she is very conscious
> of what he/she is doing and

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-04 Thread Armando M. 
On 4 February 2016 at 08:22, John Belamaric  wrote:

>
> > On Feb 4, 2016, at 11:09 AM, Carl Baldwin  wrote:
> >
> > On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar 
> wrote:
> >> I am trying to bring more attention to [1] to make final decision on
> >> approach to use.
> >> There are a few point that are not 100% clear for me at this point.
> >>
> >> 1) Do we plan to switch all current clouds to pluggable ipam
> >> implementation in Mitaka?
> >
> > I think our plan originally was only to deprecate the non-pluggable
> > implementation in Mitaka and remove it in Newton.  However, this is
> > worth some more consideration.  The pluggable version of the reference
> > implementation should, in theory, be at parity with the current
> > non-pluggable implementation.  We've tested it before and shown
> > parity.  What we're missing is regular testing in the gate to ensure
> > it continues this way.
> >
>
> Yes, it certainly should be at parity, and gate testing to ensure it would
> be best.
>
> >> yes -->
> >> Then data migration can be done as alembic_migration and it is what
> >> currently implemented in [2] PS54.
> >> In this case during upgrade from Liberty to Mitaka all users are
> >> unconditionally switched to reference ipam driver
> >> from built-in ipam implementation.
> >> If operator wants to continue using build-in ipam implementation it can
> >> manually turn off ipam_driver in neutron.conf
> >> immediately after upgrade (data is not deleted from old tables).
> >
> > This has a certain appeal to it.  I think the migration will be
> > straight-forward since the table structure doesn't really change much.
> > Doing this as an alembic migration would be the easiest from an
> > upgrade point of view because it fits seamlessly in to our current
> > upgrade strategy.
> >
> > If we go this way, we should get this in soon so that we can get the
> > gate and others running with this code for the remainder of the cycle.
> >
>
> If we do this, and the operator reverts back to the non-pluggable version,
> then we will leave stale records in the new IPAM tables. At the very least,
> we would need a way to clean those up and to migrate at a later time.
>
> >> no -->
> >> Operator is free to choose whether it will switch to pluggable ipam
> >> implementation
> >> and when. And it leads to no automatic data migration.
> >> In this case operator is supplied with script for migration to pluggable
> >> ipam (and probably from pluggable ipam),
> >> which can be executed by operator during upgrade or at any point after
> >> upgrade is done.
> >> I was testing this approach in [2] PS53 (have unresolved issues in it
> >> for now).
> >
> > If there is some risk in changing over then this should still be
> > considered.  But, the more I think about it, the more I think that we
> > should just make the switch seamlessly for the operator and be done
> > with it.  This approach puts a certain burden on the operator to
> > choose when to do the migration and go through the steps manually to
> > do it.  And, since our intention is to deprecate and remove the
> > non-pluggable implementation, it is inevitable that they will have to
> > eventually switch anyway.
> >
> > This also makes testing much more difficult.  If we go this route, we
> > really should be testing both equally.  Does this mean that we need to
> > set up a whole new job to run the pluggable implementation along side
> > the old implementation?  This kind of feels like a nightmare to me.
> > What do you think?
> >
>
> Originally (as I mentioned in the meeting), I was thinking that we should
> not automatically migrate. However, I see the appeal of your arguments.
> Seamless is best, of course. But if we offer going back to non-pluggable,
> (which I think we need to at this point in the Mitaka cycle), we probably
> need to provide a script as mentioned above. Seems feasible, though.
>
>
>
>
We're tackling more than one issue in this thread and I am having a hard
time wrapping my head around it. Let me try to sum it all up.

a) switching from non-pluggable to pluggable it's a matter of running a
data migration + a config change
b) We can either switch automatically on restart (option b1) or manually on
operator command (b2)
c) Do we make pluggable ipam default and when
d) Testing the migration
e) Deprecating the non-pluggable one.

I hope we are all in agreement on bullet point a), because knowing the
complexity of your problem is halfway to our solution.

as for b) I think that manual migration is best for two reasons: 1) In HA
scenarios, seamless upgrade (ie. on server restart) can be a challenge; 2)
the operator must 'manually' change the driver, so he/she is very conscious
of what he/she is doing and can take enough precautions should something go
astray. Technically we can make this as sophisticated and seamless as we
want, but this is a one-off, once it's done the pain goes away, and we
won't be

[openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-04 Thread Pavel Bondar 
Hi,

I am trying to bring more attention to [1] to make final decision on
approach to use.
There are a few point that are not 100% clear for me at this point.

1) Do we plan to switch all current clouds to pluggable ipam
implementation in Mitaka?

yes -->
Then data migration can be done as alembic_migration and it is what
currently implemented in [2] PS54.
In this case during upgrade from Liberty to Mitaka all users are
unconditionally switched to reference ipam driver
from built-in ipam implementation.
If operator wants to continue using build-in ipam implementation it can
manually turn off ipam_driver in neutron.conf
immediately after upgrade (data is not deleted from old tables).

no -->
Operator is free to choose whether it will switch to pluggable ipam
implementation
and when. And it leads to no automatic data migration.
In this case operator is supplied with script for migration to pluggable
ipam (and probably from pluggable ipam),
which can be executed by operator during upgrade or at any point after
upgrade is done.
I was testing this approach in [2] PS53 (have unresolved issues in it
for now).

Or we could do both, i.e. migrate data during upgrade from built-in to
pluggable ipam implementation
and supply operator with scripts to migrate from/to pluggable ipam at
any time after upgrade.

According to current feedback in [1] it most likely we go with script
approach,
so would like to confirm if that is the case.

2) Do we plan to make ipam implementation default in Mitaka for greenfields?

If answer for this question is the same as for previous (yes/yes,
no/no), then it doesn't introduce additional issues.
But if answer is different from previous then it might complicate stuff.
For example, greyfields might be migrated manually by operator to
pluggable ipam, or continue to work using built-in implementation after
upgrade in Mitaka.
But greenfields might be set to pluggable ipam implementation by default.

Is it what we are going to support?

3) How the script approach should be tested?

Currently if pluggable implementation is set as default, then grenade
test fails.
Data has to be migrated during upgrade automatically to make grenade pass.
In [1] PS53 I was using alembic migration that internally just call
external migrate script.
Is it a valid approach? I expect that better way to test script
execution during upgrade might exist.

[1] https://bugs.launchpad.net/neutron/+bug/1516156
[2] https://review.openstack.org/#/c/181023

Thanks,
Pavel

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-04 Thread Carl Baldwin 
On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar  wrote:
> I am trying to bring more attention to [1] to make final decision on
> approach to use.
> There are a few point that are not 100% clear for me at this point.
>
> 1) Do we plan to switch all current clouds to pluggable ipam
> implementation in Mitaka?

I think our plan originally was only to deprecate the non-pluggable
implementation in Mitaka and remove it in Newton.  However, this is
worth some more consideration.  The pluggable version of the reference
implementation should, in theory, be at parity with the current
non-pluggable implementation.  We've tested it before and shown
parity.  What we're missing is regular testing in the gate to ensure
it continues this way.

> yes -->
> Then data migration can be done as alembic_migration and it is what
> currently implemented in [2] PS54.
> In this case during upgrade from Liberty to Mitaka all users are
> unconditionally switched to reference ipam driver
> from built-in ipam implementation.
> If operator wants to continue using build-in ipam implementation it can
> manually turn off ipam_driver in neutron.conf
> immediately after upgrade (data is not deleted from old tables).

This has a certain appeal to it.  I think the migration will be
straight-forward since the table structure doesn't really change much.
Doing this as an alembic migration would be the easiest from an
upgrade point of view because it fits seamlessly in to our current
upgrade strategy.

If we go this way, we should get this in soon so that we can get the
gate and others running with this code for the remainder of the cycle.

> no -->
> Operator is free to choose whether it will switch to pluggable ipam
> implementation
> and when. And it leads to no automatic data migration.
> In this case operator is supplied with script for migration to pluggable
> ipam (and probably from pluggable ipam),
> which can be executed by operator during upgrade or at any point after
> upgrade is done.
> I was testing this approach in [2] PS53 (have unresolved issues in it
> for now).

If there is some risk in changing over then this should still be
considered.  But, the more I think about it, the more I think that we
should just make the switch seamlessly for the operator and be done
with it.  This approach puts a certain burden on the operator to
choose when to do the migration and go through the steps manually to
do it.  And, since our intention is to deprecate and remove the
non-pluggable implementation, it is inevitable that they will have to
eventually switch anyway.

This also makes testing much more difficult.  If we go this route, we
really should be testing both equally.  Does this mean that we need to
set up a whole new job to run the pluggable implementation along side
the old implementation?  This kind of feels like a nightmare to me.
What do you think?

> Or we could do both, i.e. migrate data during upgrade from built-in to
> pluggable ipam implementation
> and supply operator with scripts to migrate from/to pluggable ipam at
> any time after upgrade.
>
> According to current feedback in [1] it most likely we go with script
> approach,
> so would like to confirm if that is the case.
>
> 2) Do we plan to make ipam implementation default in Mitaka for greenfields?

I'll wait to respond to the remainder of this email until after we get
more clarity on your first question.  I'd like to hear from anyone in
the community but especially would like to hear from Salvatore, as the
author of the new implementation, and Armax, as our fearless and
beloved PTL.

Carl

> If answer for this question is the same as for previous (yes/yes,
> no/no), then it doesn't introduce additional issues.
> But if answer is different from previous then it might complicate stuff.
> For example, greyfields might be migrated manually by operator to
> pluggable ipam, or continue to work using built-in implementation after
> upgrade in Mitaka.
> But greenfields might be set to pluggable ipam implementation by default.
>
> Is it what we are going to support?
>
> 3) How the script approach should be tested?
>
> Currently if pluggable implementation is set as default, then grenade
> test fails.
> Data has to be migrated during upgrade automatically to make grenade pass.
> In [1] PS53 I was using alembic migration that internally just call
> external migrate script.
> Is it a valid approach? I expect that better way to test script
> execution during upgrade might exist.
>
> [1] https://bugs.launchpad.net/neutron/+bug/1516156
> [2] https://review.openstack.org/#/c/181023

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [ipam] Migration to pluggable IPAM

2016-02-04 Thread John Belamaric 

> On Feb 4, 2016, at 11:09 AM, Carl Baldwin  wrote:
> 
> On Thu, Feb 4, 2016 at 7:23 AM, Pavel Bondar  wrote:
>> I am trying to bring more attention to [1] to make final decision on
>> approach to use.
>> There are a few point that are not 100% clear for me at this point.
>> 
>> 1) Do we plan to switch all current clouds to pluggable ipam
>> implementation in Mitaka?
> 
> I think our plan originally was only to deprecate the non-pluggable
> implementation in Mitaka and remove it in Newton.  However, this is
> worth some more consideration.  The pluggable version of the reference
> implementation should, in theory, be at parity with the current
> non-pluggable implementation.  We've tested it before and shown
> parity.  What we're missing is regular testing in the gate to ensure
> it continues this way.
> 

Yes, it certainly should be at parity, and gate testing to ensure it would be 
best.

>> yes -->
>> Then data migration can be done as alembic_migration and it is what
>> currently implemented in [2] PS54.
>> In this case during upgrade from Liberty to Mitaka all users are
>> unconditionally switched to reference ipam driver
>> from built-in ipam implementation.
>> If operator wants to continue using build-in ipam implementation it can
>> manually turn off ipam_driver in neutron.conf
>> immediately after upgrade (data is not deleted from old tables).
> 
> This has a certain appeal to it.  I think the migration will be
> straight-forward since the table structure doesn't really change much.
> Doing this as an alembic migration would be the easiest from an
> upgrade point of view because it fits seamlessly in to our current
> upgrade strategy.
> 
> If we go this way, we should get this in soon so that we can get the
> gate and others running with this code for the remainder of the cycle.
> 

If we do this, and the operator reverts back to the non-pluggable version,
then we will leave stale records in the new IPAM tables. At the very least,
we would need a way to clean those up and to migrate at a later time.

>> no -->
>> Operator is free to choose whether it will switch to pluggable ipam
>> implementation
>> and when. And it leads to no automatic data migration.
>> In this case operator is supplied with script for migration to pluggable
>> ipam (and probably from pluggable ipam),
>> which can be executed by operator during upgrade or at any point after
>> upgrade is done.
>> I was testing this approach in [2] PS53 (have unresolved issues in it
>> for now).
> 
> If there is some risk in changing over then this should still be
> considered.  But, the more I think about it, the more I think that we
> should just make the switch seamlessly for the operator and be done
> with it.  This approach puts a certain burden on the operator to
> choose when to do the migration and go through the steps manually to
> do it.  And, since our intention is to deprecate and remove the
> non-pluggable implementation, it is inevitable that they will have to
> eventually switch anyway.
> 
> This also makes testing much more difficult.  If we go this route, we
> really should be testing both equally.  Does this mean that we need to
> set up a whole new job to run the pluggable implementation along side
> the old implementation?  This kind of feels like a nightmare to me.
> What do you think?
> 

Originally (as I mentioned in the meeting), I was thinking that we should not 
automatically migrate. However, I see the appeal of your arguments. Seamless is 
best, of course. But if we offer going back to non-pluggable, (which I think we 
need to at this point in the Mitaka cycle), we probably need to provide a 
script as mentioned above. Seems feasible, though.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev