Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jian Wen]

Our use case is like the Yahoo one.

2014-08-07 7:10 GMT+08:00 Ed Hall :

>
>
> tl;dr: we’re willing to be a use case, but our internal timeline is such
> that in all likelihood
> this will be as a post-mortem.
>
> We (Yahoo) have thousands of pets that need migrated as well as an
> unspecified
> number of cattle. A “live" strategy is strongly preferred (I’m not saying
> “live" migration
> since in our case it needs to be an in-place operation, not shuffling
> instances around).
> But several seconds of network outage? No problem. Disabling VM
> creation/deletion,
> or even the entire Nova API for a few hours? Well take the grumbling from
> our internal
> teams. A suspend/snapshot/cold-migrate would be an absolute last resort,
> and frankly
> could push back our aggressive migration timeline significantly.
>

>
+1
 We (letv.com) have thousands of cattle that need migrated as well
 as an unspecified number of pets. The zoo is growing really fast.


> As an alternative, we’re looking at DB-to-DB translation, with a one-shot
> script run on
> the compute nodes to move network taps. We’d actually worked this out back
> in the
> Quantum/Folsom era but backed off due to OVS/device driver issues (don’t
> ask -- I still
> get nightmares). This, of course, would require an API outage, and is a
> "big bang"
> approach (one of the attractions of Oleg’s approach is that we can migrate
> a few low-
> value instances and then examine results carefully before proceeding). But
> once again,
> our solution is likely to be of limited interest -- flat network without
> DHCP, no routers or
> floating IPs, unconventional (for OpenStack) use of VLANs -- though we’d
> be happy
> to share once the dust settles.
>
+1
Except that DHCP is used in one of our flat networks.

Our OpenStack distribution is based on the latest OpenStack Havana
release.


> -Ed Hall
> edh...@yahoo-inc.com
>
>
> On Aug 5, 2014, at 7:11 PM, "Joe Gordon"  wrote:
>
>
> On Aug 5, 2014 12:57 PM, "Jay Pipes"  wrote:
> >
> > On 08/05/2014 03:23 PM, Collins, Sean wrote:
> >>
> >> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
> >>>
> >>> However, I think the cost to providing that path far outweighs
> >>> the benefit in the face of other things on our plate.
> >>
> >>
> >> Perhaps those large operators that are hoping for a
> >> Nova-Network->Neutron zero-downtime live migration, could dedicate
> >> resources to this requirement? It is my direct experience that features
> >> that are important to a large organization will require resources
> >> from that very organization to be completed.
> >
> >
> > Indeed, that's partly why I called out Metacloud in the original post,
> as they were brought up as a deployer with this potential need. Please, if
> there are any other shops that:
>
> Perhaps I am not remembering all the details discussed at the nova
> mid-cycle, but Metacloud was brought up as an example company uses nova
> network and not neutron, not as a company that needs live migration. And
> that getting them to move to neutron would be a good litmus test for
> nova-network performance parity, something that is very hard to do in the
> gate.   But that was all said without any folks from Metacloud in the room,
> so we may both be wrong.
>
> >
> > * Currently deploy nova-network
> > * Need to move to Neutron
> > * Their tenants cannot tolerate any downtime due to a cold migration
> >
> > Please do comment on this thread and speak up.
> >
> > Best,
> > -jay
> >
> >
> > ___
> > OpenStack-dev mailing list
> > OpenStack-dev@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>  ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>  ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best,

Jian
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 09/08/14 05:09, Russell Bryant wrote:

On 08/06/2014 01:41 PM, Jay Pipes wrote:

On 08/06/2014 01:40 AM, Tom Fifield wrote:

On 06/08/14 13:30, Robert Collins wrote:

On 6 August 2014 17:27, Tom Fifield  wrote:

On 06/08/14 13:24, Robert Collins wrote:



What happened to your DB migrations then? :)



Sorry if I misunderstood, I thought we were talking about running VM
downtime here?


While DB migrations are running things like the nova metadata service
can/will misbehave - and user code within instances will be affected.
Thats arguably VM downtime.

OTOH you could define it more narrowly as 'VMs are not powered off' or
'VMs are not stalled for more than 2s without a time slice' etc etc -
my sense is that most users are going to be particularly concerned
about things for which they have to *do something* - e.g. VMs being
powered off or rebooted - but having no network for a short period
while vifs are replugged and the overlay network re-establishes itself
would be much less concerning.


I think you've got it there, Rob - nicely put :)

In many cases the users I've spoken to who are looking for a live path
out of nova-network on to neutron are actually completely OK with some
"API service" downtime (metadata service is an API service by their
definition). A little 'glitch' in the network is also OK for many of
them.

Contrast that with the original proposal in this thread ("snapshot VMs
in old nova-network deployment, store in Swift or something, then launch
VM from a snapshot in new Neutron deployment") - it is completely
unacceptable and is not considered a migration path for these users.


Who are these users? Can we speak with them? Would they be interested in
participating in the documentation and migration feature process?


Yes, I'd really like to see some participation in the development of a
solution if it's an important requirement.  Until then, it feels like a
case of an open question of "what do you want".  Of course the answer is
"a pony".



... and this is exactly why """...raising this concept only on a 
development mailing list is a bad idea


If anyone is serious about not providing a proper migration path for 
these users that need it, there is a need to be yelling this for 
probably a few of summits in a row and every OpenStack event we have in 
between, as well as the full gamut of periodic surveys, blogs, twitters, 
weibos, linkedins, facebooks etc,"""


So, get cracking :)


Regards,


Tom


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Chet Burgess]

On Aug 8, 2014, at 14:09 , Russell Bryant  wrote:

> On 08/06/2014 01:41 PM, Jay Pipes wrote:
>> On 08/06/2014 01:40 AM, Tom Fifield wrote:
>>> On 06/08/14 13:30, Robert Collins wrote:
 On 6 August 2014 17:27, Tom Fifield  wrote:
> On 06/08/14 13:24, Robert Collins wrote:
 
>> What happened to your DB migrations then? :)
> 
> 
> Sorry if I misunderstood, I thought we were talking about running VM
> downtime here?
 
 While DB migrations are running things like the nova metadata service
 can/will misbehave - and user code within instances will be affected.
 Thats arguably VM downtime.
 
 OTOH you could define it more narrowly as 'VMs are not powered off' or
 'VMs are not stalled for more than 2s without a time slice' etc etc -
 my sense is that most users are going to be particularly concerned
 about things for which they have to *do something* - e.g. VMs being
 powered off or rebooted - but having no network for a short period
 while vifs are replugged and the overlay network re-establishes itself
 would be much less concerning.
>>> 
>>> I think you've got it there, Rob - nicely put :)
>>> 
>>> In many cases the users I've spoken to who are looking for a live path
>>> out of nova-network on to neutron are actually completely OK with some
>>> "API service" downtime (metadata service is an API service by their
>>> definition). A little 'glitch' in the network is also OK for many of
>>> them.
>>> 
>>> Contrast that with the original proposal in this thread ("snapshot VMs
>>> in old nova-network deployment, store in Swift or something, then launch
>>> VM from a snapshot in new Neutron deployment") - it is completely
>>> unacceptable and is not considered a migration path for these users.
>> 
>> Who are these users? Can we speak with them? Would they be interested in
>> participating in the documentation and migration feature process?
> 
> Yes, I'd really like to see some participation in the development of a
> solution if it's an important requirement.  Until then, it feels like a
> case of an open question of "what do you want".  Of course the answer is
> "a pony”.

Sorry for coming late to the conversation but its been a fairly busy week and 
I’m just now getting caught up.

The short answer is that if Metacloud were to migrate from nova-network to 
Neutron we would absolutely require as non-disruptive a process as possible. 
While we espouse the ideals of cloud and cattle to our clients, we can’t 
control how they use their clouds and we have to deal with the fact that legacy 
applications (aka pets) exist and run successfully today on-top of our clouds. 

Our overall approach is guided by the following 2 principals.

Minimize the network disruption to individual VMs. Ideally this is measured in 
seconds, but during a major version upgrade (something like a conversion from 
nova-network to Neutron) 5 minutes could be tolerated. 
Never disrupt the running VM. If we can avoid having to restart the container 
in any way, we do. This is by far the most disrupt action for our clients, 
especially for “pets” so we avoid it.

As previously mentioned in the thread, actual orchestration (aka API) outage 
doesn’t matter. If we have to take a 2 hour orchestration outage, while not 
ideal, its with-in the realm of possibility. As an example, our in-place major 
version upgrades are 1 hour or less of orchestration outage, 5 minutes or less 
of network outage, and 0 VM downtime. ts also important to note that these are 
not arbitrary requirements we’ve made up. This is what we see the vast of 
majority of clients expect and in some cases require from us. I would think 
that most cloud deployment running production work loads would require a 
similar set of restrictions.

I’m really sorry I couldn’t be in Oregon last week to engage in these 
conversations. I’m happy to discuss via this list, or IRC, or in regular IRC 
meetings our thoughts around this, requirements, potential assistance we could 
provide etc.

--
Chet Burgess
Chief Architect | Metacloud, Inc.
Email: c...@metacloud.com | Tel: 855-638-2256, Ext. 2428___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Russell Bryant]

On 08/06/2014 01:41 PM, Jay Pipes wrote:
> On 08/06/2014 01:40 AM, Tom Fifield wrote:
>> On 06/08/14 13:30, Robert Collins wrote:
>>> On 6 August 2014 17:27, Tom Fifield  wrote:
 On 06/08/14 13:24, Robert Collins wrote:
>>>
> What happened to your DB migrations then? :)


 Sorry if I misunderstood, I thought we were talking about running VM
 downtime here?
>>>
>>> While DB migrations are running things like the nova metadata service
>>> can/will misbehave - and user code within instances will be affected.
>>> Thats arguably VM downtime.
>>>
>>> OTOH you could define it more narrowly as 'VMs are not powered off' or
>>> 'VMs are not stalled for more than 2s without a time slice' etc etc -
>>> my sense is that most users are going to be particularly concerned
>>> about things for which they have to *do something* - e.g. VMs being
>>> powered off or rebooted - but having no network for a short period
>>> while vifs are replugged and the overlay network re-establishes itself
>>> would be much less concerning.
>>
>> I think you've got it there, Rob - nicely put :)
>>
>> In many cases the users I've spoken to who are looking for a live path
>> out of nova-network on to neutron are actually completely OK with some
>> "API service" downtime (metadata service is an API service by their
>> definition). A little 'glitch' in the network is also OK for many of
>> them.
>>
>> Contrast that with the original proposal in this thread ("snapshot VMs
>> in old nova-network deployment, store in Swift or something, then launch
>> VM from a snapshot in new Neutron deployment") - it is completely
>> unacceptable and is not considered a migration path for these users.
> 
> Who are these users? Can we speak with them? Would they be interested in
> participating in the documentation and migration feature process?

Yes, I'd really like to see some participation in the development of a
solution if it's an important requirement.  Until then, it feels like a
case of an open question of "what do you want".  Of course the answer is
"a pony".

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Brent Eagles]

On Wed, Aug 06, 2014 at 01:40:28PM +0800, Tom Fifield wrote:

> >While DB migrations are running things like the nova metadata service
> >can/will misbehave - and user code within instances will be affected.
> >Thats arguably VM downtime.
> >
> >OTOH you could define it more narrowly as 'VMs are not powered off' or
> >'VMs are not stalled for more than 2s without a time slice' etc etc -
> >my sense is that most users are going to be particularly concerned
> >about things for which they have to *do something* - e.g. VMs being
> >powered off or rebooted - but having no network for a short period
> >while vifs are replugged and the overlay network re-establishes itself
> >would be much less concerning.
> 
> I think you've got it there, Rob - nicely put :)
> 
> In many cases the users I've spoken to who are looking for a live path out
> of nova-network on to neutron are actually completely OK with some "API
> service" downtime (metadata service is an API service by their definition).
> A little 'glitch' in the network is also OK for many of them.
> 
> Contrast that with the original proposal in this thread ("snapshot VMs in
> old nova-network deployment, store in Swift or something, then launch VM
> from a snapshot in new Neutron deployment") - it is completely unacceptable
> and is not considered a migration path for these users.
> 
> 
> Regards,
> 
> 
> Tom

I've thought about this off and on since it was brought up at summit. I
have some concerns about expectations. While I could probably rattle on,
I'll stick to the two for now.

- We need to be clear with expectations with connection resets and other
  odd connection behavior. There are some nice little gotchas for some
  applications when an IP address is moved depending on how connection
  is being used. Floating IPs could be interesting as well as
  nova-network and neutron differ quite a bit in how they are
  implemented. The ultimate effect on running applications will of
  course depend on whether or not they can handle things of that nature.
  Apps designed for failover, stale connections, etc, will probably fare
  better than those that are not. Apps designed for cattle vms probably
  will do okay too. I imagine pets will be higher risk and interestingly
  enough, they seem to be a more likely target use case. I suppose this
  falls under the category of "glitch", but the pessimist (realist?) in
  me is having a hard time that some deployments are going to run into
  problems... which is a nice segue into the next concern.

- I wonder about uncommunicated expectations with migration rollback in
  case of the "all gone to hell, we need to put it back" situation. We
  have been talking about migrating a live VM from nova-network to
  neutron, but what about the way back? Are new VM boots going to be
  prevented until an all-clear is given to prevent orphans if
  nova-network needs to be put back in place? Or are we saying it is a
  "never look back" type of deal? Has this  been discussed and all
  worked out and I just missed it? This concerns me a great deal because
  cannot imagine any of the admins I've ever worked with doing something
  without a failsafe backup to "known good" state whether the end up
  needing it or not.

I'm not convinced that these have been thoroughly considered, nor are
they addressable in the very near future. I also am *deeply* concerned
that placing significant focus on this PRIOR to achieving parity with
nova-network both in function and stability jeopardizes all. That is not
to diminish the efforts of those that have already contributed heavily
in this area. However, this work is all for nothing if we haven't
covered the necessary gaps so that the users have something to migrate
*to*. 

Cheers,

Brent

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Ed Hall]

On Aug 5, 2014, at 4:52 PM, Joshua Harlow  wrote:

> I'm pretty sure yahoo is another case, with a large set of clusters on 
> nova-network still ;)
> 
> I believe we have been active in these discussions, although I'm unsure what 
> was discussed at the meetup (being that I had planned vacation, right now 
> actually). 
> 
> Anyways I think yahoo is fine with being a use case, but I can check when I 
> get back.
> 

tl;dr: we’re willing to be a use case, but our internal timeline is such that 
in all likelihood
this will be as a post-mortem.

We (Yahoo) have thousands of pets that need migrated as well as an unspecified
number of cattle. A “live" strategy is strongly preferred (I’m not saying 
“live" migration
since in our case it needs to be an in-place operation, not shuffling instances 
around).
But several seconds of network outage? No problem. Disabling VM 
creation/deletion,
or even the entire Nova API for a few hours? Well take the grumbling from our 
internal
teams. A suspend/snapshot/cold-migrate would be an absolute last resort, and 
frankly
could push back our aggressive migration timeline significantly.

We’re interested in Oleg Bondarev’s solution, and I’ve even made some 
suggestions
in review comments as to how it can be made more “live," but it’s clear there 
are a
number of objections the greater Nova community has for it. Chief among these 
are the
addition of code and an API to Nova for what is essentially a one-shot 
operation, inability
to deal with more complicated configurations, and reliance on features only 
available in a
fresh release of libvirt. (As it turns out, only the latter affects us, but 
we’re a bit of an outlier
in the community.) It’s still under consideration for us, even if the community 
rejects the
approach.

As an alternative, we’re looking at DB-to-DB translation, with a one-shot 
script run on
the compute nodes to move network taps. We’d actually worked this out back in 
the
Quantum/Folsom era but backed off due to OVS/device driver issues (don’t ask -- 
I still
get nightmares). This, of course, would require an API outage, and is a "big 
bang"
approach (one of the attractions of Oleg’s approach is that we can migrate a 
few low-
value instances and then examine results carefully before proceeding). But once 
again,
our solution is likely to be of limited interest -- flat network without DHCP, 
no routers or
floating IPs, unconventional (for OpenStack) use of VLANs -- though we’d be 
happy
to share once the dust settles.

-Ed Hall
edh...@yahoo-inc.com

> 
> On Aug 5, 2014, at 7:11 PM, "Joe Gordon"  wrote:
> 
>> 
>> On Aug 5, 2014 12:57 PM, "Jay Pipes"  wrote:
>> >
>> > On 08/05/2014 03:23 PM, Collins, Sean wrote:
>> >>
>> >> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
>> >>>
>> >>> However, I think the cost to providing that path far outweighs
>> >>> the benefit in the face of other things on our plate.
>> >>
>> >>
>> >> Perhaps those large operators that are hoping for a
>> >> Nova-Network->Neutron zero-downtime live migration, could dedicate
>> >> resources to this requirement? It is my direct experience that features
>> >> that are important to a large organization will require resources
>> >> from that very organization to be completed.
>> >
>> >
>> > Indeed, that's partly why I called out Metacloud in the original post, as 
>> > they were brought up as a deployer with this potential need. Please, if 
>> > there are any other shops that:
>> Perhaps I am not remembering all the details discussed at the nova 
>> mid-cycle, but Metacloud was brought up as an example company uses nova 
>> network and not neutron, not as a company that needs live migration. And 
>> that getting them to move to neutron would be a good litmus test for 
>> nova-network performance parity, something that is very hard to do in the 
>> gate.   But that was all said without any folks from Metacloud in the room, 
>> so we may both be wrong.
>> 
>> >
>> > * Currently deploy nova-network
>> > * Need to move to Neutron
>> > * Their tenants cannot tolerate any downtime due to a cold migration
>> >
>> > Please do comment on this thread and speak up.
>> >
>> > Best,
>> > -jay
>> >
>> >
>> > ___
>> > OpenStack-dev mailing list
>> > OpenStack-dev@lists.openstack.org
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jay Pipes]

On 08/06/2014 01:40 AM, Tom Fifield wrote:

On 06/08/14 13:30, Robert Collins wrote:

On 6 August 2014 17:27, Tom Fifield  wrote:

On 06/08/14 13:24, Robert Collins wrote:



What happened to your DB migrations then? :)



Sorry if I misunderstood, I thought we were talking about running VM
downtime here?


While DB migrations are running things like the nova metadata service
can/will misbehave - and user code within instances will be affected.
Thats arguably VM downtime.

OTOH you could define it more narrowly as 'VMs are not powered off' or
'VMs are not stalled for more than 2s without a time slice' etc etc -
my sense is that most users are going to be particularly concerned
about things for which they have to *do something* - e.g. VMs being
powered off or rebooted - but having no network for a short period
while vifs are replugged and the overlay network re-establishes itself
would be much less concerning.


I think you've got it there, Rob - nicely put :)

In many cases the users I've spoken to who are looking for a live path
out of nova-network on to neutron are actually completely OK with some
"API service" downtime (metadata service is an API service by their
definition). A little 'glitch' in the network is also OK for many of them.

Contrast that with the original proposal in this thread ("snapshot VMs
in old nova-network deployment, store in Swift or something, then launch
VM from a snapshot in new Neutron deployment") - it is completely
unacceptable and is not considered a migration path for these users.


Who are these users? Can we speak with them? Would they be interested in 
participating in the documentation and migration feature process?


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Joe Gordon]

On Tue, Aug 5, 2014 at 8:25 PM, Tom Fifield  wrote:

> On 06/08/14 03:54, Jay Pipes wrote:
>
>> On 08/05/2014 03:23 PM, Collins, Sean wrote:
>>
>>> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
>>>
 However, I think the cost to providing that path far outweighs
 the benefit in the face of other things on our plate.

>>>
>>> Perhaps those large operators that are hoping for a
>>> Nova-Network->Neutron zero-downtime live migration, could dedicate
>>> resources to this requirement? It is my direct experience that features
>>> that are important to a large organization will require resources
>>> from that very organization to be completed.
>>>
>>
>> Indeed, that's partly why I called out Metacloud in the original post,
>> as they were brought up as a deployer with this potential need. Please,
>> if there are any other shops that:
>>
>> * Currently deploy nova-network
>> * Need to move to Neutron
>> * Their tenants cannot tolerate any downtime due to a cold migration
>>
>> Please do comment on this thread and speak up.
>>
>
> Just to chip in for the dozens of users I have personally spoken to that
> do have the requirement for nova-network to neutron migration, and would be
> adversely affected if it was not implemented prior to deprecating
> nova-network: raising this concept only on a development mailing list is a
> bad idea :)
>

The way I see it, a migration strategy shouldn't be required to make
neutron the recommended networking model in OpenStack (something that the
nova team is not comfortable saying today). But a migration strategy is
required for deprecating nova-network (starting the clock for a possible
removal date).  For deployers this comes down to two different questions:

* What networking model should be used in greenfield deployments?
* How do I migrate to the new networking solution?



> If anyone is serious about not providing a proper migration path for these
> users that need it, there is a need to be yelling this for probably a few
> of summits in a row and every OpenStack event we have in between, as well
> as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins,
> facebooks etc,
>
>
> Regards,
>
>
> Tom
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jesse Pretorius]

>
> In many cases the users I've spoken to who are looking for a live path out
> of nova-network on to neutron are actually completely OK with some "API
> service" downtime (metadata service is an API service by their definition).
> A little 'glitch' in the network is also OK for many of them.
>
> Contrast that with the original proposal in this thread ("snapshot VMs in
> old nova-network deployment, store in Swift or something, then launch VM
> from a snapshot in new Neutron deployment") - it is completely unacceptable
> and is not considered a migration path for these users.
>

There have been several discussions and ideas over time. I've tried to
collect as many as I've had exposure to on the whiteboard here:
https://blueprints.launchpad.net/neutron/+spec/nova-to-quantum-upgrade

Generally speaking we've all agreed before that while zero downtime would
be great, minimal downtime would be just fine. If it means a little API
downtime and some packet loss while an instance is replugged (maybe doing a
suspend while this happens would be safer) then it's still a big win.
Having to snap, delete and redeploy is less desirable but if there's an
automated process for that which can be controlled by the user (not the
admin) then that may be ok too.

Best regards,

Jesse
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 06/08/14 13:30, Robert Collins wrote:

On 6 August 2014 17:27, Tom Fifield  wrote:

On 06/08/14 13:24, Robert Collins wrote:



What happened to your DB migrations then? :)



Sorry if I misunderstood, I thought we were talking about running VM
downtime here?


While DB migrations are running things like the nova metadata service
can/will misbehave - and user code within instances will be affected.
Thats arguably VM downtime.

OTOH you could define it more narrowly as 'VMs are not powered off' or
'VMs are not stalled for more than 2s without a time slice' etc etc -
my sense is that most users are going to be particularly concerned
about things for which they have to *do something* - e.g. VMs being
powered off or rebooted - but having no network for a short period
while vifs are replugged and the overlay network re-establishes itself
would be much less concerning.


I think you've got it there, Rob - nicely put :)

In many cases the users I've spoken to who are looking for a live path 
out of nova-network on to neutron are actually completely OK with some 
"API service" downtime (metadata service is an API service by their 
definition). A little 'glitch' in the network is also OK for many of them.


Contrast that with the original proposal in this thread ("snapshot VMs 
in old nova-network deployment, store in Swift or something, then launch 
VM from a snapshot in new Neutron deployment") - it is completely 
unacceptable and is not considered a migration path for these users.



Regards,


Tom


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Robert Collins]

On 6 August 2014 17:27, Tom Fifield  wrote:
> On 06/08/14 13:24, Robert Collins wrote:

>> What happened to your DB migrations then? :)
>
>
> Sorry if I misunderstood, I thought we were talking about running VM
> downtime here?

While DB migrations are running things like the nova metadata service
can/will misbehave - and user code within instances will be affected.
Thats arguably VM downtime.

OTOH you could define it more narrowly as 'VMs are not powered off' or
'VMs are not stalled for more than 2s without a time slice' etc etc -
my sense is that most users are going to be particularly concerned
about things for which they have to *do something* - e.g. VMs being
powered off or rebooted - but having no network for a short period
while vifs are replugged and the overlay network re-establishes itself
would be much less concerning.

Is there an ontology somewhere where we can settle on terms like this :)

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 06/08/14 13:24, Robert Collins wrote:

On 6 August 2014 17:22, Tom Fifield  wrote:

On 06/08/14 13:18, Robert Collins wrote:


On 6 August 2014 16:57, Tom Fifield  wrote:


Note, however, that nobody is suggesting not having a migration path.
I'm just suggesting relaxing the requirement that the migration from
nova-network to neutron be without any downtime of instances.




These users do not consider that a migration path, so actually that is
what
is being suggested.



We can't even deploy an upgrade of Nova->Nova without some downtime
today



Actually, that's not true :) I've done even it personally on a production
system. Several versions ago :)


What happened to your DB migrations then? :)


Sorry if I misunderstood, I thought we were talking about running VM 
downtime here?


Regards,

Tom


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Robert Collins]

On 6 August 2014 17:22, Tom Fifield  wrote:
> On 06/08/14 13:18, Robert Collins wrote:
>>
>> On 6 August 2014 16:57, Tom Fifield  wrote:
>>
 Note, however, that nobody is suggesting not having a migration path.
 I'm just suggesting relaxing the requirement that the migration from
 nova-network to neutron be without any downtime of instances.
>>>
>>>
>>>
>>> These users do not consider that a migration path, so actually that is
>>> what
>>> is being suggested.
>>
>>
>> We can't even deploy an upgrade of Nova->Nova without some downtime
>> today
>
>
> Actually, that's not true :) I've done even it personally on a production
> system. Several versions ago :)

What happened to your DB migrations then? :)

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 06/08/14 13:18, Robert Collins wrote:

On 6 August 2014 16:57, Tom Fifield  wrote:


Note, however, that nobody is suggesting not having a migration path.
I'm just suggesting relaxing the requirement that the migration from
nova-network to neutron be without any downtime of instances.



These users do not consider that a migration path, so actually that is what
is being suggested.


We can't even deploy an upgrade of Nova->Nova without some downtime
today


Actually, that's not true :) I've done even it personally on a 
production system. Several versions ago :)


Regards,


Tom


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Robert Collins]

On 6 August 2014 16:57, Tom Fifield  wrote:

>> Note, however, that nobody is suggesting not having a migration path.
>> I'm just suggesting relaxing the requirement that the migration from
>> nova-network to neutron be without any downtime of instances.
>
>
> These users do not consider that a migration path, so actually that is what
> is being suggested.

We can't even deploy an upgrade of Nova->Nova without some downtime
today, so while I can *totally* agree with the aspiration, I'm not
sure that it should be a blocking thing - otherwise we should stop
releasing OpenStack at all for an extended period while we redo a
tonne of stuff.

-Rob


-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 06/08/14 12:40, Jay Pipes wrote:

On 08/05/2014 11:25 PM, Tom Fifield wrote:

On 06/08/14 03:54, Jay Pipes wrote:

On 08/05/2014 03:23 PM, Collins, Sean wrote:

On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:

However, I think the cost to providing that path far outweighs
the benefit in the face of other things on our plate.


Perhaps those large operators that are hoping for a
Nova-Network->Neutron zero-downtime live migration, could dedicate
resources to this requirement? It is my direct experience that features
that are important to a large organization will require resources
from that very organization to be completed.


Indeed, that's partly why I called out Metacloud in the original post,
as they were brought up as a deployer with this potential need. Please,
if there are any other shops that:

* Currently deploy nova-network
* Need to move to Neutron
* Their tenants cannot tolerate any downtime due to a cold migration

Please do comment on this thread and speak up.


Just to chip in for the dozens of users I have personally spoken to that
do have the requirement for nova-network to neutron migration, and would
be adversely affected if it was not implemented prior to deprecating
nova-network: raising this concept only on a development mailing list is
a bad idea :)

If anyone is serious about not providing a proper migration path for
these users that need it, there is a need to be yelling this for
probably a few of summits in a row and every OpenStack event we have in
between, as well as the full gamut of periodic surveys, blogs, twitters,
weibos, linkedins, facebooks etc,


Well, yes, I agree that other methods of gathering that information
would indeed be good. I'll work on that.

Note, however, that nobody is suggesting not having a migration path.
I'm just suggesting relaxing the requirement that the migration from
nova-network to neutron be without any downtime of instances.


These users do not consider that a migration path, so actually that is 
what is being suggested.



Regards,


Tom


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jay Pipes]

On 08/05/2014 11:25 PM, Tom Fifield wrote:

On 06/08/14 03:54, Jay Pipes wrote:

On 08/05/2014 03:23 PM, Collins, Sean wrote:

On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:

However, I think the cost to providing that path far outweighs
the benefit in the face of other things on our plate.


Perhaps those large operators that are hoping for a
Nova-Network->Neutron zero-downtime live migration, could dedicate
resources to this requirement? It is my direct experience that features
that are important to a large organization will require resources
from that very organization to be completed.


Indeed, that's partly why I called out Metacloud in the original post,
as they were brought up as a deployer with this potential need. Please,
if there are any other shops that:

* Currently deploy nova-network
* Need to move to Neutron
* Their tenants cannot tolerate any downtime due to a cold migration

Please do comment on this thread and speak up.


Just to chip in for the dozens of users I have personally spoken to that
do have the requirement for nova-network to neutron migration, and would
be adversely affected if it was not implemented prior to deprecating
nova-network: raising this concept only on a development mailing list is
a bad idea :)

If anyone is serious about not providing a proper migration path for
these users that need it, there is a need to be yelling this for
probably a few of summits in a row and every OpenStack event we have in
between, as well as the full gamut of periodic surveys, blogs, twitters,
weibos, linkedins, facebooks etc,


Well, yes, I agree that other methods of gathering that information 
would indeed be good. I'll work on that.


Note, however, that nobody is suggesting not having a migration path. 
I'm just suggesting relaxing the requirement that the migration from 
nova-network to neutron be without any downtime of instances.


Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Tom Fifield]

On 06/08/14 03:54, Jay Pipes wrote:

On 08/05/2014 03:23 PM, Collins, Sean wrote:

On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:

However, I think the cost to providing that path far outweighs
the benefit in the face of other things on our plate.


Perhaps those large operators that are hoping for a
Nova-Network->Neutron zero-downtime live migration, could dedicate
resources to this requirement? It is my direct experience that features
that are important to a large organization will require resources
from that very organization to be completed.


Indeed, that's partly why I called out Metacloud in the original post,
as they were brought up as a deployer with this potential need. Please,
if there are any other shops that:

* Currently deploy nova-network
* Need to move to Neutron
* Their tenants cannot tolerate any downtime due to a cold migration

Please do comment on this thread and speak up.


Just to chip in for the dozens of users I have personally spoken to that 
do have the requirement for nova-network to neutron migration, and would 
be adversely affected if it was not implemented prior to deprecating 
nova-network: raising this concept only on a development mailing list is 
a bad idea :)


If anyone is serious about not providing a proper migration path for 
these users that need it, there is a need to be yelling this for 
probably a few of summits in a row and every OpenStack event we have in 
between, as well as the full gamut of periodic surveys, blogs, twitters, 
weibos, linkedins, facebooks etc,



Regards,


Tom

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Joshua Harlow]

I'm pretty sure yahoo is another case, with a large set of clusters on 
nova-network still ;)

I believe we have been active in these discussions, although I'm unsure what 
was discussed at the meetup (being that I had planned vacation, right now 
actually).

Anyways I think yahoo is fine with being a use case, but I can check when I get 
back.

Sent from my really tiny device...

On Aug 5, 2014, at 7:11 PM, "Joe Gordon" 
mailto:joe.gord...@gmail.com>> wrote:


On Aug 5, 2014 12:57 PM, "Jay Pipes" 
mailto:jaypi...@gmail.com>> wrote:
>
> On 08/05/2014 03:23 PM, Collins, Sean wrote:
>>
>> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
>>>
>>> However, I think the cost to providing that path far outweighs
>>> the benefit in the face of other things on our plate.
>>
>>
>> Perhaps those large operators that are hoping for a
>> Nova-Network->Neutron zero-downtime live migration, could dedicate
>> resources to this requirement? It is my direct experience that features
>> that are important to a large organization will require resources
>> from that very organization to be completed.
>
>
> Indeed, that's partly why I called out Metacloud in the original post, as 
> they were brought up as a deployer with this potential need. Please, if there 
> are any other shops that:

Perhaps I am not remembering all the details discussed at the nova mid-cycle, 
but Metacloud was brought up as an example company uses nova network and not 
neutron, not as a company that needs live migration. And that getting them to 
move to neutron would be a good litmus test for nova-network performance 
parity, something that is very hard to do in the gate.   But that was all said 
without any folks from Metacloud in the room, so we may both be wrong.

>
> * Currently deploy nova-network
> * Need to move to Neutron
> * Their tenants cannot tolerate any downtime due to a cold migration
>
> Please do comment on this thread and speak up.
>
> Best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Joe Gordon]

On Aug 5, 2014 12:57 PM, "Jay Pipes"  wrote:
>
> On 08/05/2014 03:23 PM, Collins, Sean wrote:
>>
>> On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
>>>
>>> However, I think the cost to providing that path far outweighs
>>> the benefit in the face of other things on our plate.
>>
>>
>> Perhaps those large operators that are hoping for a
>> Nova-Network->Neutron zero-downtime live migration, could dedicate
>> resources to this requirement? It is my direct experience that features
>> that are important to a large organization will require resources
>> from that very organization to be completed.
>
>
> Indeed, that's partly why I called out Metacloud in the original post, as
they were brought up as a deployer with this potential need. Please, if
there are any other shops that:

Perhaps I am not remembering all the details discussed at the nova
mid-cycle, but Metacloud was brought up as an example company uses nova
network and not neutron, not as a company that needs live migration. And
that getting them to move to neutron would be a good litmus test for
nova-network performance parity, something that is very hard to do in the
gate.   But that was all said without any folks from Metacloud in the room,
so we may both be wrong.

>
> * Currently deploy nova-network
> * Need to move to Neutron
> * Their tenants cannot tolerate any downtime due to a cold migration
>
> Please do comment on this thread and speak up.
>
> Best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jay Pipes]

On 08/05/2014 03:23 PM, Collins, Sean wrote:

On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:

However, I think the cost to providing that path far outweighs
the benefit in the face of other things on our plate.


Perhaps those large operators that are hoping for a
Nova-Network->Neutron zero-downtime live migration, could dedicate
resources to this requirement? It is my direct experience that features
that are important to a large organization will require resources
from that very organization to be completed.


Indeed, that's partly why I called out Metacloud in the original post, 
as they were brought up as a deployer with this potential need. Please, 
if there are any other shops that:


* Currently deploy nova-network
* Need to move to Neutron
* Their tenants cannot tolerate any downtime due to a cold migration

Please do comment on this thread and speak up.

Best,
-jay

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Collins, Sean]

On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote:
> However, I think the cost to providing that path far outweighs
> the benefit in the face of other things on our plate.

Perhaps those large operators that are hoping for a
Nova-Network->Neutron zero-downtime live migration, could dedicate
resources to this requirement? It is my direct experience that features
that are important to a large organization will require resources
from that very organization to be completed.

-- 
Sean M. Collins
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Russell Bryant]

On 08/05/2014 12:18 PM, Jay Pipes wrote:
> Hello stackers, TC, Neutron contributors,
> 
> At the Nova mid-cycle meetup last week in Oregon, during the discussion
> about the future of nova-network, the topic of nova-network -> Neutron
> migration came up.
> 
> For some reason, I had been clueless about the details of one of the
> items in the gap analysis the TC had requested [1]. Namely, the 5th
> item, about nova-network -> Neutron migration, which is detailed in the
> following specification:
> 
> https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst
> 
> The above specification outlines a plan to allow migration of *running*
> instances from an OpenStack deployment using nova-network (both with and
> without multi-host mode) to an OpenStack deployment using Neutron, with
> little to no downtime using live migration techniques and an array of
> post-vm-migrate strategies to wire up the new VIFs to the Neutron ports.
> 
> I personally believe that this requirement to support a live migration
> with no downtime of running instances between a nova-network and a
> Neutron deployment *is neither realistic, nor worth the extensive time
> and technical debt needed to make this happen*.
> 
> I suggest that it would be better to instead provide good instructions
> for doing cold migration (snapshot VMs in old nova-network deployment,
> store in Swift or something, then launch VM from a snapshot in new
> Neutron deployment) -- which should cover the majority of deployments --
> and then write some instructions for what to look out for when doing a
> custom migration for environments that simply cannot afford any downtime
> and *really* want to migrate to Neutron. For these deployments, it's
> almost guaranteed that they will need to mangle their existing databases
> and do manual data migration anyway -- like RAX did when moving from
> nova-network to Neutron. The variables are too many to list here, and
> the number of deployments actually *needing* this work seems to me to be
> very limited. Someone suggested Metacloud *might* be the only deployment
> that might meet the needs for a live nova-network -> Neutron migration.
> Metacloud folks, please do respond here!
> 
> In short, I don't think the live migration requirement for nova-network
> to Neutron is either realistic or valuable, and suggest relaxing it to
> be good instructions for cold migration of instances from an older
> deployment to a newer deployment. There are other more valuable things
> that Neutron contributors could focus on, IMO -- such as the DVR
> functionality that brings parity to Neutron with nova-network's
> multi-host mode.
> 
> Thoughts?
> 
> -jay
> 
> [1]
> https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage

Yes, I agree with what you're suggesting here.  This was the approach I
was advocating for a cycle or two ago.  In a design summit session,
there were folks that seemed to really want to go off and investigate
live migration options.  Given what has (or hasn't) been done so far, I
maintain the same opinion as you've presented here, which is that it's
really not a worthwhile investment overall.  We should just provide some
good documentation on how a cold migration would work.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Monty Taylor]

On 08/05/2014 09:34 AM, Mike Spreitzer wrote:

Monty Taylor  wrote on 08/05/2014 12:27:14 PM:


On 08/05/2014 09:18 AM, Jay Pipes wrote:

Hello stackers, TC, Neutron contributors,

At the Nova mid-cycle meetup last week in Oregon, during the

discussion

about the future of nova-network, the topic of nova-network -> Neutron
migration came up.

For some reason, I had been clueless about the details of one of the
items in the gap analysis the TC had requested [1]. Namely, the 5th
item, about nova-network -> Neutron migration, which is detailed in

the

following specification:



https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst



...

I personally believe that this requirement to support a live migration
with no downtime of running instances between a nova-network and a
Neutron deployment *is neither realistic, nor worth the extensive time
and technical debt needed to make this happen*.

I suggest that it would be better to instead provide good instructions
for doing cold migration (snapshot VMs in old nova-network deployment,
store in Swift or something, then launch VM from a snapshot in new
Neutron deployment) -- which should cover the majority of deployments

--

and then write some instructions for what to look out for when doing a
custom migration for environments that simply cannot afford any

downtime

and *really* want to migrate to Neutron. For these deployments, it's
almost guaranteed that they will need to mangle their existing

databases

and do manual data migration anyway -- like RAX did when moving from
nova-network to Neutron. The variables are too many to list here, and
the number of deployments actually *needing* this work seems to me to

be

very limited. Someone suggested Metacloud *might* be the only

deployment

that might meet the needs for a live nova-network -> Neutron

migration.

Metacloud folks, please do respond here!

...


I agree 100%. Although I understand the I think it's an unreasonably
high burden in an area where there are many many other real pressing
issues that need to be solved.


I will go a little further.  My focus is on workloads that are composed of
scaling groups (one strict way of saying "cattle not pets").  In this case
I do not need to migrate individual Compute instances, just shut down
obsolete ones and start shiny new ones.


To be complete - I feel the urge to communicate that I run a very large 
production infrastructure (that you all use) that is comprised of 
_several_ precious pets. I reject the notion that cloud is only for 
ephemeral things or that you can't do old-style workloads. It works great!


So, if I was a user of a cloud that told me I needed to do a downtime to 
migrate, it would be a bad user experience. Oh wait - it WAS a bad user 
experience when Rackspace migrated us from Rackspace Classic to 
Rackspace Nova. Guess what? We got over it - and thus far it's been the 
only time that's happened - so 4 years in to the OpenStack project, our 
control plane is still running on Rackspace.


Which is to say that there are people who will have pets and not cattle, 
and not having a magical seamless upgrade path from nova-network to 
neutron will annoy them. However, I think the cost to providing that 
path far outweighs the benefit in the face of other things on our plate.


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Edgar Magana]

Jay,

I do agree with you on the focus areas. I believe Neutron should focus on
the nova-parity (DVR) and DB migrations more than ever, instead of
increasing the priority to new API such as the GBP. Actually, yesterday
Neutron IRC showed the need of having a more focused work instead of
picking in to many ³N² different areas.

The part that I disagree is with the focus on the nova-network -> Neutron
migration. I feel this activity is under control and even that it will not
deliver the ³no-downtime² expect ion, it will offer an alternative to
migration instances for those operators that could be interested.

Now, if Metacloud has a process that will work, please share it and let¹s
document it. The more the merrier, it will be up to the operators to chose
the best approach for their own clouds.

Edgar

On 8/5/14, 9:27 AM, "Monty Taylor"  wrote:

>On 08/05/2014 09:18 AM, Jay Pipes wrote:
>> Hello stackers, TC, Neutron contributors,
>>
>> At the Nova mid-cycle meetup last week in Oregon, during the discussion
>> about the future of nova-network, the topic of nova-network -> Neutron
>> migration came up.
>>
>> For some reason, I had been clueless about the details of one of the
>> items in the gap analysis the TC had requested [1]. Namely, the 5th
>> item, about nova-network -> Neutron migration, which is detailed in the
>> following specification:
>>
>> 
>>https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.r
>>st
>>
>> The above specification outlines a plan to allow migration of *running*
>> instances from an OpenStack deployment using nova-network (both with and
>> without multi-host mode) to an OpenStack deployment using Neutron, with
>> little to no downtime using live migration techniques and an array of
>> post-vm-migrate strategies to wire up the new VIFs to the Neutron ports.
>>
>> I personally believe that this requirement to support a live migration
>> with no downtime of running instances between a nova-network and a
>> Neutron deployment *is neither realistic, nor worth the extensive time
>> and technical debt needed to make this happen*.
>>
>> I suggest that it would be better to instead provide good instructions
>> for doing cold migration (snapshot VMs in old nova-network deployment,
>> store in Swift or something, then launch VM from a snapshot in new
>> Neutron deployment) -- which should cover the majority of deployments --
>> and then write some instructions for what to look out for when doing a
>> custom migration for environments that simply cannot afford any downtime
>> and *really* want to migrate to Neutron. For these deployments, it's
>> almost guaranteed that they will need to mangle their existing databases
>> and do manual data migration anyway -- like RAX did when moving from
>> nova-network to Neutron. The variables are too many to list here, and
>> the number of deployments actually *needing* this work seems to me to be
>> very limited. Someone suggested Metacloud *might* be the only deployment
>> that might meet the needs for a live nova-network -> Neutron migration.
>> Metacloud folks, please do respond here!
>>
>> In short, I don't think the live migration requirement for nova-network
>> to Neutron is either realistic or valuable, and suggest relaxing it to
>> be good instructions for cold migration of instances from an older
>> deployment to a newer deployment. There are other more valuable things
>> that Neutron contributors could focus on, IMO -- such as the DVR
>> functionality that brings parity to Neutron with nova-network's
>> multi-host mode.
>>
>> Thoughts?
>
>I agree 100%. Although I understand the I think it's an unreasonably
>high burden in an area where there are many many other real pressing
>issues that need to be solved.
>
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Mike Spreitzer]

Monty Taylor  wrote on 08/05/2014 12:27:14 PM:

> On 08/05/2014 09:18 AM, Jay Pipes wrote:
> > Hello stackers, TC, Neutron contributors,
> >
> > At the Nova mid-cycle meetup last week in Oregon, during the 
discussion
> > about the future of nova-network, the topic of nova-network -> Neutron
> > migration came up.
> >
> > For some reason, I had been clueless about the details of one of the
> > items in the gap analysis the TC had requested [1]. Namely, the 5th
> > item, about nova-network -> Neutron migration, which is detailed in 
the
> > following specification:
> >
> > 
https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst

> >
> > ...
> >
> > I personally believe that this requirement to support a live migration
> > with no downtime of running instances between a nova-network and a
> > Neutron deployment *is neither realistic, nor worth the extensive time
> > and technical debt needed to make this happen*.
> >
> > I suggest that it would be better to instead provide good instructions
> > for doing cold migration (snapshot VMs in old nova-network deployment,
> > store in Swift or something, then launch VM from a snapshot in new
> > Neutron deployment) -- which should cover the majority of deployments 
--
> > and then write some instructions for what to look out for when doing a
> > custom migration for environments that simply cannot afford any 
downtime
> > and *really* want to migrate to Neutron. For these deployments, it's
> > almost guaranteed that they will need to mangle their existing 
databases
> > and do manual data migration anyway -- like RAX did when moving from
> > nova-network to Neutron. The variables are too many to list here, and
> > the number of deployments actually *needing* this work seems to me to 
be
> > very limited. Someone suggested Metacloud *might* be the only 
deployment
> > that might meet the needs for a live nova-network -> Neutron 
migration.
> > Metacloud folks, please do respond here!
> >
> > ...
> 
> I agree 100%. Although I understand the I think it's an unreasonably 
> high burden in an area where there are many many other real pressing 
> issues that need to be solved.

I will go a little further.  My focus is on workloads that are composed of 
scaling groups (one strict way of saying "cattle not pets").  In this case 
I do not need to migrate individual Compute instances, just shut down 
obsolete ones and start shiny new ones.

Regards,
Mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Monty Taylor]

On 08/05/2014 09:18 AM, Jay Pipes wrote:

Hello stackers, TC, Neutron contributors,

At the Nova mid-cycle meetup last week in Oregon, during the discussion
about the future of nova-network, the topic of nova-network -> Neutron
migration came up.

For some reason, I had been clueless about the details of one of the
items in the gap analysis the TC had requested [1]. Namely, the 5th
item, about nova-network -> Neutron migration, which is detailed in the
following specification:

https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst

The above specification outlines a plan to allow migration of *running*
instances from an OpenStack deployment using nova-network (both with and
without multi-host mode) to an OpenStack deployment using Neutron, with
little to no downtime using live migration techniques and an array of
post-vm-migrate strategies to wire up the new VIFs to the Neutron ports.

I personally believe that this requirement to support a live migration
with no downtime of running instances between a nova-network and a
Neutron deployment *is neither realistic, nor worth the extensive time
and technical debt needed to make this happen*.

I suggest that it would be better to instead provide good instructions
for doing cold migration (snapshot VMs in old nova-network deployment,
store in Swift or something, then launch VM from a snapshot in new
Neutron deployment) -- which should cover the majority of deployments --
and then write some instructions for what to look out for when doing a
custom migration for environments that simply cannot afford any downtime
and *really* want to migrate to Neutron. For these deployments, it's
almost guaranteed that they will need to mangle their existing databases
and do manual data migration anyway -- like RAX did when moving from
nova-network to Neutron. The variables are too many to list here, and
the number of deployments actually *needing* this work seems to me to be
very limited. Someone suggested Metacloud *might* be the only deployment
that might meet the needs for a live nova-network -> Neutron migration.
Metacloud folks, please do respond here!

In short, I don't think the live migration requirement for nova-network
to Neutron is either realistic or valuable, and suggest relaxing it to
be good instructions for cold migration of instances from an older
deployment to a newer deployment. There are other more valuable things
that Neutron contributors could focus on, IMO -- such as the DVR
functionality that brings parity to Neutron with nova-network's
multi-host mode.

Thoughts?


I agree 100%. Although I understand the I think it's an unreasonably 
high burden in an area where there are many many other real pressing 
issues that need to be solved.



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Nova][Neutron][Technical Committee] nova-network -> Neutron. Throwing a wrench in the Neutron gap analysis

[Jay Pipes]

Hello stackers, TC, Neutron contributors,

At the Nova mid-cycle meetup last week in Oregon, during the discussion 
about the future of nova-network, the topic of nova-network -> Neutron 
migration came up.


For some reason, I had been clueless about the details of one of the 
items in the gap analysis the TC had requested [1]. Namely, the 5th 
item, about nova-network -> Neutron migration, which is detailed in the 
following specification:


https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst

The above specification outlines a plan to allow migration of *running* 
instances from an OpenStack deployment using nova-network (both with and 
without multi-host mode) to an OpenStack deployment using Neutron, with 
little to no downtime using live migration techniques and an array of 
post-vm-migrate strategies to wire up the new VIFs to the Neutron ports.


I personally believe that this requirement to support a live migration 
with no downtime of running instances between a nova-network and a 
Neutron deployment *is neither realistic, nor worth the extensive time 
and technical debt needed to make this happen*.


I suggest that it would be better to instead provide good instructions 
for doing cold migration (snapshot VMs in old nova-network deployment, 
store in Swift or something, then launch VM from a snapshot in new 
Neutron deployment) -- which should cover the majority of deployments -- 
and then write some instructions for what to look out for when doing a 
custom migration for environments that simply cannot afford any downtime 
and *really* want to migrate to Neutron. For these deployments, it's 
almost guaranteed that they will need to mangle their existing databases 
and do manual data migration anyway -- like RAX did when moving from 
nova-network to Neutron. The variables are too many to list here, and 
the number of deployments actually *needing* this work seems to me to be 
very limited. Someone suggested Metacloud *might* be the only deployment 
that might meet the needs for a live nova-network -> Neutron migration. 
Metacloud folks, please do respond here!


In short, I don't think the live migration requirement for nova-network 
to Neutron is either realistic or valuable, and suggest relaxing it to 
be good instructions for cold migration of instances from an older 
deployment to a newer deployment. There are other more valuable things 
that Neutron contributors could focus on, IMO -- such as the DVR 
functionality that brings parity to Neutron with nova-network's 
multi-host mode.


Thoughts?

-jay

[1] 
https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev