Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 09/08/14 05:09, Russell Bryant wrote: On 08/06/2014 01:41 PM, Jay Pipes wrote: On 08/06/2014 01:40 AM, Tom Fifield wrote: On 06/08/14 13:30, Robert Collins wrote: On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Who are these users? Can we speak with them? Would they be interested in participating in the documentation and migration feature process? Yes, I'd really like to see some participation in the development of a solution if it's an important requirement. Until then, it feels like a case of an open question of what do you want. Of course the answer is a pony. ... and this is exactly why ...raising this concept only on a development mailing list is a bad idea If anyone is serious about not providing a proper migration path for these users that need it, there is a need to be yelling this for probably a few of summits in a row and every OpenStack event we have in between, as well as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins, facebooks etc, So, get cracking :) Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Wed, Aug 06, 2014 at 01:40:28PM +0800, Tom Fifield wrote: snip/ While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Regards, Tom I've thought about this off and on since it was brought up at summit. I have some concerns about expectations. While I could probably rattle on, I'll stick to the two for now. - We need to be clear with expectations with connection resets and other odd connection behavior. There are some nice little gotchas for some applications when an IP address is moved depending on how connection is being used. Floating IPs could be interesting as well as nova-network and neutron differ quite a bit in how they are implemented. The ultimate effect on running applications will of course depend on whether or not they can handle things of that nature. Apps designed for failover, stale connections, etc, will probably fare better than those that are not. Apps designed for cattle vms probably will do okay too. I imagine pets will be higher risk and interestingly enough, they seem to be a more likely target use case. I suppose this falls under the category of glitch, but the pessimist (realist?) in me is having a hard time that some deployments are going to run into problems... which is a nice segue into the next concern. - I wonder about uncommunicated expectations with migration rollback in case of the all gone to hell, we need to put it back situation. We have been talking about migrating a live VM from nova-network to neutron, but what about the way back? Are new VM boots going to be prevented until an all-clear is given to prevent orphans if nova-network needs to be put back in place? Or are we saying it is a never look back type of deal? Has this been discussed and all worked out and I just missed it? This concerns me a great deal because cannot imagine any of the admins I've ever worked with doing something without a failsafe backup to known good state whether the end up needing it or not. I'm not convinced that these have been thoroughly considered, nor are they addressable in the very near future. I also am *deeply* concerned that placing significant focus on this PRIOR to achieving parity with nova-network both in function and stability jeopardizes all. That is not to diminish the efforts of those that have already contributed heavily in this area. However, this work is all for nothing if we haven't covered the necessary gaps so that the users have something to migrate *to*. Cheers, Brent ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/06/2014 01:41 PM, Jay Pipes wrote: On 08/06/2014 01:40 AM, Tom Fifield wrote: On 06/08/14 13:30, Robert Collins wrote: On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Who are these users? Can we speak with them? Would they be interested in participating in the documentation and migration feature process? Yes, I'd really like to see some participation in the development of a solution if it's an important requirement. Until then, it feels like a case of an open question of what do you want. Of course the answer is a pony. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Aug 8, 2014, at 14:09 , Russell Bryant rbry...@redhat.com wrote: On 08/06/2014 01:41 PM, Jay Pipes wrote: On 08/06/2014 01:40 AM, Tom Fifield wrote: On 06/08/14 13:30, Robert Collins wrote: On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Who are these users? Can we speak with them? Would they be interested in participating in the documentation and migration feature process? Yes, I'd really like to see some participation in the development of a solution if it's an important requirement. Until then, it feels like a case of an open question of what do you want. Of course the answer is a pony”. Sorry for coming late to the conversation but its been a fairly busy week and I’m just now getting caught up. The short answer is that if Metacloud were to migrate from nova-network to Neutron we would absolutely require as non-disruptive a process as possible. While we espouse the ideals of cloud and cattle to our clients, we can’t control how they use their clouds and we have to deal with the fact that legacy applications (aka pets) exist and run successfully today on-top of our clouds. Our overall approach is guided by the following 2 principals. Minimize the network disruption to individual VMs. Ideally this is measured in seconds, but during a major version upgrade (something like a conversion from nova-network to Neutron) 5 minutes could be tolerated. Never disrupt the running VM. If we can avoid having to restart the container in any way, we do. This is by far the most disrupt action for our clients, especially for “pets” so we avoid it. As previously mentioned in the thread, actual orchestration (aka API) outage doesn’t matter. If we have to take a 2 hour orchestration outage, while not ideal, its with-in the realm of possibility. As an example, our in-place major version upgrades are 1 hour or less of orchestration outage, 5 minutes or less of network outage, and 0 VM downtime. ts also important to note that these are not arbitrary requirements we’ve made up. This is what we see the vast of majority of clients expect and in some cases require from us. I would think that most cloud deployment running production work loads would require a similar set of restrictions. I’m really sorry I couldn’t be in Oregon last week to engage in these conversations. I’m happy to discuss via this list, or IRC, or in regular IRC meetings our thoughts around this, requirements, potential assistance we could provide etc. -- Chet Burgess Chief Architect | Metacloud, Inc. Email: c...@metacloud.com | Tel: 855-638-2256, Ext. 2428___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. There have been several discussions and ideas over time. I've tried to collect as many as I've had exposure to on the whiteboard here: https://blueprints.launchpad.net/neutron/+spec/nova-to-quantum-upgrade Generally speaking we've all agreed before that while zero downtime would be great, minimal downtime would be just fine. If it means a little API downtime and some packet loss while an instance is replugged (maybe doing a suspend while this happens would be safer) then it's still a big win. Having to snap, delete and redeploy is less desirable but if there's an automated process for that which can be controlled by the user (not the admin) then that may be ok too. Best regards, Jesse ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Tue, Aug 5, 2014 at 8:25 PM, Tom Fifield t...@openstack.org wrote: On 06/08/14 03:54, Jay Pipes wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Just to chip in for the dozens of users I have personally spoken to that do have the requirement for nova-network to neutron migration, and would be adversely affected if it was not implemented prior to deprecating nova-network: raising this concept only on a development mailing list is a bad idea :) The way I see it, a migration strategy shouldn't be required to make neutron the recommended networking model in OpenStack (something that the nova team is not comfortable saying today). But a migration strategy is required for deprecating nova-network (starting the clock for a possible removal date). For deployers this comes down to two different questions: * What networking model should be used in greenfield deployments? * How do I migrate to the new networking solution? If anyone is serious about not providing a proper migration path for these users that need it, there is a need to be yelling this for probably a few of summits in a row and every OpenStack event we have in between, as well as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins, facebooks etc, Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/06/2014 01:40 AM, Tom Fifield wrote: On 06/08/14 13:30, Robert Collins wrote: On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Who are these users? Can we speak with them? Would they be interested in participating in the documentation and migration feature process? Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Aug 5, 2014, at 4:52 PM, Joshua Harlow harlo...@yahoo-inc.com wrote: I'm pretty sure yahoo is another case, with a large set of clusters on nova-network still ;) I believe we have been active in these discussions, although I'm unsure what was discussed at the meetup (being that I had planned vacation, right now actually). Anyways I think yahoo is fine with being a use case, but I can check when I get back. tl;dr: we’re willing to be a use case, but our internal timeline is such that in all likelihood this will be as a post-mortem. We (Yahoo) have thousands of pets that need migrated as well as an unspecified number of cattle. A “live strategy is strongly preferred (I’m not saying “live migration since in our case it needs to be an in-place operation, not shuffling instances around). But several seconds of network outage? No problem. Disabling VM creation/deletion, or even the entire Nova API for a few hours? Well take the grumbling from our internal teams. A suspend/snapshot/cold-migrate would be an absolute last resort, and frankly could push back our aggressive migration timeline significantly. We’re interested in Oleg Bondarev’s solution, and I’ve even made some suggestions in review comments as to how it can be made more “live, but it’s clear there are a number of objections the greater Nova community has for it. Chief among these are the addition of code and an API to Nova for what is essentially a one-shot operation, inability to deal with more complicated configurations, and reliance on features only available in a fresh release of libvirt. (As it turns out, only the latter affects us, but we’re a bit of an outlier in the community.) It’s still under consideration for us, even if the community rejects the approach. As an alternative, we’re looking at DB-to-DB translation, with a one-shot script run on the compute nodes to move network taps. We’d actually worked this out back in the Quantum/Folsom era but backed off due to OVS/device driver issues (don’t ask -- I still get nightmares). This, of course, would require an API outage, and is a big bang approach (one of the attractions of Oleg’s approach is that we can migrate a few low- value instances and then examine results carefully before proceeding). But once again, our solution is likely to be of limited interest -- flat network without DHCP, no routers or floating IPs, unconventional (for OpenStack) use of VLANs -- though we’d be happy to share once the dust settles. -Ed Hall edh...@yahoo-inc.com On Aug 5, 2014, at 7:11 PM, Joe Gordon joe.gord...@gmail.com wrote: On Aug 5, 2014 12:57 PM, Jay Pipes jaypi...@gmail.com wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: Perhaps I am not remembering all the details discussed at the nova mid-cycle, but Metacloud was brought up as an example company uses nova network and not neutron, not as a company that needs live migration. And that getting them to move to neutron would be a good litmus test for nova-network performance parity, something that is very hard to do in the gate. But that was all said without any folks from Metacloud in the room, so we may both be wrong. * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/05/2014 09:18 AM, Jay Pipes wrote: Hello stackers, TC, Neutron contributors, At the Nova mid-cycle meetup last week in Oregon, during the discussion about the future of nova-network, the topic of nova-network - Neutron migration came up. For some reason, I had been clueless about the details of one of the items in the gap analysis the TC had requested [1]. Namely, the 5th item, about nova-network - Neutron migration, which is detailed in the following specification: https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst The above specification outlines a plan to allow migration of *running* instances from an OpenStack deployment using nova-network (both with and without multi-host mode) to an OpenStack deployment using Neutron, with little to no downtime using live migration techniques and an array of post-vm-migrate strategies to wire up the new VIFs to the Neutron ports. I personally believe that this requirement to support a live migration with no downtime of running instances between a nova-network and a Neutron deployment *is neither realistic, nor worth the extensive time and technical debt needed to make this happen*. I suggest that it would be better to instead provide good instructions for doing cold migration (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) -- which should cover the majority of deployments -- and then write some instructions for what to look out for when doing a custom migration for environments that simply cannot afford any downtime and *really* want to migrate to Neutron. For these deployments, it's almost guaranteed that they will need to mangle their existing databases and do manual data migration anyway -- like RAX did when moving from nova-network to Neutron. The variables are too many to list here, and the number of deployments actually *needing* this work seems to me to be very limited. Someone suggested Metacloud *might* be the only deployment that might meet the needs for a live nova-network - Neutron migration. Metacloud folks, please do respond here! In short, I don't think the live migration requirement for nova-network to Neutron is either realistic or valuable, and suggest relaxing it to be good instructions for cold migration of instances from an older deployment to a newer deployment. There are other more valuable things that Neutron contributors could focus on, IMO -- such as the DVR functionality that brings parity to Neutron with nova-network's multi-host mode. Thoughts? I agree 100%. Although I understand the I think it's an unreasonably high burden in an area where there are many many other real pressing issues that need to be solved. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
Monty Taylor mord...@inaugust.com wrote on 08/05/2014 12:27:14 PM: On 08/05/2014 09:18 AM, Jay Pipes wrote: Hello stackers, TC, Neutron contributors, At the Nova mid-cycle meetup last week in Oregon, during the discussion about the future of nova-network, the topic of nova-network - Neutron migration came up. For some reason, I had been clueless about the details of one of the items in the gap analysis the TC had requested [1]. Namely, the 5th item, about nova-network - Neutron migration, which is detailed in the following specification: https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst ... I personally believe that this requirement to support a live migration with no downtime of running instances between a nova-network and a Neutron deployment *is neither realistic, nor worth the extensive time and technical debt needed to make this happen*. I suggest that it would be better to instead provide good instructions for doing cold migration (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) -- which should cover the majority of deployments -- and then write some instructions for what to look out for when doing a custom migration for environments that simply cannot afford any downtime and *really* want to migrate to Neutron. For these deployments, it's almost guaranteed that they will need to mangle their existing databases and do manual data migration anyway -- like RAX did when moving from nova-network to Neutron. The variables are too many to list here, and the number of deployments actually *needing* this work seems to me to be very limited. Someone suggested Metacloud *might* be the only deployment that might meet the needs for a live nova-network - Neutron migration. Metacloud folks, please do respond here! ... I agree 100%. Although I understand the I think it's an unreasonably high burden in an area where there are many many other real pressing issues that need to be solved. I will go a little further. My focus is on workloads that are composed of scaling groups (one strict way of saying cattle not pets). In this case I do not need to migrate individual Compute instances, just shut down obsolete ones and start shiny new ones. Regards, Mike ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
Jay, I do agree with you on the focus areas. I believe Neutron should focus on the nova-parity (DVR) and DB migrations more than ever, instead of increasing the priority to new API such as the GBP. Actually, yesterday Neutron IRC showed the need of having a more focused work instead of picking in to many ³N² different areas. The part that I disagree is with the focus on the nova-network - Neutron migration. I feel this activity is under control and even that it will not deliver the ³no-downtime² expect ion, it will offer an alternative to migration instances for those operators that could be interested. Now, if Metacloud has a process that will work, please share it and let¹s document it. The more the merrier, it will be up to the operators to chose the best approach for their own clouds. Edgar On 8/5/14, 9:27 AM, Monty Taylor mord...@inaugust.com wrote: On 08/05/2014 09:18 AM, Jay Pipes wrote: Hello stackers, TC, Neutron contributors, At the Nova mid-cycle meetup last week in Oregon, during the discussion about the future of nova-network, the topic of nova-network - Neutron migration came up. For some reason, I had been clueless about the details of one of the items in the gap analysis the TC had requested [1]. Namely, the 5th item, about nova-network - Neutron migration, which is detailed in the following specification: https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.r st The above specification outlines a plan to allow migration of *running* instances from an OpenStack deployment using nova-network (both with and without multi-host mode) to an OpenStack deployment using Neutron, with little to no downtime using live migration techniques and an array of post-vm-migrate strategies to wire up the new VIFs to the Neutron ports. I personally believe that this requirement to support a live migration with no downtime of running instances between a nova-network and a Neutron deployment *is neither realistic, nor worth the extensive time and technical debt needed to make this happen*. I suggest that it would be better to instead provide good instructions for doing cold migration (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) -- which should cover the majority of deployments -- and then write some instructions for what to look out for when doing a custom migration for environments that simply cannot afford any downtime and *really* want to migrate to Neutron. For these deployments, it's almost guaranteed that they will need to mangle their existing databases and do manual data migration anyway -- like RAX did when moving from nova-network to Neutron. The variables are too many to list here, and the number of deployments actually *needing* this work seems to me to be very limited. Someone suggested Metacloud *might* be the only deployment that might meet the needs for a live nova-network - Neutron migration. Metacloud folks, please do respond here! In short, I don't think the live migration requirement for nova-network to Neutron is either realistic or valuable, and suggest relaxing it to be good instructions for cold migration of instances from an older deployment to a newer deployment. There are other more valuable things that Neutron contributors could focus on, IMO -- such as the DVR functionality that brings parity to Neutron with nova-network's multi-host mode. Thoughts? I agree 100%. Although I understand the I think it's an unreasonably high burden in an area where there are many many other real pressing issues that need to be solved. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/05/2014 09:34 AM, Mike Spreitzer wrote: Monty Taylor mord...@inaugust.com wrote on 08/05/2014 12:27:14 PM: On 08/05/2014 09:18 AM, Jay Pipes wrote: Hello stackers, TC, Neutron contributors, At the Nova mid-cycle meetup last week in Oregon, during the discussion about the future of nova-network, the topic of nova-network - Neutron migration came up. For some reason, I had been clueless about the details of one of the items in the gap analysis the TC had requested [1]. Namely, the 5th item, about nova-network - Neutron migration, which is detailed in the following specification: https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst ... I personally believe that this requirement to support a live migration with no downtime of running instances between a nova-network and a Neutron deployment *is neither realistic, nor worth the extensive time and technical debt needed to make this happen*. I suggest that it would be better to instead provide good instructions for doing cold migration (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) -- which should cover the majority of deployments -- and then write some instructions for what to look out for when doing a custom migration for environments that simply cannot afford any downtime and *really* want to migrate to Neutron. For these deployments, it's almost guaranteed that they will need to mangle their existing databases and do manual data migration anyway -- like RAX did when moving from nova-network to Neutron. The variables are too many to list here, and the number of deployments actually *needing* this work seems to me to be very limited. Someone suggested Metacloud *might* be the only deployment that might meet the needs for a live nova-network - Neutron migration. Metacloud folks, please do respond here! ... I agree 100%. Although I understand the I think it's an unreasonably high burden in an area where there are many many other real pressing issues that need to be solved. I will go a little further. My focus is on workloads that are composed of scaling groups (one strict way of saying cattle not pets). In this case I do not need to migrate individual Compute instances, just shut down obsolete ones and start shiny new ones. To be complete - I feel the urge to communicate that I run a very large production infrastructure (that you all use) that is comprised of _several_ precious pets. I reject the notion that cloud is only for ephemeral things or that you can't do old-style workloads. It works great! So, if I was a user of a cloud that told me I needed to do a downtime to migrate, it would be a bad user experience. Oh wait - it WAS a bad user experience when Rackspace migrated us from Rackspace Classic to Rackspace Nova. Guess what? We got over it - and thus far it's been the only time that's happened - so 4 years in to the OpenStack project, our control plane is still running on Rackspace. Which is to say that there are people who will have pets and not cattle, and not having a magical seamless upgrade path from nova-network to neutron will annoy them. However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/05/2014 12:18 PM, Jay Pipes wrote: Hello stackers, TC, Neutron contributors, At the Nova mid-cycle meetup last week in Oregon, during the discussion about the future of nova-network, the topic of nova-network - Neutron migration came up. For some reason, I had been clueless about the details of one of the items in the gap analysis the TC had requested [1]. Namely, the 5th item, about nova-network - Neutron migration, which is detailed in the following specification: https://review.openstack.org/#/c/101921/12/specs/juno/neutron-migration.rst The above specification outlines a plan to allow migration of *running* instances from an OpenStack deployment using nova-network (both with and without multi-host mode) to an OpenStack deployment using Neutron, with little to no downtime using live migration techniques and an array of post-vm-migrate strategies to wire up the new VIFs to the Neutron ports. I personally believe that this requirement to support a live migration with no downtime of running instances between a nova-network and a Neutron deployment *is neither realistic, nor worth the extensive time and technical debt needed to make this happen*. I suggest that it would be better to instead provide good instructions for doing cold migration (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) -- which should cover the majority of deployments -- and then write some instructions for what to look out for when doing a custom migration for environments that simply cannot afford any downtime and *really* want to migrate to Neutron. For these deployments, it's almost guaranteed that they will need to mangle their existing databases and do manual data migration anyway -- like RAX did when moving from nova-network to Neutron. The variables are too many to list here, and the number of deployments actually *needing* this work seems to me to be very limited. Someone suggested Metacloud *might* be the only deployment that might meet the needs for a live nova-network - Neutron migration. Metacloud folks, please do respond here! In short, I don't think the live migration requirement for nova-network to Neutron is either realistic or valuable, and suggest relaxing it to be good instructions for cold migration of instances from an older deployment to a newer deployment. There are other more valuable things that Neutron contributors could focus on, IMO -- such as the DVR functionality that brings parity to Neutron with nova-network's multi-host mode. Thoughts? -jay [1] https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage Yes, I agree with what you're suggesting here. This was the approach I was advocating for a cycle or two ago. In a design summit session, there were folks that seemed to really want to go off and investigate live migration options. Given what has (or hasn't) been done so far, I maintain the same opinion as you've presented here, which is that it's really not a worthwhile investment overall. We should just provide some good documentation on how a cold migration would work. -- Russell Bryant ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. -- Sean M. Collins ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On Aug 5, 2014 12:57 PM, Jay Pipes jaypi...@gmail.com wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: Perhaps I am not remembering all the details discussed at the nova mid-cycle, but Metacloud was brought up as an example company uses nova network and not neutron, not as a company that needs live migration. And that getting them to move to neutron would be a good litmus test for nova-network performance parity, something that is very hard to do in the gate. But that was all said without any folks from Metacloud in the room, so we may both be wrong. * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
I'm pretty sure yahoo is another case, with a large set of clusters on nova-network still ;) I believe we have been active in these discussions, although I'm unsure what was discussed at the meetup (being that I had planned vacation, right now actually). Anyways I think yahoo is fine with being a use case, but I can check when I get back. Sent from my really tiny device... On Aug 5, 2014, at 7:11 PM, Joe Gordon joe.gord...@gmail.commailto:joe.gord...@gmail.com wrote: On Aug 5, 2014 12:57 PM, Jay Pipes jaypi...@gmail.commailto:jaypi...@gmail.com wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: Perhaps I am not remembering all the details discussed at the nova mid-cycle, but Metacloud was brought up as an example company uses nova network and not neutron, not as a company that needs live migration. And that getting them to move to neutron would be a good litmus test for nova-network performance parity, something that is very hard to do in the gate. But that was all said without any folks from Metacloud in the room, so we may both be wrong. * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.orgmailto:OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 06/08/14 03:54, Jay Pipes wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Just to chip in for the dozens of users I have personally spoken to that do have the requirement for nova-network to neutron migration, and would be adversely affected if it was not implemented prior to deprecating nova-network: raising this concept only on a development mailing list is a bad idea :) If anyone is serious about not providing a proper migration path for these users that need it, there is a need to be yelling this for probably a few of summits in a row and every OpenStack event we have in between, as well as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins, facebooks etc, Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 08/05/2014 11:25 PM, Tom Fifield wrote: On 06/08/14 03:54, Jay Pipes wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Just to chip in for the dozens of users I have personally spoken to that do have the requirement for nova-network to neutron migration, and would be adversely affected if it was not implemented prior to deprecating nova-network: raising this concept only on a development mailing list is a bad idea :) If anyone is serious about not providing a proper migration path for these users that need it, there is a need to be yelling this for probably a few of summits in a row and every OpenStack event we have in between, as well as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins, facebooks etc, Well, yes, I agree that other methods of gathering that information would indeed be good. I'll work on that. Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 06/08/14 12:40, Jay Pipes wrote: On 08/05/2014 11:25 PM, Tom Fifield wrote: On 06/08/14 03:54, Jay Pipes wrote: On 08/05/2014 03:23 PM, Collins, Sean wrote: On Tue, Aug 05, 2014 at 12:50:45PM EDT, Monty Taylor wrote: However, I think the cost to providing that path far outweighs the benefit in the face of other things on our plate. Perhaps those large operators that are hoping for a Nova-Network-Neutron zero-downtime live migration, could dedicate resources to this requirement? It is my direct experience that features that are important to a large organization will require resources from that very organization to be completed. Indeed, that's partly why I called out Metacloud in the original post, as they were brought up as a deployer with this potential need. Please, if there are any other shops that: * Currently deploy nova-network * Need to move to Neutron * Their tenants cannot tolerate any downtime due to a cold migration Please do comment on this thread and speak up. Just to chip in for the dozens of users I have personally spoken to that do have the requirement for nova-network to neutron migration, and would be adversely affected if it was not implemented prior to deprecating nova-network: raising this concept only on a development mailing list is a bad idea :) If anyone is serious about not providing a proper migration path for these users that need it, there is a need to be yelling this for probably a few of summits in a row and every OpenStack event we have in between, as well as the full gamut of periodic surveys, blogs, twitters, weibos, linkedins, facebooks etc, Well, yes, I agree that other methods of gathering that information would indeed be good. I'll work on that. Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. These users do not consider that a migration path, so actually that is what is being suggested. Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 6 August 2014 16:57, Tom Fifield t...@openstack.org wrote: Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. These users do not consider that a migration path, so actually that is what is being suggested. We can't even deploy an upgrade of Nova-Nova without some downtime today, so while I can *totally* agree with the aspiration, I'm not sure that it should be a blocking thing - otherwise we should stop releasing OpenStack at all for an extended period while we redo a tonne of stuff. -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 06/08/14 13:18, Robert Collins wrote: On 6 August 2014 16:57, Tom Fifield t...@openstack.org wrote: Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. These users do not consider that a migration path, so actually that is what is being suggested. We can't even deploy an upgrade of Nova-Nova without some downtime today Actually, that's not true :) I've done even it personally on a production system. Several versions ago :) Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 6 August 2014 17:22, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:18, Robert Collins wrote: On 6 August 2014 16:57, Tom Fifield t...@openstack.org wrote: Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. These users do not consider that a migration path, so actually that is what is being suggested. We can't even deploy an upgrade of Nova-Nova without some downtime today Actually, that's not true :) I've done even it personally on a production system. Several versions ago :) What happened to your DB migrations then? :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 06/08/14 13:24, Robert Collins wrote: On 6 August 2014 17:22, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:18, Robert Collins wrote: On 6 August 2014 16:57, Tom Fifield t...@openstack.org wrote: Note, however, that nobody is suggesting not having a migration path. I'm just suggesting relaxing the requirement that the migration from nova-network to neutron be without any downtime of instances. These users do not consider that a migration path, so actually that is what is being suggested. We can't even deploy an upgrade of Nova-Nova without some downtime today Actually, that's not true :) I've done even it personally on a production system. Several versions ago :) What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. Is there an ontology somewhere where we can settle on terms like this :) -Rob -- Robert Collins rbtcoll...@hp.com Distinguished Technologist HP Converged Cloud ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova][Neutron][Technical Committee] nova-network - Neutron. Throwing a wrench in the Neutron gap analysis
On 06/08/14 13:30, Robert Collins wrote: On 6 August 2014 17:27, Tom Fifield t...@openstack.org wrote: On 06/08/14 13:24, Robert Collins wrote: What happened to your DB migrations then? :) Sorry if I misunderstood, I thought we were talking about running VM downtime here? While DB migrations are running things like the nova metadata service can/will misbehave - and user code within instances will be affected. Thats arguably VM downtime. OTOH you could define it more narrowly as 'VMs are not powered off' or 'VMs are not stalled for more than 2s without a time slice' etc etc - my sense is that most users are going to be particularly concerned about things for which they have to *do something* - e.g. VMs being powered off or rebooted - but having no network for a short period while vifs are replugged and the overlay network re-establishes itself would be much less concerning. I think you've got it there, Rob - nicely put :) In many cases the users I've spoken to who are looking for a live path out of nova-network on to neutron are actually completely OK with some API service downtime (metadata service is an API service by their definition). A little 'glitch' in the network is also OK for many of them. Contrast that with the original proposal in this thread (snapshot VMs in old nova-network deployment, store in Swift or something, then launch VM from a snapshot in new Neutron deployment) - it is completely unacceptable and is not considered a migration path for these users. Regards, Tom ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
