Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi again, Divakar, sorry for the delayed response! On Wed, 2014-04-09 at 14:52 +, Nandavar, Divakar Padiyar wrote: > Hi Jay, Managing multiple clusters using the "Compute Proxy" is not new > right? Prior to this "nova baremetal" driver has used this model > already. Yes, unfortunately. However, nova-baremetal has moved to the Ironic project, which has its own scheduler that is built to deal with the problem of having a single compute manager responsible for >1 set of compute resources. > Also this "Proxy Compute" model gives flexibility to deploy as many > computes required based on the requirement. For example, one can > setup one proxy compute node to manage a set of clusters and another > proxy compute to manage a separate set of clusters or launch compute > node for each of the clusters. It actually reduces the flexibility of Nova in two main ways: 1) It breaks the horizontal scale-out model that exists when a single nova-compute is responsible for a wholly-separate group of compute resources. Under the proxy compute model, if the proxy compute goes down, the whole cluster goes down. 2) It surrenders control of the VMs to an external system and in doing so, must constantly poll this external system in order to know the state of the externally-controlled resources. We do this already at the hypervisor driver level. The source of truth for VM state is the hypervisor, and nova-compute must periodically query the hypervisor for live data about the VMs running on the hypervisor. The Nova scheduler queries nova-compute workers for information about the VMs running on the host it controls. For the proxy-compute model, the Nova scheduler queries proxy-computes, which then must query some externally-controlled management platform, and then aggregate this data back up to the Nova scheduler (a job that the Nova scheduler is designed to do, not the nova-compute worker). This additional layer of complexity does not allow greater flexibility -- it instead limits the flexibility of Nova, since it changes the definition and responsibility of two of its major components: nova-compute and nova-scheduler. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
It's z/VM which you might got more information here :)
http://www.vm.ibm.com/
Best Regards!
Kevin (Chen) Ji 纪 晨
Engineer, zVM Development, CSTL
Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com
Phone: +86-10-82454158
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
From: Matthew Booth
To: "OpenStack Development Mailing List (not for usage questions)"
,
Date: 04/09/2014 07:02 PM
Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live
migration with one nova compute
On 09/04/14 07:07, Chen CH Ji wrote:
> we used to have one compute service corresponding to multiple
> hypervisors (like host and nodes concept )
> our major issue on our platform is we can't run nova-compute service on
> the hypervisor and we need to find another place to run the nova-compute
> in order to talk to
> hypervisor management API through REST API
It may not be directly relevant to this discussion, but I'm interested
to know what constraint prevents you running nova-compute on the
hypervisor.
Matt
--
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team
GPG ID: D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
<>___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
@Divakar, yes, the "Proxy Compute" model is not new, but I'm not sure if this model can be accepted by community to manage both VM and PM. Anyway, I will try to file a bp and get more comments then. Thanks. 2014-04-09 22:52 GMT+08:00 Nandavar, Divakar Padiyar < divakar.padiyar-nanda...@hp.com>: > Hi Jay, > Managing multiple clusters using the "Compute Proxy" is not new right? > Prior to this "nova baremetal" driver has used this model already. Also > this "Proxy Compute" model gives flexibility to deploy as many computes > required based on the requirement. For example, one can setup one proxy > compute node to manage a set of clusters and another proxy compute to > manage a separate set of clusters or launch compute node for each of the > clusters. > > Thanks, > Divakar > > -Original Message- > From: Jay Pipes [mailto:jaypi...@gmail.com] > Sent: Wednesday, April 09, 2014 6:23 PM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live > migration with one nova compute > Importance: High > > Hi Juan, thanks for your response. Comments inline. > > On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: > > Hi, > > > > I'm fairly new to this list, actually this is my first email sent, and > > to OpenStack in general, but I'm not new at all to VMware so I'll try > > to give you my point of view about possible use case here. > > > > Jay you are saying that by using Nova to manage ESXi hosts we don't > > need vCenter because they basically overlap in their capabilities. > > Actually, no, this is not my main point. My main point is that Nova should > not change its architecture to fit the needs of one particular host > management platform (vCenter). > > Nova should, as much as possible, communicate with vCenter to perform some > operations -- in the same way that Nova communicates with KVM or XenServer > to perform some operations. But Nova should not be re-architected (and I > believe that is what has gone on here with the code change to have one > nova-compute worker talking to multiple vCenter > clusters) just so that one particular host management scheduler/platform > (vCenter) can have all of its features exposed to Nova. > > > I agree with you to some extent, Nova may have similar capabilities > > as vCenter Server but as you know OpenStack as a full cloud solution > > adds a lot more features that vCenter lacks, like multitenancy just to > > name one. > > Sure, however, my point is that Nova shouldn't need to be re-architected > just to adhere to one particular host management platform's concepts of an > atomic provider of compute resources. > > > Also in any vSphere environment managing ESXi hosts individually, this > > is without vCenter, is completely out of the question. vCenter is the > > enabler of many vSphere features. And precisely that's is, IMHO, the > > use case of using Nova to manage vCenter to manage vSphere. Without > > vCenter we only have a bunch of hypervisors and none of the HA or DRS > > (dynamic resource balancing) capabilities that a vSphere cluster > > provides, this in my experience with vSphere users/customers is a no > > go scenario. > > Understood. Still doesn't change my opinion though :) > > Best, > -jay > > > I don't know why the decision to manage vCenter with Nova was made but > > based on the above I understand the reasoning. > > > > > > Best, > > --- > > Juan Manuel Rey > > > > @jreypo > > > > > > On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes wrote: > > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar > > wrote: > > > >> Well, it seems to me that the problem is the above > > blueprint and the code it introduced. This is an anti-feature > > IMO, and probably the best solution would be to remove the > > above code and go back to having a single >> nova-compute > > managing a single vCenter cluster, not multiple ones. > > > > > > Problem is not introduced by managing multiple clusters from > > single nova-compute proxy node. > > > > > > I strongly disagree. > > > > > Internally this proxy driver is still presenting the > > "compute-node" for each of the cluster its managing. > > > > > > In what way? > > > > > What we need to think about is applicability of the live > > mi
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
- Original Message - > I'm not writing off vCenter or its capabilities. I am arguing that the > bar for modifying a fundamental design decision in Nova -- that of being > horizontally scalable by having a single nova-compute worker responsible > for managing a single provider of compute resources -- was WAY too low, > and that this decision should be revisited in the future (and possibly > as part of the vmware driver refactoring efforts currently underway by > the good folks at RH and VMWare). +1, This is my main concern about having more than one ESX cluster under a single nova-compute agent as well. Currently it works, but it doesn't seem particularly advisable as on face value as such an architecture seems to break a number of the Nova design guidelines around high availability and fault tolerance. To me it seems like such an architecture effectively elevates nova-compute into being part of the control plane where it needs to have high availability (when discussing on IRC yesterday it seemed like this *may* be possible today but more testing is required to shake out any bugs). Now may well be the right approach *is* to make some changes to these expectations about Nova, but I think it's disingenuous to suggest that what is being suggested here isn't a significant re-architecting to resolve issues resulting from earlier hacks that allowed this functionality to work in the first place. Should be an interesting summit session. -Steve ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Jay, Managing multiple clusters using the "Compute Proxy" is not new right? Prior to this "nova baremetal" driver has used this model already. Also this "Proxy Compute" model gives flexibility to deploy as many computes required based on the requirement. For example, one can setup one proxy compute node to manage a set of clusters and another proxy compute to manage a separate set of clusters or launch compute node for each of the clusters. Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Wednesday, April 09, 2014 6:23 PM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High Hi Juan, thanks for your response. Comments inline. On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: > Hi, > > I'm fairly new to this list, actually this is my first email sent, and > to OpenStack in general, but I'm not new at all to VMware so I'll try > to give you my point of view about possible use case here. > > Jay you are saying that by using Nova to manage ESXi hosts we don't > need vCenter because they basically overlap in their capabilities. Actually, no, this is not my main point. My main point is that Nova should not change its architecture to fit the needs of one particular host management platform (vCenter). Nova should, as much as possible, communicate with vCenter to perform some operations -- in the same way that Nova communicates with KVM or XenServer to perform some operations. But Nova should not be re-architected (and I believe that is what has gone on here with the code change to have one nova-compute worker talking to multiple vCenter clusters) just so that one particular host management scheduler/platform (vCenter) can have all of its features exposed to Nova. > I agree with you to some extent, Nova may have similar capabilities > as vCenter Server but as you know OpenStack as a full cloud solution > adds a lot more features that vCenter lacks, like multitenancy just to > name one. Sure, however, my point is that Nova shouldn't need to be re-architected just to adhere to one particular host management platform's concepts of an atomic provider of compute resources. > Also in any vSphere environment managing ESXi hosts individually, this > is without vCenter, is completely out of the question. vCenter is the > enabler of many vSphere features. And precisely that's is, IMHO, the > use case of using Nova to manage vCenter to manage vSphere. Without > vCenter we only have a bunch of hypervisors and none of the HA or DRS > (dynamic resource balancing) capabilities that a vSphere cluster > provides, this in my experience with vSphere users/customers is a no > go scenario. Understood. Still doesn't change my opinion though :) Best, -jay > I don't know why the decision to manage vCenter with Nova was made but > based on the above I understand the reasoning. > > > Best, > --- > Juan Manuel Rey > > @jreypo > > > On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes wrote: > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar > wrote: > > >> Well, it seems to me that the problem is the above > blueprint and the code it introduced. This is an anti-feature > IMO, and probably the best solution would be to remove the > above code and go back to having a single >> nova-compute > managing a single vCenter cluster, not multiple ones. > > > > Problem is not introduced by managing multiple clusters from > single nova-compute proxy node. > > > I strongly disagree. > > > Internally this proxy driver is still presenting the > "compute-node" for each of the cluster its managing. > > > In what way? > > > What we need to think about is applicability of the live > migration use case when a "cluster" is modelled as a compute. > Since the "cluster" is modelled as a compute, it is assumed > that a typical use case of live-move is taken care by the > underlying "cluster" itself. With this there are other > use cases which are no-op today like host maintenance mode, > live move, setting instance affinity etc., In order to > resolve this I was thinking of > > "A way to expose operations on individual ESX Hosts like > Putting host in maintenance mode, live move, instance > affinity etc., by introducing Parent - Child compute node >
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Mon, 2014-04-07 at 15:47 +0100, Matthew Booth wrote: > On 07/04/14 06:20, Jay Pipes wrote: > > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: > Well, it seems to me that the problem is the above blueprint and the > code it introduced. This is an anti-feature IMO, and probably the best > solution would be to remove the above code and go back to having a > single >> nova-compute managing a single vCenter cluster, not multiple > ones. > >> > >> Problem is not introduced by managing multiple clusters from single > >> nova-compute proxy node. > > > > I strongly disagree. > > > >> Internally this proxy driver is still presenting the "compute-node" for > >> each of the cluster its managing. > > > > In what way? > > > >> What we need to think about is applicability of the live migration use > >> case when a "cluster" is modelled as a compute. Since the "cluster" is > >> modelled as a compute, it is assumed that a typical use case of live-move > >> is taken care by the underlying "cluster" itself. With this there > >> are other use cases which are no-op today like host maintenance mode, live > >> move, setting instance affinity etc., In order to resolve this I was > >> thinking of > >> "A way to expose operations on individual ESX Hosts like Putting host in > >> maintenance mode, live move, instance affinity etc., by introducing > >> Parent - Child compute node concept. Scheduling can be restricted to > >> Parent compute node and Child compute node can be used for providing more > >> drill down on compute and also enable additional compute operations". > >> Any thoughts on this? > > > > The fundamental problem is that hacks were put in place in order to make > > Nova defer control to vCenter, when the design of Nova and vCenter are > > not compatible, and we're paying the price for that right now. > > > > All of the operations you describe above -- putting a host in > > maintenance mode, live-migration of an instance, ensuring a new instance > > is launched near or not-near another instance -- depend on a fundamental > > design feature in Nova: that a nova-compute worker fully controls and > > manages a host that provides a place to put server instances. We have > > internal driver interfaces for the *hypervisor*, not for the *manager of > > hypervisors*, because, you know, that's what Nova does. > > I'm going to take you to task here for use of the word 'fundamental'. > What does Nova do? Apparently: 'OpenStack Nova provides a cloud > computing fabric controller, supporting a wide variety of virtualization > technologies, including KVM, Xen, LXC, VMware, and more. In addition to > its native API, it includes compatibility with the commonly encountered > Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of > Nova instances to hypervisors: that's an implementation detail. Now this > change may or may not sit well with design decisions which have been > made in the past, but the concept of managing multiple clusters from a > single Nova instance is certainly not fundamentally wrong. It may not be > pragmatic; it may require further changes to Nova which were not made, > but there is nothing about it which is fundamentally at odds with the > stated goals of the project. > > Why did I bother with that? I think it's in danger of being lost. Nova > has been around for a while now and it has a lot of code and a lot of > developers behind it. We need to remember, though, that's it's all for > nothing if nobody wants to use it. VMware is different, but not wrong. > Let's stay fresh. Please see my previous email to Juan about this. I'm not anti-VMWare. I'm just opposed to changing an important part of the implementation of Nova just so that certain vCenter operations can be supported. > > The problem with all of the vCenter stuff is that it is trying to say to > > Nova "don't worry, I got this" but unfortunately, Nova wants and needs > > to manage these things, not surrender control to a different system that > > handles orchestration and scheduling in its own unique way. > > Again, I'll flip that round. Nova *currently* manages these things, and > working efficiently with a platform which also does these things would > require rethinking some design above the driver level. It's not > something we want to do naively, which the VMware driver is suffering > from in this area. It may take time to get this right, but we shouldn't > write it off as fundamentally wrong. It's useful to users and not > fundamentally at odds with the project's goals. I'm not writing off vCenter or its capabilities. I am arguing that the bar for modifying a fundamental design decision in Nova -- that of being horizontally scalable by having a single nova-compute worker responsible for managing a single provider of compute resources -- was WAY too low, and that this decision should be revisited in the future (and possibly as part of the v
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Juan, thanks for your response. Comments inline. On Mon, 2014-04-07 at 10:22 +0200, Juan Manuel Rey wrote: > Hi, > > I'm fairly new to this list, actually this is my first email sent, and > to OpenStack in general, but I'm not new at all to VMware so I'll try > to give you my point of view about possible use case here. > > Jay you are saying that by using Nova to manage ESXi hosts we don't > need vCenter because they basically overlap in their capabilities. Actually, no, this is not my main point. My main point is that Nova should not change its architecture to fit the needs of one particular host management platform (vCenter). Nova should, as much as possible, communicate with vCenter to perform some operations -- in the same way that Nova communicates with KVM or XenServer to perform some operations. But Nova should not be re-architected (and I believe that is what has gone on here with the code change to have one nova-compute worker talking to multiple vCenter clusters) just so that one particular host management scheduler/platform (vCenter) can have all of its features exposed to Nova. > I agree with you to some extent, Nova may have similar capabilities > as vCenter Server but as you know OpenStack as a full cloud solution > adds a lot more features that vCenter lacks, like multitenancy just to > name one. Sure, however, my point is that Nova shouldn't need to be re-architected just to adhere to one particular host management platform's concepts of an atomic provider of compute resources. > Also in any vSphere environment managing ESXi hosts individually, this > is without vCenter, is completely out of the question. vCenter is the > enabler of many vSphere features. And precisely that's is, IMHO, the > use case of using Nova to manage vCenter to manage vSphere. Without > vCenter we only have a bunch of hypervisors and none of the HA or DRS > (dynamic resource balancing) capabilities that a vSphere cluster > provides, this in my experience with vSphere users/customers is a no > go scenario. Understood. Still doesn't change my opinion though :) Best, -jay > I don't know why the decision to manage vCenter with Nova was made but > based on the above I understand the reasoning. > > > Best, > --- > Juan Manuel Rey > > @jreypo > > > On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes wrote: > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar > wrote: > > >> Well, it seems to me that the problem is the above > blueprint and the code it introduced. This is an anti-feature > IMO, and probably the best solution would be to remove the > above code and go back to having a single >> nova-compute > managing a single vCenter cluster, not multiple ones. > > > > Problem is not introduced by managing multiple clusters from > single nova-compute proxy node. > > > I strongly disagree. > > > Internally this proxy driver is still presenting the > "compute-node" for each of the cluster its managing. > > > In what way? > > > What we need to think about is applicability of the live > migration use case when a "cluster" is modelled as a compute. > Since the "cluster" is modelled as a compute, it is assumed > that a typical use case of live-move is taken care by the > underlying "cluster" itself. With this there are other > use cases which are no-op today like host maintenance mode, > live move, setting instance affinity etc., In order to > resolve this I was thinking of > > "A way to expose operations on individual ESX Hosts like > Putting host in maintenance mode, live move, instance > affinity etc., by introducing Parent - Child compute node > concept. Scheduling can be restricted to Parent compute node > and Child compute node can be used for providing more drill > down on compute and also enable additional compute > operations".Any thoughts on this? > > > The fundamental problem is that hacks were put in place in > order to make > Nova defer control to vCenter, when the design of Nova and > vCenter are > not compatible, and we're paying the price for that right now. > > All of the operations you describe above -- putting a host in > maintenance mode, live-migration of an instance, ensuring a > new instance > is launched near or not-near another instance -- depend on a > fundamental > design feature in Nova: that a nova-compute worker fully > controls and > manages a host that provides a place to put server instances. > We have > internal driver interfaces for the *hypervisor*, not for the > *manager of > hypervisors*, because, you know, that'
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
2014-04-09 19:04 GMT+08:00 Matthew Booth : > On 09/04/14 07:07, Chen CH Ji wrote: > > we used to have one compute service corresponding to multiple > > hypervisors (like host and nodes concept ) > > our major issue on our platform is we can't run nova-compute service on > > the hypervisor and we need to find another place to run the nova-compute > > in order to talk to > > hypervisor management API through REST API > > It may not be directly relevant to this discussion, but I'm interested > to know what constraint prevents you running nova-compute on the > hypervisor. > Actually, VMWare has two drivers, one is ESXDriver and the other is VCDriver. When using ESXDrvier, one nova compute can only manage one ESX host, but ESXDriver do not support some advanced features such as live migration, resize etc. And this driver has been deprecated. We are now talking about VCDriver which will talk to vCenter via wsdl API and the VCDriver is intend to support all VM operations, but we need some enhancement to make VCDriver can work well for some advacend features such as live migration. > Matt > > -- > Matthew Booth, RHCA, RHCSS > Red Hat Engineering, Virtualisation Team > > GPG ID: D33C3490 > GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490 > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 09/04/14 07:07, Chen CH Ji wrote: > we used to have one compute service corresponding to multiple > hypervisors (like host and nodes concept ) > our major issue on our platform is we can't run nova-compute service on > the hypervisor and we need to find another place to run the nova-compute > in order to talk to > hypervisor management API through REST API It may not be directly relevant to this discussion, but I'm interested to know what constraint prevents you running nova-compute on the hypervisor. Matt -- Matthew Booth, RHCA, RHCSS Red Hat Engineering, Virtualisation Team GPG ID: D33C3490 GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490 ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Kevin, Thanks for the contribution. Shawn from VMware already filed a bp to export those resources https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory , but this bp might some redesign as we need to decide how we will handle configuration and convention when it comes to vSphere inventory. Yes, I think that we need support host-node mode for live migration. Divakar give some idea of "Parent - Child compute" mode, I'm just wondering what is the difference of this mode with bare-metal "host-node" mode. 2014-04-09 14:07 GMT+08:00 Chen CH Ji : > we used to have one compute service corresponding to multiple hypervisors > (like host and nodes concept ) > our major issue on our platform is we can't run nova-compute service on > the hypervisor and we need to find another place to run the nova-compute in > order to talk to > hypervisor management API through REST API > which means we have to run multiple compute service out side of our > hypervisors and it's hard for us to control the compute services at that > time, > but we have no choice since nova migration only can be migrated to another > host instead of node ,so we implement according to it > if we can support host + node, then it might be helpful for the > hypervisors with different arch > > The point is whether we are able to expose the internal (say, not only the > host concept but also the node concept ) to outside > guess live-migration is admin only feature, can we expose those node > concept to admin and let admin decide it? > > Best Regards! > > Kevin (Chen) Ji 纪 晨 > > Engineer, zVM Development, CSTL > Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com > Phone: +86-10-82454158 > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, > Beijing 100193, PRC > > [image: Inactive hide details for Jay Lau ---04/06/2014 07:02:15 PM---Hi > Divakar, Can I say that the bare metal provisioning is now usi]Jay Lau > ---04/06/2014 07:02:15 PM---Hi Divakar, Can I say that the bare metal > provisioning is now using kind of "Parent - > > From: Jay Lau > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org>, > Date: 04/06/2014 07:02 PM > > Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live > migration with one nova compute > -- > > > > Hi Divakar, > > Can I say that the bare metal provisioning is now using kind of "Parent - > Child compute" mode? I was also thinking that we can use host:node to > identify a kind of "Parent-Child" or "Hierarchy Compute". So can you please > show some difference for your "Parent - Child Compute Node" and bare metal > provisioning? > > Thanks! > > > 2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar < > *divakar.padiyar-nanda...@hp.com* >: > >>> Well, it seems to me that the problem is the above blueprint and >the code it introduced. This is an anti-feature IMO, and probably the best >solution would be to remove the above code and go back to having a single >>> nova-compute managing a single vCenter cluster, not multiple ones. > >Problem is not introduced by managing multiple clusters from single >nova-compute proxy node. Internally this proxy driver is still presenting >the "compute-node" for each of the cluster its managing.What we need to >think about is applicability of the live migration use case when a >"cluster" is modelled as a compute. Since the "cluster" is modelled as a >compute, it is assumed that a typical use case of live-move is taken care >by the underlying "cluster" itself. With this there are other use >cases which are no-op today like host maintenance mode, live move, setting >instance affinity etc., In order to resolve this I was thinking of >"A way to expose operations on individual ESX Hosts like Putting host >in maintenance mode, live move, instance affinity etc., by introducing >Parent - Child compute node concept. Scheduling can be restricted to >Parent compute node and Child compute node can be used for providing more > drill down on compute and also enable additional compute operations". > Any thoughts on this? > >Thanks, >Divakar > > >-Original Message- >From: Jay Pipes [mailto:*jaypi...@gmail.com* ] >Sent: Sunday, April 06, 2014 2:02 AM >To: *openstack-dev@lists.openstack.org* >Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live >migration with one nova compute >Imp
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
we used to have one compute service corresponding to multiple hypervisors
(like host and nodes concept )
our major issue on our platform is we can't run nova-compute service on the
hypervisor and we need to find another place to run the nova-compute in
order to talk to
hypervisor management API through REST API
which means we have to run multiple compute service out side of our
hypervisors and it's hard for us to control the compute services at that
time,
but we have no choice since nova migration only can be migrated to another
host instead of node ,so we implement according to it
if we can support host + node, then it might be helpful for the hypervisors
with different arch
The point is whether we are able to expose the internal (say, not only the
host concept but also the node concept ) to outside
guess live-migration is admin only feature, can we expose those node
concept to admin and let admin decide it?
Best Regards!
Kevin (Chen) Ji 纪 晨
Engineer, zVM Development, CSTL
Notes: Chen CH Ji/China/IBM@IBMCN Internet: jiche...@cn.ibm.com
Phone: +86-10-82454158
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District,
Beijing 100193, PRC
From: Jay Lau
To: "OpenStack Development Mailing List (not for usage questions)"
,
Date: 04/06/2014 07:02 PM
Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live
migration with one nova compute
Hi Divakar,
Can I say that the bare metal provisioning is now using kind of "Parent -
Child compute" mode? I was also thinking that we can use host:node to
identify a kind of "Parent-Child" or "Hierarchy Compute". So can you please
show some difference for your "Parent - Child Compute Node" and bare metal
provisioning?
Thanks!
2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar <
divakar.padiyar-nanda...@hp.com>:
>> Well, it seems to me that the problem is the above blueprint and the
code it introduced. This is an anti-feature IMO, and probably the best
solution would be to remove the above code and go back to having a single
>> nova-compute managing a single vCenter cluster, not multiple ones.
Problem is not introduced by managing multiple clusters from single
nova-compute proxy node. Internally this proxy driver is still
presenting the "compute-node" for each of the cluster its managing.
What we need to think about is applicability of the live migration use
case when a "cluster" is modelled as a compute. Since the "cluster" is
modelled as a compute, it is assumed that a typical use case of live-move
is taken care by the underlying "cluster" itself. With this there
are other use cases which are no-op today like host maintenance mode,
live move, setting instance affinity etc., In order to resolve this I
was thinking of
"A way to expose operations on individual ESX Hosts like Putting host in
maintenance mode, live move, instance affinity etc., by introducing
Parent - Child compute node concept. Scheduling can be restricted to
Parent compute node and Child compute node can be used for providing more
drill down on compute and also enable additional compute operations".
Any thoughts on this?
Thanks,
Divakar
-Original Message-
From: Jay Pipes [mailto:jaypi...@gmail.com]
Sent: Sunday, April 06, 2014 2:02 AM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live
migration with one nova compute
Importance: High
On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote:
>
>
>
> 2014-04-04 12:46 GMT+08:00 Jay Pipes :
> On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote:
> > Thanks Jay and Chris for the comments!
> >
> > @Jay Pipes, I think that we still need to enable "one nova
> compute
> > live migration" as one nova compute can manage multiple
> clusters and
> > VMs can be migrated between those clusters managed by one
> nova
> > compute.
>
>
> Why, though? That is what I am asking... seems to me like this
> is an
> anti-feature. What benefit does the user get from moving an
> instance
> from one VCenter cluster to another VCenter cluster if the two
> clusters
> are on the same physical machine?
> @Jay Pipes, for VMWare, one physical machine (ESX server) can only
> belong to one VCenter cluster, so we may have following scenarios.
>
> DC
> |
>
> |---Cluster1
> | |
>
> | |---host1
> |
>
> |---Cluser2
> |
>
> |---host2
>
>
> Then when using VCDriver,
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
@Matt Booth: I think you make a lot of good points, but I think the main gist of the "opposing" argument, so to speak, is that currently, the way we differentiate between potential compute resources (whether they be an individual hypervisor or a cluster) is by having each have its own compute node. I think some of the reluctance here is to change that model -- the idea that a Nova compute node represents one resource which is, for all intents and purposes, atomic to OpenStack. While I get your point that this is an implementation detail, I think it's a rather large one, and a fundamental assumption in current OpenStack code (for the most part). If we change that assumption, we shouldn't really change it piecemeal. IMHO, this model (compute nodes as "atomic" resources) fits the overall design well. That being said, I personally would not be averse to something like expanding a NUMA-style API to cover the cluster, as I think this continues to fit the existing model -- a NUMA-style API breaks down an atomic resource, so for a VMWare cluster that would allow tuning to individual hypervisors, while for an individual hypervisor that would allow tuning to individual cores, etc. Best Regards, Solly Ross - Original Message - From: "Matthew Booth" To: "OpenStack Development Mailing List (not for usage questions)" Sent: Monday, April 7, 2014 10:47:35 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute On 07/04/14 06:20, Jay Pipes wrote: > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: >>>> Well, it seems to me that the problem is the above blueprint and the code >>>> it introduced. This is an anti-feature IMO, and probably the best solution >>>> would be to remove the above code and go back to having a single >> >>>> nova-compute managing a single vCenter cluster, not multiple ones. >> >> Problem is not introduced by managing multiple clusters from single >> nova-compute proxy node. > > I strongly disagree. > >> Internally this proxy driver is still presenting the "compute-node" for each >> of the cluster its managing. > > In what way? > >> What we need to think about is applicability of the live migration use case >> when a "cluster" is modelled as a compute. Since the "cluster" is modelled >> as a compute, it is assumed that a typical use case of live-move is taken >> care by the underlying "cluster" itself. With this there are other use >> cases which are no-op today like host maintenance mode, live move, setting >> instance affinity etc., In order to resolve this I was thinking of >> "A way to expose operations on individual ESX Hosts like Putting host in >> maintenance mode, live move, instance affinity etc., by introducing Parent >> - Child compute node concept. Scheduling can be restricted to Parent >> compute node and Child compute node can be used for providing more drill >> down on compute and also enable additional compute operations".Any >> thoughts on this? > > The fundamental problem is that hacks were put in place in order to make > Nova defer control to vCenter, when the design of Nova and vCenter are > not compatible, and we're paying the price for that right now. > > All of the operations you describe above -- putting a host in > maintenance mode, live-migration of an instance, ensuring a new instance > is launched near or not-near another instance -- depend on a fundamental > design feature in Nova: that a nova-compute worker fully controls and > manages a host that provides a place to put server instances. We have > internal driver interfaces for the *hypervisor*, not for the *manager of > hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 07/04/14 06:20, Jay Pipes wrote: > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single >> nova-compute managing a single vCenter cluster, not multiple ones. >> >> Problem is not introduced by managing multiple clusters from single >> nova-compute proxy node. > > I strongly disagree. > >> Internally this proxy driver is still presenting the "compute-node" for each >> of the cluster its managing. > > In what way? > >> What we need to think about is applicability of the live migration use case >> when a "cluster" is modelled as a compute. Since the "cluster" is modelled >> as a compute, it is assumed that a typical use case of live-move is taken >> care by the underlying "cluster" itself. With this there are other use >> cases which are no-op today like host maintenance mode, live move, setting >> instance affinity etc., In order to resolve this I was thinking of >> "A way to expose operations on individual ESX Hosts like Putting host in >> maintenance mode, live move, instance affinity etc., by introducing Parent >> - Child compute node concept. Scheduling can be restricted to Parent >> compute node and Child compute node can be used for providing more drill >> down on compute and also enable additional compute operations".Any >> thoughts on this? > > The fundamental problem is that hacks were put in place in order to make > Nova defer control to vCenter, when the design of Nova and vCenter are > not compatible, and we're paying the price for that right now. > > All of the operations you describe above -- putting a host in > maintenance mode, live-migration of an instance, ensuring a new instance > is launched near or not-near another instance -- depend on a fundamental > design feature in Nova: that a nova-compute worker fully controls and > manages a host that provides a place to put server instances. We have > internal driver interfaces for the *hypervisor*, not for the *manager of > hypervisors*, because, you know, that's what Nova does. I'm going to take you to task here for use of the word 'fundamental'. What does Nova do? Apparently: 'OpenStack Nova provides a cloud computing fabric controller, supporting a wide variety of virtualization technologies, including KVM, Xen, LXC, VMware, and more. In addition to its native API, it includes compatibility with the commonly encountered Amazon EC2 and S3 APIs.' There's nothing in there about the ratio of Nova instances to hypervisors: that's an implementation detail. Now this change may or may not sit well with design decisions which have been made in the past, but the concept of managing multiple clusters from a single Nova instance is certainly not fundamentally wrong. It may not be pragmatic; it may require further changes to Nova which were not made, but there is nothing about it which is fundamentally at odds with the stated goals of the project. Why did I bother with that? I think it's in danger of being lost. Nova has been around for a while now and it has a lot of code and a lot of developers behind it. We need to remember, though, that's it's all for nothing if nobody wants to use it. VMware is different, but not wrong. Let's stay fresh. > The problem with all of the vCenter stuff is that it is trying to say to > Nova "don't worry, I got this" but unfortunately, Nova wants and needs > to manage these things, not surrender control to a different system that > handles orchestration and scheduling in its own unique way. Again, I'll flip that round. Nova *currently* manages these things, and working efficiently with a platform which also does these things would require rethinking some design above the driver level. It's not something we want to do naively, which the VMware driver is suffering from in this area. It may take time to get this right, but we shouldn't write it off as fundamentally wrong. It's useful to users and not fundamentally at odds with the project's goals. > If a shop really wants to use vCenter for scheduling and orchestration > of server instances, what exactly is the point of using OpenStack Nova > to begin with? What exactly is the point of trying to use OpenStack Nova > for scheduling and host operations when you've already shelled out US > $6,000 for vCenter Server and a boatload more money for ESX licensing? I confess I wondered this myself. However, I have now spoken to real people who are spending real money doing exactly this. The drivers seem to be: * The external API * A heterogeneous cloud vSphere isn't really designed for the former and doesn't do it well. It obviously doesn't help with the latter at all. For example, users want to be able to give non-admin customers the ability to deploy across both KVM and VMware. To my
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi, I'm fairly new to this list, actually this is my first email sent, and to OpenStack in general, but I'm not new at all to VMware so I'll try to give you my point of view about possible use case here. Jay you are saying that by using Nova to manage ESXi hosts we don't need vCenter because they basically overlap in their capabilities. I agree with you to some extent, Nova may have similar capabilities as vCenter Server but as you know OpenStack as a full cloud solution adds a lot more features that vCenter lacks, like multitenancy just to name one. Also in any vSphere environment managing ESXi hosts individually, this is without vCenter, is completely out of the question. vCenter is the enabler of many vSphere features. And precisely that's is, IMHO, the use case of using Nova to manage vCenter to manage vSphere. Without vCenter we only have a bunch of hypervisors and none of the HA or DRS (dynamic resource balancing) capabilities that a vSphere cluster provides, this in my experience with vSphere users/customers is a no go scenario. I don't know why the decision to manage vCenter with Nova was made but based on the above I understand the reasoning. Best, --- Juan Manuel Rey @jreypo On Mon, Apr 7, 2014 at 7:20 AM, Jay Pipes wrote: > On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: > > >> Well, it seems to me that the problem is the above blueprint and the > code it introduced. This is an anti-feature IMO, and probably the best > solution would be to remove the above code and go back to having a single > >> nova-compute managing a single vCenter cluster, not multiple ones. > > > > Problem is not introduced by managing multiple clusters from single > nova-compute proxy node. > > I strongly disagree. > > > Internally this proxy driver is still presenting the "compute-node" for > each of the cluster its managing. > > In what way? > > > What we need to think about is applicability of the live migration use > case when a "cluster" is modelled as a compute. Since the "cluster" is > modelled as a compute, it is assumed that a typical use case of live-move > is taken care by the underlying "cluster" itself. With this there are > other use cases which are no-op today like host maintenance mode, live > move, setting instance affinity etc., In order to resolve this I was > thinking of > > "A way to expose operations on individual ESX Hosts like Putting host in > maintenance mode, live move, instance affinity etc., by introducing Parent > - Child compute node concept. Scheduling can be restricted to Parent > compute node and Child compute node can be used for providing more drill > down on compute and also enable additional compute operations".Any > thoughts on this? > > The fundamental problem is that hacks were put in place in order to make > Nova defer control to vCenter, when the design of Nova and vCenter are > not compatible, and we're paying the price for that right now. > > All of the operations you describe above -- putting a host in > maintenance mode, live-migration of an instance, ensuring a new instance > is launched near or not-near another instance -- depend on a fundamental > design feature in Nova: that a nova-compute worker fully controls and > manages a host that provides a place to put server instances. We have > internal driver interfaces for the *hypervisor*, not for the *manager of > hypervisors*, because, you know, that's what Nova does. > > The problem with all of the vCenter stuff is that it is trying to say to > Nova "don't worry, I got this" but unfortunately, Nova wants and needs > to manage these things, not surrender control to a different system that > handles orchestration and scheduling in its own unique way. > > If a shop really wants to use vCenter for scheduling and orchestration > of server instances, what exactly is the point of using OpenStack Nova > to begin with? What exactly is the point of trying to use OpenStack Nova > for scheduling and host operations when you've already shelled out US > $6,000 for vCenter Server and a boatload more money for ESX licensing? > > Sorry, I'm just at a loss why Nova was changed to accomodate vCenter > cluster and management concepts to begin with. I just don't understand > the use case here. > > Best, > -jay > > > > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Sun, 2014-04-06 at 06:59 +, Nandavar, Divakar Padiyar wrote: > >> Well, it seems to me that the problem is the above blueprint and the code > >> it introduced. This is an anti-feature IMO, and probably the best solution > >> would be to remove the above code and go back to having a single >> > >> nova-compute managing a single vCenter cluster, not multiple ones. > > Problem is not introduced by managing multiple clusters from single > nova-compute proxy node. I strongly disagree. > Internally this proxy driver is still presenting the "compute-node" for each > of the cluster its managing. In what way? > What we need to think about is applicability of the live migration use case > when a "cluster" is modelled as a compute. Since the "cluster" is modelled > as a compute, it is assumed that a typical use case of live-move is taken > care by the underlying "cluster" itself. With this there are other use > cases which are no-op today like host maintenance mode, live move, setting > instance affinity etc., In order to resolve this I was thinking of > "A way to expose operations on individual ESX Hosts like Putting host in > maintenance mode, live move, instance affinity etc., by introducing Parent - > Child compute node concept. Scheduling can be restricted to Parent compute > node and Child compute node can be used for providing more drill down on > compute and also enable additional compute operations".Any thoughts on > this? The fundamental problem is that hacks were put in place in order to make Nova defer control to vCenter, when the design of Nova and vCenter are not compatible, and we're paying the price for that right now. All of the operations you describe above -- putting a host in maintenance mode, live-migration of an instance, ensuring a new instance is launched near or not-near another instance -- depend on a fundamental design feature in Nova: that a nova-compute worker fully controls and manages a host that provides a place to put server instances. We have internal driver interfaces for the *hypervisor*, not for the *manager of hypervisors*, because, you know, that's what Nova does. The problem with all of the vCenter stuff is that it is trying to say to Nova "don't worry, I got this" but unfortunately, Nova wants and needs to manage these things, not surrender control to a different system that handles orchestration and scheduling in its own unique way. If a shop really wants to use vCenter for scheduling and orchestration of server instances, what exactly is the point of using OpenStack Nova to begin with? What exactly is the point of trying to use OpenStack Nova for scheduling and host operations when you've already shelled out US $6,000 for vCenter Server and a boatload more money for ESX licensing? Sorry, I'm just at a loss why Nova was changed to accomodate vCenter cluster and management concepts to begin with. I just don't understand the use case here. Best, -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi Divakar, Can I say that the bare metal provisioning is now using kind of "Parent - Child compute" mode? I was also thinking that we can use host:node to identify a kind of "Parent-Child" or "Hierarchy Compute". So can you please show some difference for your "Parent - Child Compute Node" and bare metal provisioning? Thanks! 2014-04-06 14:59 GMT+08:00 Nandavar, Divakar Padiyar < divakar.padiyar-nanda...@hp.com>: > >> Well, it seems to me that the problem is the above blueprint and the > code it introduced. This is an anti-feature IMO, and probably the best > solution would be to remove the above code and go back to having a single > >> nova-compute managing a single vCenter cluster, not multiple ones. > > Problem is not introduced by managing multiple clusters from single > nova-compute proxy node. Internally this proxy driver is still presenting > the "compute-node" for each of the cluster its managing.What we need to > think about is applicability of the live migration use case when a > "cluster" is modelled as a compute. Since the "cluster" is modelled as a > compute, it is assumed that a typical use case of live-move is taken care > by the underlying "cluster" itself. With this there are other use > cases which are no-op today like host maintenance mode, live move, setting > instance affinity etc., In order to resolve this I was thinking of > "A way to expose operations on individual ESX Hosts like Putting host in > maintenance mode, live move, instance affinity etc., by introducing Parent > - Child compute node concept. Scheduling can be restricted to Parent > compute node and Child compute node can be used for providing more drill > down on compute and also enable additional compute operations".Any > thoughts on this? > > Thanks, > Divakar > > > -Original Message- > From: Jay Pipes [mailto:jaypi...@gmail.com] > Sent: Sunday, April 06, 2014 2:02 AM > To: openstack-dev@lists.openstack.org > Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live > migration with one nova compute > Importance: High > > On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: > > > > > > > > 2014-04-04 12:46 GMT+08:00 Jay Pipes : > > On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > > > Thanks Jay and Chris for the comments! > > > > > > @Jay Pipes, I think that we still need to enable "one nova > > compute > > > live migration" as one nova compute can manage multiple > > clusters and > > > VMs can be migrated between those clusters managed by one > > nova > > > compute. > > > > > > Why, though? That is what I am asking... seems to me like this > > is an > > anti-feature. What benefit does the user get from moving an > > instance > > from one VCenter cluster to another VCenter cluster if the two > > clusters > > are on the same physical machine? > > @Jay Pipes, for VMWare, one physical machine (ESX server) can only > > belong to one VCenter cluster, so we may have following scenarios. > > > > DC > > | > > > > |---Cluster1 > > | | > > > > | |---host1 > > | > > > > |---Cluser2 > > | > > > > |---host2 > > > > > > Then when using VCDriver, I can use one nova compute manage both > > Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to > > host1 ;-( > > > > > > The bp was introduced by > > https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed- > > by-one-service > > Well, it seems to me that the problem is the above blueprint and the code > it introduced. This is an anti-feature IMO, and probably the best solution > would be to remove the above code and go back to having a single > nova-compute managing a single vCenter cluster, not multiple ones. > > -jay > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
>> Well, it seems to me that the problem is the above blueprint and the code it >> introduced. This is an anti-feature IMO, and probably the best solution >> would be to remove the above code and go back to having a single >> >> nova-compute managing a single vCenter cluster, not multiple ones. Problem is not introduced by managing multiple clusters from single nova-compute proxy node. Internally this proxy driver is still presenting the "compute-node" for each of the cluster its managing.What we need to think about is applicability of the live migration use case when a "cluster" is modelled as a compute. Since the "cluster" is modelled as a compute, it is assumed that a typical use case of live-move is taken care by the underlying "cluster" itself. With this there are other use cases which are no-op today like host maintenance mode, live move, setting instance affinity etc., In order to resolve this I was thinking of "A way to expose operations on individual ESX Hosts like Putting host in maintenance mode, live move, instance affinity etc., by introducing Parent - Child compute node concept. Scheduling can be restricted to Parent compute node and Child compute node can be used for providing more drill down on compute and also enable additional compute operations".Any thoughts on this? Thanks, Divakar -Original Message- From: Jay Pipes [mailto:jaypi...@gmail.com] Sent: Sunday, April 06, 2014 2:02 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Importance: High On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: > > > > 2014-04-04 12:46 GMT+08:00 Jay Pipes : > On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > > Thanks Jay and Chris for the comments! > > > > @Jay Pipes, I think that we still need to enable "one nova > compute > > live migration" as one nova compute can manage multiple > clusters and > > VMs can be migrated between those clusters managed by one > nova > > compute. > > > Why, though? That is what I am asking... seems to me like this > is an > anti-feature. What benefit does the user get from moving an > instance > from one VCenter cluster to another VCenter cluster if the two > clusters > are on the same physical machine? > @Jay Pipes, for VMWare, one physical machine (ESX server) can only > belong to one VCenter cluster, so we may have following scenarios. > > DC > | > > |---Cluster1 > | | > > | |---host1 > | > > |---Cluser2 > | > > |---host2 > > > Then when using VCDriver, I can use one nova compute manage both > Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to > host1 ;-( > > > The bp was introduced by > https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed- > by-one-service Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Jay Pipes. If go back to having a single nova-compute managing a single vCenter cluster, then there might be problems in a large sacle vCenter cluster. There are still problems that we can not handle: 1) The VCDriver can also manage multiple resource pools with a single nova compute, the resource pool is another concept, we can create multiple resource pools in one vCenter cluster or create multiple resource pools in one ESX host. In a large scale cluster, there can be thousands of resource pools, it would make the admin crazy for the configuration. ;-) 2) How to manage ESX host which not belong to any cluster or resource pools? Such as following case: DC | |--- ESX host1 | |--- ESX host2 3) There is another bp https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory filed by Shawn, this bp want to report all resources including clusters, resource pools, esx hosts, this bp can be treated as the base for VCDriver, as if the VCDriver can get all resources, then it would be very easy to do what we want. Thanks! 2014-04-06 4:32 GMT+08:00 Jay Pipes : > On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: > > > > > > > > 2014-04-04 12:46 GMT+08:00 Jay Pipes : > > On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > > > Thanks Jay and Chris for the comments! > > > > > > @Jay Pipes, I think that we still need to enable "one nova > > compute > > > live migration" as one nova compute can manage multiple > > clusters and > > > VMs can be migrated between those clusters managed by one > > nova > > > compute. > > > > > > Why, though? That is what I am asking... seems to me like this > > is an > > anti-feature. What benefit does the user get from moving an > > instance > > from one VCenter cluster to another VCenter cluster if the two > > clusters > > are on the same physical machine? > > @Jay Pipes, for VMWare, one physical machine (ESX server) can only > > belong to one VCenter cluster, so we may have following scenarios. > > > > DC > > | > > > > |---Cluster1 > > | | > > > > | |---host1 > > | > > > > |---Cluser2 > > | > > > > |---host2 > > > > > > Then when using VCDriver, I can use one nova compute manage both > > Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to > > host1 ;-( > > > > > > The bp was introduced by > > > https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service > > Well, it seems to me that the problem is the above blueprint and the > code it introduced. This is an anti-feature IMO, and probably the best > solution would be to remove the above code and go back to having a > single nova-compute managing a single vCenter cluster, not multiple > ones. > > -jay > > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Fri, 2014-04-04 at 13:30 +0800, Jay Lau wrote: > > > > 2014-04-04 12:46 GMT+08:00 Jay Pipes : > On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > > Thanks Jay and Chris for the comments! > > > > @Jay Pipes, I think that we still need to enable "one nova > compute > > live migration" as one nova compute can manage multiple > clusters and > > VMs can be migrated between those clusters managed by one > nova > > compute. > > > Why, though? That is what I am asking... seems to me like this > is an > anti-feature. What benefit does the user get from moving an > instance > from one VCenter cluster to another VCenter cluster if the two > clusters > are on the same physical machine? > @Jay Pipes, for VMWare, one physical machine (ESX server) can only > belong to one VCenter cluster, so we may have following scenarios. > > DC > | > > |---Cluster1 > | | > > | |---host1 > | > > |---Cluser2 > | > > |---host2 > > > Then when using VCDriver, I can use one nova compute manage both > Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to > host1 ;-( > > > The bp was introduced by > https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service Well, it seems to me that the problem is the above blueprint and the code it introduced. This is an anti-feature IMO, and probably the best solution would be to remove the above code and go back to having a single nova-compute managing a single vCenter cluster, not multiple ones. -jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
2014-04-04 12:46 GMT+08:00 Jay Pipes : > On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > > Thanks Jay and Chris for the comments! > > > > @Jay Pipes, I think that we still need to enable "one nova compute > > live migration" as one nova compute can manage multiple clusters and > > VMs can be migrated between those clusters managed by one nova > > compute. > > Why, though? That is what I am asking... seems to me like this is an > anti-feature. What benefit does the user get from moving an instance > from one VCenter cluster to another VCenter cluster if the two clusters > are on the same physical machine? > @Jay Pipes, for VMWare, one physical machine (ESX server) can only belong to one VCenter cluster, so we may have following scenarios. DC | |---Cluster1 | | | |---host1 | |---Cluser2 | |---host2 Then when using VCDriver, I can use one nova compute manage both Cluster1 and Cluster2, this caused me cannot migrate VM from host2 to host1 ;-( The bp was introduced by https://blueprints.launchpad.net/nova/+spec/multiple-clusters-managed-by-one-service > > Secondly, why is it that a single nova-compute manages multiple VCenter > clusters? This seems like a hack to me... perhaps someone who wrote the > code for this or knows the decision behind it could chime in here? > > > For cell, IMHO, each "cell" can be treated as a small "cloud" but not > > a "compute", each "cell cloud" should be able to handle VM operations > > in the small cloud itself. Please correct me if I am wrong. > > Yes, I agree with you that a cell is not a compute. Not sure if I said > otherwise in my previous response. Sorry if it was confusing! :) > > Best, > -jay > > > @Chris, "OS-EXT-SRV-ATTR:host" is the host where nova compute is > > running and "OS-EXT-SRV-ATTR:hypervisor_hostname" is the hypervisor > > host where the VM is running. Live migration is now using "host" for > > live migration. What I want to do is enable migration with one "host" > > and the "host" managing multiple "hyperviosrs". > > > > > > I'm planning to draft a bp for review which depend on > > https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory > > > > > > Thanks! > > > > > > > > 2014-04-04 8:03 GMT+08:00 Chris Friesen : > > On 04/03/2014 05:48 PM, Jay Pipes wrote: > > On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: > > Hi, > > > > Currently with VMWare VCDriver, one nova > > compute can manage multiple > > clusters/RPs, this caused cluster admin cannot > > do live migration > > between clusters/PRs if those clusters/PRs > > managed by one nova compute > > as the current live migration logic request at > > least two nova > > computes. > > > > > > A bug [1] was also filed to trace VMWare live > > migration issue. > > > > I'm now trying the following solution to see > > if it is acceptable for a > > fix, the fix wants enable live migration with > > one nova compute: > > 1) When live migration check if host are same, > > check both host and > > node for the VM instance. > > 2) When nova scheduler select destination for > > live migration, the live > > migration task should put (host, node) to > > attempted hosts. > > 3) Nova scheduler needs to be enhanced to > > support ignored_nodes. > > 4) nova compute need to be enhanced to check > > host and node when doing > > live migration. > > > > What precisely is the point of "live migrating" an > > instance to the exact > > same host as it is already on? The failure domain is > > the host, so moving > > the instance from one "cluster" to another, but on the > > same host is kind > > of a silly use case IMO. > > > > > > Here is where precise definitions of "compute node", > > "OS-EXT-SRV-ATTR:host", and > > "OS-EXT-SRV-ATTR:hypervisor_hostname", and "host" as > > understood by novaclient would be nice. > > > > Currently the "nova live-migration" command takes a "host" > > argument. It's not clear which of the above this corresponds > > to. > > > > My understanding is that one nova-compute process can manage > > multiple VMWare physical hosts. So it could make sense to > > support live migration betwee
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Fri, 2014-04-04 at 11:08 +0800, Jay Lau wrote: > Thanks Jay and Chris for the comments! > > @Jay Pipes, I think that we still need to enable "one nova compute > live migration" as one nova compute can manage multiple clusters and > VMs can be migrated between those clusters managed by one nova > compute. Why, though? That is what I am asking... seems to me like this is an anti-feature. What benefit does the user get from moving an instance from one VCenter cluster to another VCenter cluster if the two clusters are on the same physical machine? Secondly, why is it that a single nova-compute manages multiple VCenter clusters? This seems like a hack to me... perhaps someone who wrote the code for this or knows the decision behind it could chime in here? > For cell, IMHO, each "cell" can be treated as a small "cloud" but not > a "compute", each "cell cloud" should be able to handle VM operations > in the small cloud itself. Please correct me if I am wrong. Yes, I agree with you that a cell is not a compute. Not sure if I said otherwise in my previous response. Sorry if it was confusing! :) Best, -jay > @Chris, "OS-EXT-SRV-ATTR:host" is the host where nova compute is > running and "OS-EXT-SRV-ATTR:hypervisor_hostname" is the hypervisor > host where the VM is running. Live migration is now using "host" for > live migration. What I want to do is enable migration with one "host" > and the "host" managing multiple "hyperviosrs". > > > I'm planning to draft a bp for review which depend on > https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory > > > Thanks! > > > > 2014-04-04 8:03 GMT+08:00 Chris Friesen : > On 04/03/2014 05:48 PM, Jay Pipes wrote: > On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: > Hi, > > Currently with VMWare VCDriver, one nova > compute can manage multiple > clusters/RPs, this caused cluster admin cannot > do live migration > between clusters/PRs if those clusters/PRs > managed by one nova compute > as the current live migration logic request at > least two nova > computes. > > > A bug [1] was also filed to trace VMWare live > migration issue. > > I'm now trying the following solution to see > if it is acceptable for a > fix, the fix wants enable live migration with > one nova compute: > 1) When live migration check if host are same, > check both host and > node for the VM instance. > 2) When nova scheduler select destination for > live migration, the live > migration task should put (host, node) to > attempted hosts. > 3) Nova scheduler needs to be enhanced to > support ignored_nodes. > 4) nova compute need to be enhanced to check > host and node when doing > live migration. > > What precisely is the point of "live migrating" an > instance to the exact > same host as it is already on? The failure domain is > the host, so moving > the instance from one "cluster" to another, but on the > same host is kind > of a silly use case IMO. > > > Here is where precise definitions of "compute node", > "OS-EXT-SRV-ATTR:host", and > "OS-EXT-SRV-ATTR:hypervisor_hostname", and "host" as > understood by novaclient would be nice. > > Currently the "nova live-migration" command takes a "host" > argument. It's not clear which of the above this corresponds > to. > > My understanding is that one nova-compute process can manage > multiple VMWare physical hosts. So it could make sense to > support live migration between separate VMWare hosts even if > they're managed by a single nova-compute process. > > Chris > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > > > -- > Thanks, > > > Jay > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-b
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Jay and Chris for the comments! @Jay Pipes, I think that we still need to enable "one nova compute live migration" as one nova compute can manage multiple clusters and VMs can be migrated between those clusters managed by one nova compute. For cell, IMHO, each "cell" can be treated as a small "cloud" but not a "compute", each "cell cloud" should be able to handle VM operations in the small cloud itself. Please correct me if I am wrong. @Chris, "OS-EXT-SRV-ATTR:host" is the host where nova compute is running and "OS-EXT-SRV-ATTR:hypervisor_hostname" is the hypervisor host where the VM is running. Live migration is now using "host" for live migration. What I want to do is enable migration with one "host" and the "host" managing multiple "hyperviosrs". I'm planning to draft a bp for review which depend on https://blueprints.launchpad.net/nova/+spec/vmware-auto-inventory Thanks! 2014-04-04 8:03 GMT+08:00 Chris Friesen : > On 04/03/2014 05:48 PM, Jay Pipes wrote: > >> On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: >> >>> Hi, >>> >>> Currently with VMWare VCDriver, one nova compute can manage multiple >>> clusters/RPs, this caused cluster admin cannot do live migration >>> between clusters/PRs if those clusters/PRs managed by one nova compute >>> as the current live migration logic request at least two nova >>> computes. >>> >>> >>> A bug [1] was also filed to trace VMWare live migration issue. >>> >>> I'm now trying the following solution to see if it is acceptable for a >>> fix, the fix wants enable live migration with one nova compute: >>> 1) When live migration check if host are same, check both host and >>> node for the VM instance. >>> 2) When nova scheduler select destination for live migration, the live >>> migration task should put (host, node) to attempted hosts. >>> 3) Nova scheduler needs to be enhanced to support ignored_nodes. >>> 4) nova compute need to be enhanced to check host and node when doing >>> live migration. >>> >> >> What precisely is the point of "live migrating" an instance to the exact >> same host as it is already on? The failure domain is the host, so moving >> the instance from one "cluster" to another, but on the same host is kind >> of a silly use case IMO. >> > > Here is where precise definitions of "compute node", > "OS-EXT-SRV-ATTR:host", and "OS-EXT-SRV-ATTR:hypervisor_hostname", and > "host" as understood by novaclient would be nice. > > Currently the "nova live-migration" command takes a "host" argument. It's > not clear which of the above this corresponds to. > > My understanding is that one nova-compute process can manage multiple > VMWare physical hosts. So it could make sense to support live migration > between separate VMWare hosts even if they're managed by a single > nova-compute process. > > Chris > > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 04/03/2014 05:48 PM, Jay Pipes wrote: On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. What precisely is the point of "live migrating" an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one "cluster" to another, but on the same host is kind of a silly use case IMO. Here is where precise definitions of "compute node", "OS-EXT-SRV-ATTR:host", and "OS-EXT-SRV-ATTR:hypervisor_hostname", and "host" as understood by novaclient would be nice. Currently the "nova live-migration" command takes a "host" argument. It's not clear which of the above this corresponds to. My understanding is that one nova-compute process can manage multiple VMWare physical hosts. So it could make sense to support live migration between separate VMWare hosts even if they're managed by a single nova-compute process. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On Mon, 2014-03-31 at 17:11 +0800, Jay Lau wrote: > Hi, > > Currently with VMWare VCDriver, one nova compute can manage multiple > clusters/RPs, this caused cluster admin cannot do live migration > between clusters/PRs if those clusters/PRs managed by one nova compute > as the current live migration logic request at least two nova > computes. > > > A bug [1] was also filed to trace VMWare live migration issue. > > I'm now trying the following solution to see if it is acceptable for a > fix, the fix wants enable live migration with one nova compute: > 1) When live migration check if host are same, check both host and > node for the VM instance. > 2) When nova scheduler select destination for live migration, the live > migration task should put (host, node) to attempted hosts. > 3) Nova scheduler needs to be enhanced to support ignored_nodes. > 4) nova compute need to be enhanced to check host and node when doing > live migration. What precisely is the point of "live migrating" an instance to the exact same host as it is already on? The failure domain is the host, so moving the instance from one "cluster" to another, but on the same host is kind of a silly use case IMO. But... if this really is something that is considered useful, then it seems to me that it would be more useful to simply expand the definition of the "compute node" object within Nova to be generic enough that a compute node could be a VCenter cluster/PR. That way there would be no need to hack the scheduler to account for more than the host (compute node internally in Nova)? In the same way, a cell could be a "compute node" as well, and we wouldn't need separate hacks in the scheduler and elsewhere that treated cells differently than "regular" compute nodes. Best, -jay > I also uploaded a WIP patch [2] for you to review the idea of the fix > and hope can get some comments from you. > > > [1] https://bugs.launchpad.net/nova/+bug/1192192 > [2] https://review.openstack.org/#/c/84085 > > > -- > Thanks, > > > Jay > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks Solly and Alessandro! @Solly, Yes, I also want to make the code change not VMWare-specific, but perhaps we can consider of what @Alessandro said, we are going to have hyper-V cluster support in next cycle, and maybe Power HMC in the future. All of them can be managed in cluster level and one cluster can have multiple hypervisors. So I think that it might be a time to enhance live migration to handle not only the case of single hypervisor but also multiple hypervisors managed by one cluster. Hope we can also get some comments from VMWare guys. Thanks. 2014-04-01 6:57 GMT+08:00 Alessandro Pilotti < apilo...@cloudbasesolutions.com>: > > On 31 Mar 2014, at 18:13, Solly Ross wrote: > > > Building on what John said, I'm a bit wary of introducing semantics into > the Conductor's live migration code > > that are VMWare-specific. The conductor's live-migration code is > supposed to be driver-agnostic. IMHO, it > > would be much better if we could handle this at a level where the code > was already VMWare-specific. > > > > In terms of driver specific features, we're evaluating cluster support for > Hyper-V in the next cycle which would encounter the same issue for live > migration. > Hyper-V does not require clustering for supporting live migration (it's > already available since Grizzly), but various users are requesting Windows > clustering support > for supporting specific scenarios, which requires a separate Nova Hyper-V > failover clustering driver with resemblances to the VCenter driver in terms > of > cells / hosts management. Note: this is not related to Microsoft System > Center. > > Evaluating such a feature solely in base of blueprints barely under > drafting for other drivers and never discussed for approval isn't obviously > requested, but it might be > useful to consider the possibility that VMWare's might not be the only > Nova driver with this requirement in the relatively short term future. > > Thanks, > > Alessandro > > > > Best Regards, > > Solly Ross > > > > - Original Message - > > From: "Jay Lau" > > To: "OpenStack Development Mailing List (not for usage questions)" < > openstack-dev@lists.openstack.org> > > Sent: Monday, March 31, 2014 10:36:17 AM > > Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live > migration with one nova compute > > > > Thanks John. Yes, I also think that this should be a bp as it is going > to make some changes to enable live migration with only one nova compute, > will file a blueprint later. > > > > For your proposal "specify the same host as the instance", this can > resolve the issue of live migration with target host, but what about the > case of live migration without target host? If we still allow "specify the > same host as the instance", the the live migration will goes to dead loop. > > > > So it seems we definitely need to find a way to specify the node for > live migration, hope someone else can show some light here. > > > > Of course, I will file bp and go through the new bp review process for > this feature. > > > > Thanks! > > > > > > 2014-03-31 21:02 GMT+08:00 John Garbutt < j...@johngarbutt.com > : > > > > > > > > On 31 March 2014 10:11, Jay Lau < jay.lau@gmail.com > wrote: > >> Hi, > >> > >> Currently with VMWare VCDriver, one nova compute can manage multiple > >> clusters/RPs, this caused cluster admin cannot do live migration between > >> clusters/PRs if those clusters/PRs managed by one nova compute as the > >> current live migration logic request at least two nova computes. > >> > >> A bug [1] was also filed to trace VMWare live migration issue. > >> > >> I'm now trying the following solution to see if it is acceptable for a > fix, > >> the fix wants enable live migration with one nova compute: > >> 1) When live migration check if host are same, check both host and node > for > >> the VM instance. > >> 2) When nova scheduler select destination for live migration, the live > >> migration task should put (host, node) to attempted hosts. > >> 3) Nova scheduler needs to be enhanced to support ignored_nodes. > >> 4) nova compute need to be enhanced to check host and node when doing > live > >> migration. > >> > >> I also uploaded a WIP patch [2] for you to review the idea of the fix > and > >> hope can get some comments from you. > >> > >> [1] https://bugs.launchpad.net/nova/+bug/1192192 > >
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 31 Mar 2014, at 18:13, Solly Ross wrote: > Building on what John said, I'm a bit wary of introducing semantics into the > Conductor's live migration code > that are VMWare-specific. The conductor's live-migration code is supposed to > be driver-agnostic. IMHO, it > would be much better if we could handle this at a level where the code was > already VMWare-specific. > In terms of driver specific features, we’re evaluating cluster support for Hyper-V in the next cycle which would encounter the same issue for live migration. Hyper-V does not require clustering for supporting live migration (it’s already available since Grizzly), but various users are requesting Windows clustering support for supporting specific scenarios, which requires a separate Nova Hyper-V failover clustering driver with resemblances to the VCenter driver in terms of cells / hosts management. Note: this is not related to Microsoft System Center. Evaluating such a feature solely in base of blueprints barely under drafting for other drivers and never discussed for approval isn’t obviously requested, but it might be useful to consider the possibility that VMWare’s might not be the only Nova driver with this requirement in the relatively short term future. Thanks, Alessandro > Best Regards, > Solly Ross > > - Original Message - > From: "Jay Lau" > To: "OpenStack Development Mailing List (not for usage questions)" > > Sent: Monday, March 31, 2014 10:36:17 AM > Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live > migration with one nova compute > > Thanks John. Yes, I also think that this should be a bp as it is going to > make some changes to enable live migration with only one nova compute, will > file a blueprint later. > > For your proposal "specify the same host as the instance", this can resolve > the issue of live migration with target host, but what about the case of live > migration without target host? If we still allow "specify the same host as > the instance", the the live migration will goes to dead loop. > > So it seems we definitely need to find a way to specify the node for live > migration, hope someone else can show some light here. > > Of course, I will file bp and go through the new bp review process for this > feature. > > Thanks! > > > 2014-03-31 21:02 GMT+08:00 John Garbutt < j...@johngarbutt.com > : > > > > On 31 March 2014 10:11, Jay Lau < jay.lau@gmail.com > wrote: >> Hi, >> >> Currently with VMWare VCDriver, one nova compute can manage multiple >> clusters/RPs, this caused cluster admin cannot do live migration between >> clusters/PRs if those clusters/PRs managed by one nova compute as the >> current live migration logic request at least two nova computes. >> >> A bug [1] was also filed to trace VMWare live migration issue. >> >> I'm now trying the following solution to see if it is acceptable for a fix, >> the fix wants enable live migration with one nova compute: >> 1) When live migration check if host are same, check both host and node for >> the VM instance. >> 2) When nova scheduler select destination for live migration, the live >> migration task should put (host, node) to attempted hosts. >> 3) Nova scheduler needs to be enhanced to support ignored_nodes. >> 4) nova compute need to be enhanced to check host and node when doing live >> migration. >> >> I also uploaded a WIP patch [2] for you to review the idea of the fix and >> hope can get some comments from you. >> >> [1] https://bugs.launchpad.net/nova/+bug/1192192 >> [2] https://review.openstack.org/#/c/84085 > > Long term, finding a way to unify how cells and the VMware driver > manages multiple hosts, seems like the best way forward. It would be a > shame for this API to be different between cells and VMware, although > right now, that might not work too well :( > > A better short term fix, might be to allow you to specify the same > host as the instance, and the scheduling of the node could be > delegated to the VMware driver, which might just delegate that to > vCenter. I assume we still need some way to specify the node, and I > can't immediately think of a good way forward. > > I feel this should really be treated as a blueprint, and go through > the new blueprint review process. That should help decide the right > approach to take. > > John > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinf
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Building on what John said, I'm a bit wary of introducing semantics into the Conductor's live migration code that are VMWare-specific. The conductor's live-migration code is supposed to be driver-agnostic. IMHO, it would be much better if we could handle this at a level where the code was already VMWare-specific. Best Regards, Solly Ross - Original Message - From: "Jay Lau" To: "OpenStack Development Mailing List (not for usage questions)" Sent: Monday, March 31, 2014 10:36:17 AM Subject: Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal "specify the same host as the instance", this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow "specify the same host as the instance", the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt < j...@johngarbutt.com > : On 31 March 2014 10:11, Jay Lau < jay.lau@gmail.com > wrote: > Hi, > > Currently with VMWare VCDriver, one nova compute can manage multiple > clusters/RPs, this caused cluster admin cannot do live migration between > clusters/PRs if those clusters/PRs managed by one nova compute as the > current live migration logic request at least two nova computes. > > A bug [1] was also filed to trace VMWare live migration issue. > > I'm now trying the following solution to see if it is acceptable for a fix, > the fix wants enable live migration with one nova compute: > 1) When live migration check if host are same, check both host and node for > the VM instance. > 2) When nova scheduler select destination for live migration, the live > migration task should put (host, node) to attempted hosts. > 3) Nova scheduler needs to be enhanced to support ignored_nodes. > 4) nova compute need to be enhanced to check host and node when doing live > migration. > > I also uploaded a WIP patch [2] for you to review the idea of the fix and > hope can get some comments from you. > > [1] https://bugs.launchpad.net/nova/+bug/1192192 > [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Thanks John. Yes, I also think that this should be a bp as it is going to make some changes to enable live migration with only one nova compute, will file a blueprint later. For your proposal "specify the same host as the instance", this can resolve the issue of live migration with target host, but what about the case of live migration without target host? If we still allow "specify the same host as the instance", the the live migration will goes to dead loop. So it seems we definitely need to find a way to specify the node for live migration, hope someone else can show some light here. Of course, I will file bp and go through the new bp review process for this feature. Thanks! 2014-03-31 21:02 GMT+08:00 John Garbutt : > On 31 March 2014 10:11, Jay Lau wrote: > > Hi, > > > > Currently with VMWare VCDriver, one nova compute can manage multiple > > clusters/RPs, this caused cluster admin cannot do live migration between > > clusters/PRs if those clusters/PRs managed by one nova compute as the > > current live migration logic request at least two nova computes. > > > > A bug [1] was also filed to trace VMWare live migration issue. > > > > I'm now trying the following solution to see if it is acceptable for a > fix, > > the fix wants enable live migration with one nova compute: > > 1) When live migration check if host are same, check both host and node > for > > the VM instance. > > 2) When nova scheduler select destination for live migration, the live > > migration task should put (host, node) to attempted hosts. > > 3) Nova scheduler needs to be enhanced to support ignored_nodes. > > 4) nova compute need to be enhanced to check host and node when doing > live > > migration. > > > > I also uploaded a WIP patch [2] for you to review the idea of the fix and > > hope can get some comments from you. > > > > [1] https://bugs.launchpad.net/nova/+bug/1192192 > > [2] https://review.openstack.org/#/c/84085 > > Long term, finding a way to unify how cells and the VMware driver > manages multiple hosts, seems like the best way forward. It would be a > shame for this API to be different between cells and VMware, although > right now, that might not work too well :( > > A better short term fix, might be to allow you to specify the same > host as the instance, and the scheduling of the node could be > delegated to the VMware driver, which might just delegate that to > vCenter. I assume we still need some way to specify the node, and I > can't immediately think of a good way forward. > > I feel this should really be treated as a blueprint, and go through > the new blueprint review process. That should help decide the right > approach to take. > > John > > ___ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
On 31 March 2014 10:11, Jay Lau wrote: > Hi, > > Currently with VMWare VCDriver, one nova compute can manage multiple > clusters/RPs, this caused cluster admin cannot do live migration between > clusters/PRs if those clusters/PRs managed by one nova compute as the > current live migration logic request at least two nova computes. > > A bug [1] was also filed to trace VMWare live migration issue. > > I'm now trying the following solution to see if it is acceptable for a fix, > the fix wants enable live migration with one nova compute: > 1) When live migration check if host are same, check both host and node for > the VM instance. > 2) When nova scheduler select destination for live migration, the live > migration task should put (host, node) to attempted hosts. > 3) Nova scheduler needs to be enhanced to support ignored_nodes. > 4) nova compute need to be enhanced to check host and node when doing live > migration. > > I also uploaded a WIP patch [2] for you to review the idea of the fix and > hope can get some comments from you. > > [1] https://bugs.launchpad.net/nova/+bug/1192192 > [2] https://review.openstack.org/#/c/84085 Long term, finding a way to unify how cells and the VMware driver manages multiple hosts, seems like the best way forward. It would be a shame for this API to be different between cells and VMware, although right now, that might not work too well :( A better short term fix, might be to allow you to specify the same host as the instance, and the scheduling of the node could be delegated to the VMware driver, which might just delegate that to vCenter. I assume we still need some way to specify the node, and I can't immediately think of a good way forward. I feel this should really be treated as a blueprint, and go through the new blueprint review process. That should help decide the right approach to take. John ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [OpenStack-Dev][Nova][VMWare] Enable live migration with one nova compute
Hi, Currently with VMWare VCDriver, one nova compute can manage multiple clusters/RPs, this caused cluster admin cannot do live migration between clusters/PRs if those clusters/PRs managed by one nova compute as the current live migration logic request at least two nova computes. A bug [1] was also filed to trace VMWare live migration issue. I'm now trying the following solution to see if it is acceptable for a fix, the fix wants enable live migration with one nova compute: 1) When live migration check if host are same, check both host and node for the VM instance. 2) When nova scheduler select destination for live migration, the live migration task should put (host, node) to attempted hosts. 3) Nova scheduler needs to be enhanced to support ignored_nodes. 4) nova compute need to be enhanced to check host and node when doing live migration. I also uploaded a WIP patch [2] for you to review the idea of the fix and hope can get some comments from you. [1] https://bugs.launchpad.net/nova/+bug/1192192 [2] https://review.openstack.org/#/c/84085 -- Thanks, Jay ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
