Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On 2/27/2018 6:34 PM, John Griffith wrote: So replication is set on create of the volume, you could have a rule that keeps the two features mutually exclusive, but I'm still not quite sure why that would be a requirement here. Yeah I didn't think of that either, the attachment record has the instance uuid in it right? So cinder could just iterate the list of attachments for the volume and send multiple requests to nova. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On Tue, Feb 27, 2018 at 9:34 AM, Walter Boring wrote: > I think you might be able to get away with just calling os-brick's > connect_volume again without the need to call disconnect_volume first. > calling disconnect_volume wouldn't be good for volumes that are being > used, just to refresh the connection_info on that volume. > Hmm... but then you'd have an orphaned connection left hanging around for the old connection no? > > On Tue, Feb 27, 2018 at 2:56 PM, Matt Riedemann > wrote: > >> On 2/27/2018 10:02 AM, Matthew Booth wrote: >> >>> Sounds like the work Nova will have to do is identical to volume update >>> (swap volume). i.e. Change where a disk's backing store is without actually >>> changing the disk. >>> >> >> That's not what I'm hearing. I'm hearing disconnect/reconnect. Only the >> libvirt driver supports swap volume, but I assume all other virt drivers >> could support this generically. >> >> >>> Multi-attach! There might be more than 1 instance per volume, and we >>> can't currently support volume update for multi-attached volumes. >>> >> Not sure I follow... why not? It's just refreshing connections, only difference is you might have to do this "n" times instead of once? > >> Good point - cinder would likely need to reject a request to replicate an >> in-use multiattach volume if the volume has more than one attachment. > > So replication is set on create of the volume, you could have a rule that keeps the two features mutually exclusive, but I'm still not quite sure why that would be a requirement here. > >> >> -- >> >> Thanks, >> >> Matt >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscrib >> e >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
I think you might be able to get away with just calling os-brick's connect_volume again without the need to call disconnect_volume first. calling disconnect_volume wouldn't be good for volumes that are being used, just to refresh the connection_info on that volume. On Tue, Feb 27, 2018 at 2:56 PM, Matt Riedemann wrote: > On 2/27/2018 10:02 AM, Matthew Booth wrote: > >> Sounds like the work Nova will have to do is identical to volume update >> (swap volume). i.e. Change where a disk's backing store is without actually >> changing the disk. >> > > That's not what I'm hearing. I'm hearing disconnect/reconnect. Only the > libvirt driver supports swap volume, but I assume all other virt drivers > could support this generically. > > >> Multi-attach! There might be more than 1 instance per volume, and we >> can't currently support volume update for multi-attached volumes. >> > > Good point - cinder would likely need to reject a request to replicate an > in-use multiattach volume if the volume has more than one attachment. > > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On 2/27/2018 10:02 AM, Matthew Booth wrote: Sounds like the work Nova will have to do is identical to volume update (swap volume). i.e. Change where a disk's backing store is without actually changing the disk. That's not what I'm hearing. I'm hearing disconnect/reconnect. Only the libvirt driver supports swap volume, but I assume all other virt drivers could support this generically. Multi-attach! There might be more than 1 instance per volume, and we can't currently support volume update for multi-attached volumes. Good point - cinder would likely need to reject a request to replicate an in-use multiattach volume if the volume has more than one attachment. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
Couple of thoughts: Sounds like the work Nova will have to do is identical to volume update (swap volume). i.e. Change where a disk's backing store is without actually changing the disk. Multi-attach! There might be more than 1 instance per volume, and we can't currently support volume update for multi-attached volumes. Matt On 27 February 2018 at 09:45, Matt Riedemann wrote: > On 2/26/2018 9:52 PM, John Griffith wrote: > >> Yeah, it seems like this would be pretty handy with what's there. So >> are folks good with that? Wanted to make sure there's nothing contentious >> there before I propose a spec on the Nova and Cinder sides. If you think it >> seems at least worth proposing I'll work on it and get something ready as a >> welcome home from Dublin gift for everyone :) >> > > I'll put it on the nova/cinder PTG etherpad agenda for Thursday morning. > This seems like simple plumbing on the nova side, so not any major problems > from me. > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Matthew Booth Red Hat OpenStack Engineer, Compute DFG Phone: +442070094448 (UK) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On 2/26/2018 9:52 PM, John Griffith wrote: Yeah, it seems like this would be pretty handy with what's there. So are folks good with that? Wanted to make sure there's nothing contentious there before I propose a spec on the Nova and Cinder sides. If you think it seems at least worth proposing I'll work on it and get something ready as a welcome home from Dublin gift for everyone :) I'll put it on the nova/cinder PTG etherpad agenda for Thursday morning. This seems like simple plumbing on the nova side, so not any major problems from me. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On Mon, Feb 26, 2018 at 2:47 PM, Matt Riedemann wrote: > On 2/26/2018 9:28 PM, John Griffith wrote: > >> I'm also wondering how much of the extend actions we can leverage here, >> but I haven't looked through all of that yet. >> > > The os-server-external-events API in nova is generic. We'd just add a new > microversion to register a new tag for this event. Like the extend volume > event, the volume ID would be provided as input to the API and nova would > use that to identify the instance + volume to refresh on the compute host. > > We'd also register a new instance action / event record so that users > could poll the os-instance-actions API for completion of the operation. Yeah, it seems like this would be pretty handy with what's there. So are folks good with that? Wanted to make sure there's nothing contentious there before I propose a spec on the Nova and Cinder sides. If you think it seems at least worth proposing I'll work on it and get something ready as a welcome home from Dublin gift for everyone :) > > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On 2/26/2018 9:28 PM, John Griffith wrote: I'm also wondering how much of the extend actions we can leverage here, but I haven't looked through all of that yet. The os-server-external-events API in nova is generic. We'd just add a new microversion to register a new tag for this event. Like the extend volume event, the volume ID would be provided as input to the API and nova would use that to identify the instance + volume to refresh on the compute host. We'd also register a new instance action / event record so that users could poll the os-instance-actions API for completion of the operation. -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On Mon, Feb 26, 2018 at 2:13 PM, Matt Riedemann wrote: > On 2/26/2018 8:09 PM, John Griffith wrote: > >> I'm interested in looking at creating a mechanism to "refresh" all of the >> existing/current attachments as part of the Cinder Failover process. >> > > What would be involved on the nova side for the refresh? I'm guessing > disconnect/connect the volume via os-brick (or whatever for non-libvirt > drivers), resulting in a new host connector from os-brick that nova would > use to update the existing volume attachment for the volume/server instance > combo? Yep, that's pretty much exactly what I'm thinking about / looking at. I'm also wondering how much of the extend actions we can leverage here, but I haven't looked through all of that yet. > > > -- > > Thanks, > > Matt > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder][nova] Update attachments on replication failover
On 2/26/2018 8:09 PM, John Griffith wrote: I'm interested in looking at creating a mechanism to "refresh" all of the existing/current attachments as part of the Cinder Failover process. What would be involved on the nova side for the refresh? I'm guessing disconnect/connect the volume via os-brick (or whatever for non-libvirt drivers), resulting in a new host connector from os-brick that nova would use to update the existing volume attachment for the volume/server instance combo? -- Thanks, Matt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [cinder][nova] Update attachments on replication failover
Hey Everyone, Something I've been looking at with Cinder's replication (sort of the next step in the evolution if you will) is the ability to refresh/renew in-use volumes that were part of a migration event. We do something similar with extend-volume on the Nova side through the use of Instance Actions I believe, and I'm wondering how folks would feel about the same sort of thing being added upon failover/failback for replicated Cinder volumes? If you're not familiar, Cinder allows a volume to be replicated to multiple physical backend devices, and in the case of a DR situation an Operator can failover a backend device (or even a single volume). This process results in Cinder making some calls to the respective backend device, it doing it's magic and updating the Cinder Volume Model with new attachment info. This works great, except for the case of users that have a bunch of in-use volumes on that particular backend. We don't currently do anything to refresh/update them, so it's a manual process of running through a detach/attach loop. I'm interested in looking at creating a mechanism to "refresh" all of the existing/current attachments as part of the Cinder Failover process. Curious if anybody has any thoughts on this, or if anyone has already done something related to this topic? Thanks, John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
