Re: [openstack-dev] [cinder][nova] about re-image the volume

[Gorka Eguileor]

On 09/04, Sean McGinnis wrote:
> On Mon, Apr 09, 2018 at 07:00:56PM +0100, Duncan Thomas wrote:
> > Hopefully this flow means we can do rebuild root filesystem from
> > snapshot/backup too? It seems rather artificially limiting to only do
> > restore-from-image. I'd expect restore-from-snap to be a more common
> > use case, personally.
> >
>
> That could get tricky. We only support reverting to the last snapshot if we
> reuse the same volume. Otherwise, we can create volume from snapshot, but I
> don't think it's often that the first thing a user does is create a snapshot 
> on
> initial creation of a boot image. If it was created from image cache, and the
> backend creates those cached volume by using a snapshot, then that might be an
> option.
>
> But these are a lot of ifs, so that seems like it would make the logic for 
> this
> much more complicated.
>
> Maybe a phase II optimization we can look into?
>

From the Cinder side of things I think these two would be easier than
the re-image, because we would have even fewer steps, and the
functionality to do the copying is exactly what we have now, as it will
copy the data to the same volume, so we wouldn't need to fiddle with the
UUID fields etc.

Moreover I know customers who have asked about this functionality in the
past, mostly interested in restoring the root volume of an existing VM
from a backup to preserve the system ID and not break licenses.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 4/9/2018 1:00 PM, Duncan Thomas wrote:

Hopefully this flow means we can do rebuild root filesystem from
snapshot/backup too? It seems rather artificially limiting to only do
restore-from-image. I'd expect restore-from-snap to be a more common
use case, personally.


Hmm, now you've got me thinking about image-defined block device 
mappings, which is something you'd have if you snapshot a volume-backed 
instance and then later use that image snapshot, which has metadata 
about the volume snapshot in it, to later create (or rebuild?) a server.


Tempest has a scenario test for the boot from volume case here:

https://review.openstack.org/#/c/555495/

I should note that even if you did snapshot a volume-backed server and 
then used that image to rebuild another non-volume-backed server, nova 
won't even look at the block_device_mapping_v2 metadata in the snapshot 
image during rebuild, it doesn't treat it like boot from volume does 
where nova uses the image-defined BDM to create a new volume-backed 
instance.


And now that I've said that, I wonder if people would expect the same 
semantics for rebuild as boot from volume with those types of 
images...it makes my head hurt. Maybe mdbooth would like to weigh in on 
this given he's present in this thread.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Sean McGinnis]

On Mon, Apr 09, 2018 at 07:00:56PM +0100, Duncan Thomas wrote:
> Hopefully this flow means we can do rebuild root filesystem from
> snapshot/backup too? It seems rather artificially limiting to only do
> restore-from-image. I'd expect restore-from-snap to be a more common
> use case, personally.
> 

That could get tricky. We only support reverting to the last snapshot if we
reuse the same volume. Otherwise, we can create volume from snapshot, but I
don't think it's often that the first thing a user does is create a snapshot on
initial creation of a boot image. If it was created from image cache, and the
backend creates those cached volume by using a snapshot, then that might be an
option.

But these are a lot of ifs, so that seems like it would make the logic for this
much more complicated.

Maybe a phase II optimization we can look into?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Duncan Thomas]

Hopefully this flow means we can do rebuild root filesystem from
snapshot/backup too? It seems rather artificially limiting to only do
restore-from-image. I'd expect restore-from-snap to be a more common
use case, personally.

On 9 April 2018 at 09:51, Gorka Eguileor  wrote:
> On 06/04, Matt Riedemann wrote:
>> On 4/6/2018 5:09 AM, Matthew Booth wrote:
>> > I think you're talking at cross purposes here: this won't require a
>> > swap volume. Apart from anything else, swap volume only works on an
>> > attached volume, and as previously discussed Nova will detach and
>> > re-attach.
>> >
>> > Gorka, the Nova api Matt is referring to is called volume update
>> > externally. It's the operation required for live migrating an attached
>> > volume between backends. It's called swap volume internally in Nova.
>>
>> Yeah I was hoping we were just having a misunderstanding of what 'swap
>> volume' in nova is, which is the blockRebase for an already attached volume
>> to the guest, called from cinder during a volume retype or migration.
>>
>> As for the re-image thing, nova would be detaching the volume from the guest
>> prior to calling the new cinder re-image API, and then re-attach to the
>> guest afterward - similar to how shelve and unshelve work, and for that
>> matter how rebuild works today with non-root volumes.
>>
>> --
>>
>> Thanks,
>>
>> Matt
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> Hi,
>
> Thanks for the clarification.  When I was talking about "swapping" I was
> referring to the fact that Nova will have to not only detach the volume
> locally using OS-Brick, but it will also need to use new connection
> information to do the attach after the volume has been re-imaged.
>
> As I see it, the process would look something like this:
>
> - Nova detaches volume using OS-Brick
> - Nova calls Cinder re-image passing the node's info (like we do when
>   attaching a new volume)
> - Cinder would:
>   - Ensure only that node is connected to the volume
>   - Terminate connection to the original volume
>   - If we can do optimized volume creation:
> - If encrypted volume we create a copy of the encryption key on
>   Barbican or copy the ID field from the DB and ensure we don't
>   delete the Barbican key on the delete.
> - Create new volume from image
> - Swap DB fields to preserve the UUID
> - Delete original volume
>   - If it cannot do optimized volume creation:
> - Initialize+Attach volume to Cinder node
> - DD the new image into the volume
> - Detach+Terminate volume
>   - Initialize connection for the new volume to the Nova node
>   - Return connection information to the volume
> - Nova attaches volume with OS-Brick using returned connection
>   information.
>
> So I agree, it's not a blockRebase operation, just a change in the
> volume that is used.
>
> Regards,
> Gorka.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Duncan Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 4/9/2018 3:51 AM, Gorka Eguileor wrote:

As I see it, the process would look something like this:

- Nova detaches volume using OS-Brick
- Nova calls Cinder re-image passing the node's info (like we do when
   attaching a new volume)
- Cinder would:
   - Ensure only that node is connected to the volume
   - Terminate connection to the original volume
   - If we can do optimized volume creation:
 - If encrypted volume we create a copy of the encryption key on
   Barbican or copy the ID field from the DB and ensure we don't
   delete the Barbican key on the delete.
 - Create new volume from image
 - Swap DB fields to preserve the UUID
 - Delete original volume
   - If it cannot do optimized volume creation:
 - Initialize+Attach volume to Cinder node
 - DD the new image into the volume
 - Detach+Terminate volume
   - Initialize connection for the new volume to the Nova node
   - Return connection information to the volume
- Nova attaches volume with OS-Brick using returned connection
   information.

So I agree, it's not a blockRebase operation, just a change in the
volume that is used.


Yeah we're on the same page with respect to the high level changes on 
the nova side.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Gorka Eguileor]

On 06/04, Matt Riedemann wrote:
> On 4/6/2018 5:09 AM, Matthew Booth wrote:
> > I think you're talking at cross purposes here: this won't require a
> > swap volume. Apart from anything else, swap volume only works on an
> > attached volume, and as previously discussed Nova will detach and
> > re-attach.
> >
> > Gorka, the Nova api Matt is referring to is called volume update
> > externally. It's the operation required for live migrating an attached
> > volume between backends. It's called swap volume internally in Nova.
>
> Yeah I was hoping we were just having a misunderstanding of what 'swap
> volume' in nova is, which is the blockRebase for an already attached volume
> to the guest, called from cinder during a volume retype or migration.
>
> As for the re-image thing, nova would be detaching the volume from the guest
> prior to calling the new cinder re-image API, and then re-attach to the
> guest afterward - similar to how shelve and unshelve work, and for that
> matter how rebuild works today with non-root volumes.
>
> --
>
> Thanks,
>
> Matt
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Hi,

Thanks for the clarification.  When I was talking about "swapping" I was
referring to the fact that Nova will have to not only detach the volume
locally using OS-Brick, but it will also need to use new connection
information to do the attach after the volume has been re-imaged.

As I see it, the process would look something like this:

- Nova detaches volume using OS-Brick
- Nova calls Cinder re-image passing the node's info (like we do when
  attaching a new volume)
- Cinder would:
  - Ensure only that node is connected to the volume
  - Terminate connection to the original volume
  - If we can do optimized volume creation:
- If encrypted volume we create a copy of the encryption key on
  Barbican or copy the ID field from the DB and ensure we don't
  delete the Barbican key on the delete.
- Create new volume from image
- Swap DB fields to preserve the UUID
- Delete original volume
  - If it cannot do optimized volume creation:
- Initialize+Attach volume to Cinder node
- DD the new image into the volume
- Detach+Terminate volume
  - Initialize connection for the new volume to the Nova node
  - Return connection information to the volume
- Nova attaches volume with OS-Brick using returned connection
  information.

So I agree, it's not a blockRebase operation, just a change in the
volume that is used.

Regards,
Gorka.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 4/6/2018 5:09 AM, Matthew Booth wrote:

I think you're talking at cross purposes here: this won't require a
swap volume. Apart from anything else, swap volume only works on an
attached volume, and as previously discussed Nova will detach and
re-attach.

Gorka, the Nova api Matt is referring to is called volume update
externally. It's the operation required for live migrating an attached
volume between backends. It's called swap volume internally in Nova.


Yeah I was hoping we were just having a misunderstanding of what 'swap 
volume' in nova is, which is the blockRebase for an already attached 
volume to the guest, called from cinder during a volume retype or migration.


As for the re-image thing, nova would be detaching the volume from the 
guest prior to calling the new cinder re-image API, and then re-attach 
to the guest afterward - similar to how shelve and unshelve work, and 
for that matter how rebuild works today with non-root volumes.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matthew Booth]

On 6 April 2018 at 09:31, Gorka Eguileor  wrote:
> On 05/04, Matt Riedemann wrote:
>> On 4/5/2018 3:15 AM, Gorka Eguileor wrote:
>> > But just to be clear, Nova will have to initialize the connection with
>> > the re-imagined volume and attach it again to the node, as in all cases
>> > (except when defaulting to downloading the image and dd-ing it to the
>> > volume) the result will be a new volume in the backend.
>>
>> Yeah I think I pointed this out earlier in this thread on what I thought the
>> steps would be on the nova side with respect to creating a new empty
>> attachment to keep the volume 'reserved' while we delete the old attachment,
>> re-image the volume, and then update the volume attachment for the new
>> connection. I think that would be similar to how shelve and unshelve works
>> in nova.
>>
>> Would this really require a swap volume call from Cinder? I'd hope not since
>> swap volume in itself is a pretty gross operation on the nova side.
>>
>> --
>>
>> Thanks,
>>
>> Matt
>>
>
> Hi Matt,
>
> Yes, it will require a volume swap, with the worst case scenario
> exception where we dd the image into the volume.

I think you're talking at cross purposes here: this won't require a
swap volume. Apart from anything else, swap volume only works on an
attached volume, and as previously discussed Nova will detach and
re-attach.

Gorka, the Nova api Matt is referring to is called volume update
externally. It's the operation required for live migrating an attached
volume between backends. It's called swap volume internally in Nova.

Matt

>
> In the same way that anyone would expect a re-imaging preserving the
> volume id, one would also expect it to behave like creating a new volume
> from the same image: be as fast and take up as much space on the
> backend.
>
> And to do so we have to use existing optimized mechanisms that will only
> work when creating a new volume.
>
> The alternative would be to have the worst case scenario as the default
> (attach and dd the image) and make *ALL* Cinder drivers implement the
> optimized mechanism where they can efficiently re-imagine a volume.  I
> can't talk for the Cinder team, but I for one would oppose this
> alternative.
>
> Cheers,
> Gorka.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Matthew Booth
Red Hat OpenStack Engineer, Compute DFG

Phone: +442070094448 (UK)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Gorka Eguileor]

On 05/04, Matt Riedemann wrote:
> On 4/5/2018 3:15 AM, Gorka Eguileor wrote:
> > But just to be clear, Nova will have to initialize the connection with
> > the re-imagined volume and attach it again to the node, as in all cases
> > (except when defaulting to downloading the image and dd-ing it to the
> > volume) the result will be a new volume in the backend.
>
> Yeah I think I pointed this out earlier in this thread on what I thought the
> steps would be on the nova side with respect to creating a new empty
> attachment to keep the volume 'reserved' while we delete the old attachment,
> re-image the volume, and then update the volume attachment for the new
> connection. I think that would be similar to how shelve and unshelve works
> in nova.
>
> Would this really require a swap volume call from Cinder? I'd hope not since
> swap volume in itself is a pretty gross operation on the nova side.
>
> --
>
> Thanks,
>
> Matt
>

Hi Matt,

Yes, it will require a volume swap, with the worst case scenario
exception where we dd the image into the volume.

In the same way that anyone would expect a re-imaging preserving the
volume id, one would also expect it to behave like creating a new volume
from the same image: be as fast and take up as much space on the
backend.

And to do so we have to use existing optimized mechanisms that will only
work when creating a new volume.

The alternative would be to have the worst case scenario as the default
(attach and dd the image) and make *ALL* Cinder drivers implement the
optimized mechanism where they can efficiently re-imagine a volume.  I
can't talk for the Cinder team, but I for one would oppose this
alternative.

Cheers,
Gorka.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 4/5/2018 3:15 AM, Gorka Eguileor wrote:

But just to be clear, Nova will have to initialize the connection with
the re-imagined volume and attach it again to the node, as in all cases
(except when defaulting to downloading the image and dd-ing it to the
volume) the result will be a new volume in the backend.


Yeah I think I pointed this out earlier in this thread on what I thought 
the steps would be on the nova side with respect to creating a new empty 
attachment to keep the volume 'reserved' while we delete the old 
attachment, re-image the volume, and then update the volume attachment 
for the new connection. I think that would be similar to how shelve and 
unshelve works in nova.


Would this really require a swap volume call from Cinder? I'd hope not 
since swap volume in itself is a pretty gross operation on the nova side.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Gorka Eguileor]

On 04/04, Matt Riedemann wrote:
> On 4/2/2018 6:59 AM, Gorka Eguileor wrote:
> > I can only see one benefit from implementing this feature in Cinder
> > versus doing it in Nova, and that is that we can preserve the volume's
> > UUID, but I don't think this is even relevant for this use case, so why
> > is it better to implement this in Cinder than in Nova?
>
> With a new image, the volume_image_metadata in the volume would also be
> wrong, and I don't think nova should (or even can) update that information.
> So nova re-imaging the volume doesn't seem like a good fit to me given
> Cinder "owns" the volume along with any metadata about it.
>
> If Cinder isn't agreeable to this new re-image API, then I think we're stuck

Hi Matt,

I didn't mean to imply that the Cinder team is against this proposal, I
just want to make sure that Cinder is the right place to do it and that
we will actually get some benefits from doing it in Cinder, because
right now I don't see that many...



> with the original proposal of creating a new volume and swapping out the
> root disk, along with all of the problems that can arise from that (original
> volume type is gone, tenant goes over-quota, what do we do with the original
> volume (delete it?), etc).
>
> --
>
> Thanks,
>
> Matt
>

This is what I thought the Nova alternative was, so that's why I didn't
understand the image metadata issue.

For clarification, the original volume type cannot be gone, as the type
delete operation prevents used volume types to be deleted, and if for
some reason it were gone (though I don't see how) Cinder would find
itself with the exact same problem, so there's no difference here.

The flow you are describing is basically what the generic implementation
for that functionality would do in Cinder:

- Create a new volume from image using the same volume type
- Swap the volume information like we do in the live migration case
- Delete the original volume
- Nova will have to swap the root volume (request new connection
  information for that volume and attach it to the node).

Because the alternative is for Cinder to download the image and dd it
into the original volume, which breaks all the optimizations that Cinder
has for speed and storage saving in the backend (there would be no
cloning).

So reading your response I expand the benefits to 2 if done by Cinder:

- Preserve volume UUID
- Remove unlikely race condition of someone deleting the volume type
  between Nova deleting the original volume and creating the new one (in
  this order to avoid the quota issue) when there is no other volume
  using that volume type.

I guess the user facing volume UUID preservation is good enough reason
to have this API in Cinder, as one would assume re-imaging a volume
would never result in having a new volume ID.

But just to be clear, Nova will have to initialize the connection with
the re-imagined volume and attach it again to the node, as in all cases
(except when defaulting to downloading the image and dd-ing it to the
volume) the result will be a new volume in the backend.

Cheers,
Gorka.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 4/2/2018 6:59 AM, Gorka Eguileor wrote:

I can only see one benefit from implementing this feature in Cinder
versus doing it in Nova, and that is that we can preserve the volume's
UUID, but I don't think this is even relevant for this use case, so why
is it better to implement this in Cinder than in Nova?


With a new image, the volume_image_metadata in the volume would also be 
wrong, and I don't think nova should (or even can) update that 
information. So nova re-imaging the volume doesn't seem like a good fit 
to me given Cinder "owns" the volume along with any metadata about it.


If Cinder isn't agreeable to this new re-image API, then I think we're 
stuck with the original proposal of creating a new volume and swapping 
out the root disk, along with all of the problems that can arise from 
that (original volume type is gone, tenant goes over-quota, what do we 
do with the original volume (delete it?), etc).


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Gorka Eguileor]

On 29/03, Sean McGinnis wrote:
> >   This is the spec [0] about rebuild the volumed backed server.
> > The question raised in the spec is about how to bandle the root volume.
> > Finally,in Nova team,we think that the cleanest / best solution to this is 
> > to
> > add a volume action API to cinder for re-imaging the volume.Once that is
> > available in a new cinder v3 microversion, nova can use it. The reason I
> > ...
> >   So Nova team want Cinder to achieve the re-image api.But, I see a spec
> > about volume revert by snapshot[1].It is so good for rebuild operation.In
> > short,I have two ideas,one is change the volume revert by snapshot spec to
> > re-image spec,not only it can let the volume revert by snapshot,but also can
> > re-image the volume which the image's size is greater than 0;another idea is
> > add a only re-image spec,it only can re-image the volume which the image's
> > size is greater than 0.
> >
>
> I do not think changing the revert to snapshot implementation is appropriate
> here. There may be some cases where this can get the desired result, but there
> is no guarantee that there is a snapshot on the volume's base image state to
> revert to. It also would not make sense to overload this functionality to
> "revert to snapshot if you can, otherwise do all this other stuff instead."
>
> This would need to be a new API (microversioned) to add a reimage call. I
> wouldn't expect implementation to be too difficult as we already have that
> functionality for new volumes. We would just need to figure out the most
> appropriate way to take an already in-use volume, detach it, rewrite the 
> image,
> then reattach it.
>

Hi,

The implementation may be more complex that we think, as we have 4
create volume from image mechanisms we have to consider:

- When Glance is using Cinder as backend
- When using Glance image location to do cloning
- When using Cinder cache and we do cloning
- Basic case where we download the image, attach the volume, and copy
  the data.

The only simple, yet efficient, solution I can see is calling the
driver's delete volume method (without soft-deleting it from the DB),
clear the volume DB information of the image metadata, and then run the
create volume from image flow with the same volume information but the
new image metadata.

I can only see one benefit from implementing this feature in Cinder
versus doing it in Nova, and that is that we can preserve the volume's
UUID, but I don't think this is even relevant for this use case, so why
is it better to implement this in Cinder than in Nova?

Cheers,
Gorka.


> Ideally, from my perspective, Nova would take care of the detach/attach 
> portion
> and Cinder would only need to take care of imaging the volume.
>
> Sean
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Jay S Bryant]

On 3/29/2018 8:36 PM, Matt Riedemann wrote:

On 3/29/2018 6:50 PM, Sean McGinnis wrote:
May we can add a "Reimaging" state to the volume? Then Nova could 
poll for it

to go from that back to Available?


That would be fine with me, and maybe similar to how 'extending' and 
'retyping' work for an attached volume?


Nova wouldn't wait for the volume to go to 'available', we don't want 
it to go to 'available', we'd just wait for it to go back to 
'reserved'. During a rebuild the instance still needs to keep the 
volume logically attached to it so another instance can't grab it.



This all sounds reasonable to me.

Thanks for hashing it out guys!

Jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 3/29/2018 6:50 PM, Sean McGinnis wrote:

May we can add a "Reimaging" state to the volume? Then Nova could poll for it
to go from that back to Available?


That would be fine with me, and maybe similar to how 'extending' and 
'retyping' work for an attached volume?


Nova wouldn't wait for the volume to go to 'available', we don't want it 
to go to 'available', we'd just wait for it to go back to 'reserved'. 
During a rebuild the instance still needs to keep the volume logically 
attached to it so another instance can't grab it.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Sean McGinnis]

> 
> >
> >Ideally, from my perspective, Nova would take care of the detach/attach 
> >portion
> >and Cinder would only need to take care of imaging the volume.
> 
> Agree. :) And yeah, I pointed this out in the nova spec for volume-backed
> rebuild also. I think nova can basically handle this like it does for shelve
> today, and we'd do something like this:
> 
> 1. disconnect the volume from the host
> 2. create a new empty volume attachment for the volume and instance - this
> is needed so the volume stays 'reserved' while we re-image it
> 3. delete the old volume attachment
> 4. call the new cinder re-image API
> 5. once the volume is available (TODO: how would we know?)

May we can add a "Reimaging" state to the volume? Then Nova could poll for it
to go from that back to Available? Since Nova is driving things, I would be
hesitant to expect and assume that Cinder is appropriately configured to call
back in to Nova.

Or a notification?

Or...?

> 6. re-attach the volume by updating the attachment with the host connector,
> connect on the host, and complete the attachment (marks the volume as in-use
> again)
> 
> -- 
> 
> Thanks,
> 
> Matt
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Matt Riedemann]

On 3/29/2018 9:28 AM, Sean McGinnis wrote:

I do not think changing the revert to snapshot implementation is appropriate
here. There may be some cases where this can get the desired result, but there
is no guarantee that there is a snapshot on the volume's base image state to
revert to. It also would not make sense to overload this functionality to
"revert to snapshot if you can, otherwise do all this other stuff instead."



Agree.


This would need to be a new API (microversioned) to add a reimage call. I
wouldn't expect implementation to be too difficult as we already have that
functionality for new volumes. We would just need to figure out the most
appropriate way to take an already in-use volume, detach it, rewrite the image,
then reattach it.


Agree.



Ideally, from my perspective, Nova would take care of the detach/attach portion
and Cinder would only need to take care of imaging the volume.


Agree. :) And yeah, I pointed this out in the nova spec for 
volume-backed rebuild also. I think nova can basically handle this like 
it does for shelve today, and we'd do something like this:


1. disconnect the volume from the host
2. create a new empty volume attachment for the volume and instance - 
this is needed so the volume stays 'reserved' while we re-image it

3. delete the old volume attachment
4. call the new cinder re-image API
5. once the volume is available (TODO: how would we know?)
6. re-attach the volume by updating the attachment with the host 
connector, connect on the host, and complete the attachment (marks the 
volume as in-use again)


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] about re-image the volume

[Sean McGinnis]

>   This is the spec [0] about rebuild the volumed backed server.
> The question raised in the spec is about how to bandle the root volume.
> Finally,in Nova team,we think that the cleanest / best solution to this is to
> add a volume action API to cinder for re-imaging the volume.Once that is
> available in a new cinder v3 microversion, nova can use it. The reason I
> ...
>   So Nova team want Cinder to achieve the re-image api.But, I see a spec
> about volume revert by snapshot[1].It is so good for rebuild operation.In
> short,I have two ideas,one is change the volume revert by snapshot spec to
> re-image spec,not only it can let the volume revert by snapshot,but also can
> re-image the volume which the image's size is greater than 0;another idea is
> add a only re-image spec,it only can re-image the volume which the image's
> size is greater than 0.
> 

I do not think changing the revert to snapshot implementation is appropriate
here. There may be some cases where this can get the desired result, but there
is no guarantee that there is a snapshot on the volume's base image state to
revert to. It also would not make sense to overload this functionality to
"revert to snapshot if you can, otherwise do all this other stuff instead."

This would need to be a new API (microversioned) to add a reimage call. I
wouldn't expect implementation to be too difficult as we already have that
functionality for new volumes. We would just need to figure out the most
appropriate way to take an already in-use volume, detach it, rewrite the image,
then reattach it.

Ideally, from my perspective, Nova would take care of the detach/attach portion
and Cinder would only need to take care of imaging the volume.

Sean

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder][nova] about re-image the volume

[李杰]

Hi,all


  This is the spec [0] about rebuild the volumed backed server.The question 
raised in the spec is about how to bandle the root volume.Finally,in Nova 
team,we think that the cleanest / best solution to this is to add a volume 
action API to cinder for re-imaging the volume.Once that is available in a new 
cinder v3 microversion, nova can use it. The reason I think this should be done 
in Cinder with re-imaging the volume there is (1) it's cleaner from the nova 
side and (2) then Cinder is in control of how that re-image should happen, 
along with any details it needs to update, e.g. the volume's 
"volume_image_metadata" information would need to be updated.We really aren't 
suitable to do the volume create/delete/swap orchestration thing since that 
entails issues with the volume type being gone, going over quota, what to do 
about deleting the old volume, etc.
  So Nova team want Cinder to achieve the re-image api.But, I see a spec 
about volume revert by snapshot[1].It is so good for rebuild operation.In 
short,I have two ideas,one is change the volume revert by snapshot spec to 
re-image spec,not only it can let the volume revert by snapshot,but also can 
re-image the volume which the image's size is greater than 0;another idea is 
add a only re-image spec,it only can re-image the volume which the image's size 
is greater than 0.
  What do you think of the two ideas?Any suggestion is welcome.Thank you!
  Note:the instance snapshot for image backed server's image size is 
greater than 0,but the volume backed server 's image size is equal 0.
  Re:
  [0]https://review.openstack.org/#/c/532407/
  
[1]https://specs.openstack.org/openstack/cinder-specs/specs/pike/cinder-volume-revert-by-snapshot.html














Best Regards
Rambo__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev