Re: [ceph-users] Migrating to new pools

2018-02-26 Thread Jason Dillaman 
On Mon, Feb 26, 2018 at 9:56 AM, Eugen Block  wrote:
> I'm following up on the rbd export/import option with a little delay.
>
> The fact that the snapshot is not protected after the image is reimported is
> not a big problem, you could deal with that or wait for a fix.
> But there's one major problem using this method: the VMs lose their
> rbd_children and parent data!

Correct -- the images are "flattened". The data is consistent but you
lose any savings from the re-use of non-overwritten parent data.

> Although the imported VM launches successfully, it has no parent
> information. So this will eventually lead to a problem reading data from the
> parent image, I assume.
>
> This brings up another issue: deleting glance images is now easily possible
> since the image has no clones. And if the VM loses its base image it
> probably will run into a failed state.
>
> ---cut here---
> # New glance image with new VM
> root@control:~ # rbd children
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
> cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk
>
> # Parent data available
> root@control:~ # rbd info cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk |
> grep parent
> parent: glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
>
>
> # Export base image
> root@control:~ # rbd export --export-format 2
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1 /var/lib/glance/images/cirros
> Exporting image: 100% complete...done.
>
> # Export VM's disk
> root@control:~ # rbd export --export-format 2
> cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk
> /var/lib/glance/images/cirros_disk
> Exporting image: 100% complete...done.
>
>
> # Delete VM
> root@control:~ # rbd rm cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk
> Removing image: 100% complete...done.
>
> # Reimport VM's disk
> root@control:~ # rbd import --export-format 2
> /var/lib/glance/images/cirros_disk
> cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk
> Importing image: 100% complete...done.
>
>
> # Delete glance image
> root@control:~ # rbd snap unprotect
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
> root@control:~ # rbd snap purge glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1
> Removing all snapshots: 100% complete...done.
> root@control:~ # rbd rm glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1
> Removing image: 100% complete...done.
>
> # Reimport glance image
> root@control:~ # rbd import --export-format 2 /var/lib/glance/images/cirros
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1
> Importing image: 100% complete...done.
>
> root@control:~ # rbd snap protect
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
>
> # There are no children
> root@control:~ # rbd children
> glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
> root@control:~ #
>
> # VM starts successfully
> root@control:~ # nova start c1
> Request to start server c1 has been accepted.
>
> # But no data in rbd_children
> root@control:~ # rados -p cinder listomapvals rbd_children
> root@control:~ #
> ---cut here---
>
> So in conclusion, this method is not suited for OpenStack. You could
> probably consider it in case of desaster recovery for single VMs, but not
> for a whole cloud environment where you would lose all relationships between
> base images and their clones.
>
> Regards,
> Eugen
>
>
> Zitat von Eugen Block :
>
>
>> Hi,
>>
>> I created a ticket for the rbd import issue:
>>
>> https://tracker.ceph.com/issues/23038
>>
>> Regards,
>> Eugen
>>
>>
>> Zitat von Jason Dillaman :
>>
>>> On Fri, Feb 16, 2018 at 11:20 AM, Eugen Block  wrote:

 Hi Jason,

> ... also forgot to mention "rbd export --export-format 2" / "rbd
> import --export-format 2" that will also deeply export/import all
> snapshots associated with an image and that feature is available in
> the Luminous release.



 thanks for that information, this could be very valuable for us. I'll
 have
 to test that intesively, but not before next week.

 But a first quick test brought up a couple of issues which I'll have to
 re-check before bringing them up here.

 One issue is worth mentioning, though: After I exported (rbd export
 --export-format ...) a glance image and imported it back to a different
 pool
 (rbd import --export-format ...) its snapshot was copied, but not
 protected.
 This prevented nova from cloning the base image and leaving that
 instance in
 error state. Protecting the snapshot manually and launch another
 instance
 enabled nova to clone the image successfully.

 Could this be worth a bug report or is it rather something I did wrong
 or
 missed?
>>>
>>>
>>> Definitely deserves a bug tracker ticket opened. Thanks.
>>>
 I wish you all a nice weekend!

 Regards
 Eugen


 Zitat von Jason Dillaman :

> On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman

Re: [ceph-users] Migrating to new pools

2018-02-26 Thread Eugen Block 

I'm following up on the rbd export/import option with a little delay.

The fact that the snapshot is not protected after the image is  
reimported is not a big problem, you could deal with that or wait for  
a fix.
But there's one major problem using this method: the VMs lose their  
rbd_children and parent data!


Although the imported VM launches successfully, it has no parent  
information. So this will eventually lead to a problem reading data  
from the parent image, I assume.


This brings up another issue: deleting glance images is now easily  
possible since the image has no clones. And if the VM loses its base  
image it probably will run into a failed state.


---cut here---
# New glance image with new VM
root@control:~ # rbd children glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk

# Parent data available
root@control:~ # rbd info  
cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk | grep parent

parent: glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap


# Export base image
root@control:~ # rbd export --export-format 2  
glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1  
/var/lib/glance/images/cirros

Exporting image: 100% complete...done.

# Export VM's disk
root@control:~ # rbd export --export-format 2  
cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk  
/var/lib/glance/images/cirros_disk

Exporting image: 100% complete...done.


# Delete VM
root@control:~ # rbd rm cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk
Removing image: 100% complete...done.

# Reimport VM's disk
root@control:~ # rbd import --export-format 2  
/var/lib/glance/images/cirros_disk  
cinder/f43265e9-beab-4f83-be46-a51da013f70a_disk

Importing image: 100% complete...done.


# Delete glance image
root@control:~ # rbd snap unprotect  
glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap

root@control:~ # rbd snap purge glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1
Removing all snapshots: 100% complete...done.
root@control:~ # rbd rm glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1
Removing image: 100% complete...done.

# Reimport glance image
root@control:~ # rbd import --export-format 2  
/var/lib/glance/images/cirros  
glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1

Importing image: 100% complete...done.

root@control:~ # rbd snap protect  
glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap


# There are no children
root@control:~ # rbd children glance/4479820a-d58b-4ac6-ba2a-e8871b24fcb1@snap
root@control:~ #

# VM starts successfully
root@control:~ # nova start c1
Request to start server c1 has been accepted.

# But no data in rbd_children
root@control:~ # rados -p cinder listomapvals rbd_children
root@control:~ #
---cut here---

So in conclusion, this method is not suited for OpenStack. You could  
probably consider it in case of desaster recovery for single VMs, but  
not for a whole cloud environment where you would lose all  
relationships between base images and their clones.


Regards,
Eugen


Zitat von Eugen Block :


Hi,

I created a ticket for the rbd import issue:

https://tracker.ceph.com/issues/23038

Regards,
Eugen


Zitat von Jason Dillaman :


On Fri, Feb 16, 2018 at 11:20 AM, Eugen Block  wrote:

Hi Jason,


... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.



thanks for that information, this could be very valuable for us. I'll have
to test that intesively, but not before next week.

But a first quick test brought up a couple of issues which I'll have to
re-check before bringing them up here.

One issue is worth mentioning, though: After I exported (rbd export
--export-format ...) a glance image and imported it back to a  
different pool
(rbd import --export-format ...) its snapshot was copied, but not  
protected.
This prevented nova from cloning the base image and leaving that  
instance in

error state. Protecting the snapshot manually and launch another instance
enabled nova to clone the image successfully.

Could this be worth a bug report or is it rather something I did wrong or
missed?


Definitely deserves a bug tracker ticket opened. Thanks.


I wish you all a nice weekend!

Regards
Eugen


Zitat von Jason Dillaman :


On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman 
wrote:


On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:


Dear list, hello Jason,

you may have seen my message on the Ceph mailing list about RDB pool
migration - it's a common subject that pools were created in a
sub-optimum
fashion and i. e. pgnum is (not yet) reducible, so we're looking into
means
to "clone" an RBD pool into a new pool within the same cluster
(including
snapshots).

We had looked into creating a tool for this job, but soon noticed that
we're
duplicating basic functionality of rbd-mirror.

Re: [ceph-users] Migrating to new pools

2018-02-22 Thread Anthony D'Atri 
Cumulative followup to various insightful replies.

I wrote:

 No, it's not really possible currently and we have no plans to add
>>> such support since it would not be of any long-term value.
>> 
>> The long-term value would be the ability to migrate volumes from, say, a 
>> replicated pool to an an EC pool without extended downtime.

Among the replies:

> That's why the Mimic release should offer a specific set of "rbd
> migration XYX" actions to perform minimal downtime migrations.

That'd be awesome, if it can properly handle client-side attachments.

> I was saying that hacking around rbd-mirror to add such a feature has
> limited long-term value given the release plan. Plus, in the immediate
> term you can use a kernel md RAID device or QEMU+librbd to perform the
> migration for you w/ minimal downtime (albeit with potentially more
> hands-on setup involved).

Great ideas if one controls both the back end and the guest OS.  In our case we 
can't go mucking around inside the guests.

> if you use qemu, it's also possible to use drive-mirror feature from qemu.
> (can mirror and migrate from 1 storage to another storage without downtime).


Interesting idea, and a qemu function I was not previously aware of.  Has some 
potential wrinkles though:

o Considerably more network traffic and I/O load to/from the hypervisor (or 
whatever you call the hosts where your guest VMs run)
o Scaling to thousands of volumes each potentially TBs in size
o Handling unattached volumes
o Co-ordination with / prevention of user attach/detach operations during the 
process

-- aad


___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-22 Thread Jason Dillaman 
On Wed, Feb 21, 2018 at 7:27 PM, Anthony D'Atri  wrote:
>>> I was thinking we might be able to configure/hack rbd mirroring to mirror to
>>> a pool on the same cluster but I gather from the OP and your post that this
>>> is not really possible?
>>
>> No, it's not really possible currently and we have no plans to add
>> such support since it would not be of any long-term value.
>
> The long-term value would be the ability to migrate volumes from, say, a 
> replicated pool to an an EC pool without extended downtime.
>

That's why the Mimic release should offer a specific set of "rbd
migration XYX" actions to perform minimal downtime migrations. I was
saying that hacking around rbd-mirror to add such a feature has
limited long-term value given the release plan. Plus, in the immediate
term you can use a kernel md RAID device or QEMU+librbd to perform the
migration for you w/ minimal downtime (albeit with potentially more
hands-on setup involved).

>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



-- 
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-21 Thread David Turner 
I recently migrated several VMs from an HDD pool to an SSD pool without any
downtime with proxmox. It is definitely possible with qemu to do no
downtime migrations between pools.

On Wed, Feb 21, 2018, 8:32 PM Alexandre DERUMIER <aderum...@odiso.com>
wrote:

> Hi,
>
> if you use qemu, it's also possible to use drive-mirror feature from qemu.
> (can mirror and migrate from 1 storage to another storage without
> downtime).
>
> I don't known if openstack has implemented it, but It's working fine on
> proxmox.
>
>
> - Mail original -
> De: "Anthony D'Atri" <a...@dreamsnake.net>
> À: "ceph-users" <ceph-users@lists.ceph.com>
> Envoyé: Jeudi 22 Février 2018 01:27:23
> Objet: Re: [ceph-users] Migrating to new pools
>
> >> I was thinking we might be able to configure/hack rbd mirroring to
> mirror to
> >> a pool on the same cluster but I gather from the OP and your post that
> this
> >> is not really possible?
> >
> > No, it's not really possible currently and we have no plans to add
> > such support since it would not be of any long-term value.
>
> The long-term value would be the ability to migrate volumes from, say, a
> replicated pool to an an EC pool without extended downtime.
>
>
>
>
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-21 Thread Alexandre DERUMIER 
Hi,

if you use qemu, it's also possible to use drive-mirror feature from qemu.
(can mirror and migrate from 1 storage to another storage without downtime).

I don't known if openstack has implemented it, but It's working fine on proxmox.


- Mail original -
De: "Anthony D'Atri" <a...@dreamsnake.net>
À: "ceph-users" <ceph-users@lists.ceph.com>
Envoyé: Jeudi 22 Février 2018 01:27:23
Objet: Re: [ceph-users] Migrating to new pools

>> I was thinking we might be able to configure/hack rbd mirroring to mirror to 
>> a pool on the same cluster but I gather from the OP and your post that this 
>> is not really possible? 
> 
> No, it's not really possible currently and we have no plans to add 
> such support since it would not be of any long-term value. 

The long-term value would be the ability to migrate volumes from, say, a 
replicated pool to an an EC pool without extended downtime. 





___ 
ceph-users mailing list 
ceph-users@lists.ceph.com 
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com 

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-21 Thread Anthony D'Atri 
>> I was thinking we might be able to configure/hack rbd mirroring to mirror to
>> a pool on the same cluster but I gather from the OP and your post that this
>> is not really possible?
> 
> No, it's not really possible currently and we have no plans to add
> such support since it would not be of any long-term value.

The long-term value would be the ability to migrate volumes from, say, a 
replicated pool to an an EC pool without extended downtime.





___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-21 Thread Jason Dillaman 
On Tue, Feb 20, 2018 at 8:35 PM, Rafael Lopez  wrote:
>> There is also work-in-progress for online
>> image migration [1] that will allow you to keep using the image while
>> it's being migrated to a new destination image.
>
>
> Hi Jason,
>
> Is there any recommended method/workaround for live rbd migration in
> luminous? eg. snapshot/copy or export/import then export/import-diff?
> We are looking at options for moving large RBDs (100T) to a new pool with
> minimal downtime.
>
> I was thinking we might be able to configure/hack rbd mirroring to mirror to
> a pool on the same cluster but I gather from the OP and your post that this
> is not really possible?

No, it's not really possible currently and we have no plans to add
such support since it would not be of any long-term value. If you are
using RBD with a kernel block device, you could temporarily wrap two
mapped RBD volumes (original and new) under a md RAID1 with the
original as the primary -- and then do a RAID repair. If using
QEMU+librbd, you could use its built-in block live migration feature
(I've never played w/ this before and I believe you would need to use
the QEMU monitor instead of libvirt to configure).

> --
> Rafael Lopez
> Research Devops Engineer
> Monash University eResearch Centre
>
>



-- 
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-20 Thread Rafael Lopez 
>
> There is also work-in-progress for online
> image migration [1] that will allow you to keep using the image while
> it's being migrated to a new destination image.


Hi Jason,

Is there any recommended method/workaround for live rbd migration in
luminous? eg. snapshot/copy or export/import then export/import-diff?
We are looking at options for moving large RBDs (100T) to a new pool with
minimal downtime.

I was thinking we might be able to configure/hack rbd mirroring to mirror
to a pool on the same cluster but I gather from the OP and your post that
this is not really possible?

-- 
*Rafael Lopez*
Research Devops Engineer
Monash University eResearch Centre
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-19 Thread Jason Dillaman 
Thanks!

On Mon, Feb 19, 2018 at 10:33 AM, Eugen Block  wrote:
> Hi,
>
> I created a ticket for the rbd import issue:
>
> https://tracker.ceph.com/issues/23038
>
> Regards,
>
> Eugen
>
>
> Zitat von Jason Dillaman :
>
>> On Fri, Feb 16, 2018 at 11:20 AM, Eugen Block  wrote:
>>>
>>> Hi Jason,
>>>
 ... also forgot to mention "rbd export --export-format 2" / "rbd
 import --export-format 2" that will also deeply export/import all
 snapshots associated with an image and that feature is available in
 the Luminous release.
>>>
>>>
>>>
>>> thanks for that information, this could be very valuable for us. I'll
>>> have
>>> to test that intesively, but not before next week.
>>>
>>> But a first quick test brought up a couple of issues which I'll have to
>>> re-check before bringing them up here.
>>>
>>> One issue is worth mentioning, though: After I exported (rbd export
>>> --export-format ...) a glance image and imported it back to a different
>>> pool
>>> (rbd import --export-format ...) its snapshot was copied, but not
>>> protected.
>>> This prevented nova from cloning the base image and leaving that instance
>>> in
>>> error state. Protecting the snapshot manually and launch another instance
>>> enabled nova to clone the image successfully.
>>>
>>> Could this be worth a bug report or is it rather something I did wrong or
>>> missed?
>>
>>
>> Definitely deserves a bug tracker ticket opened. Thanks.
>>
>>> I wish you all a nice weekend!
>>>
>>> Regards
>>> Eugen
>>>
>>>
>>> Zitat von Jason Dillaman :
>>>
 On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman 
 wrote:
>
>
> On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen 
> wrote:
>>
>>
>> Dear list, hello Jason,
>>
>> you may have seen my message on the Ceph mailing list about RDB pool
>> migration - it's a common subject that pools were created in a
>> sub-optimum
>> fashion and i. e. pgnum is (not yet) reducible, so we're looking into
>> means
>> to "clone" an RBD pool into a new pool within the same cluster
>> (including
>> snapshots).
>>
>> We had looked into creating a tool for this job, but soon noticed that
>> we're
>> duplicating basic functionality of rbd-mirror. So we tested the
>> following,
>> which worked out nicely:
>>
>> - create a test cluster (Ceph cluster plus an Openstack cluster using
>> an
>> RBD
>> pool) and some Openstack instances
>>
>> - create a second Ceph test cluster
>>
>> - stop Openstack
>>
>> - use rbd-mirror to clone the RBD pool from the first to the second
>> Ceph
>> cluster (IOW aborting rbd-mirror once the initial coping was done)
>>
>> - recreate the RDB pool on the first cluster
>>
>> - use rbd-mirror to clone the mirrored pool back to the (newly
>> created)
>> pool
>> on the first cluster
>>
>> - start Openstack and work with the (recreated) pool on the first
>> cluster
>>
>> So using rbd-mirror, we could clone an RBD pool's content to a
>> differently
>> structured pool on the same cluster - by using an intermediate
>> cluster.
>>
>> @Jason: Looking at the commit history for rbd-mirror, it seems you
>> might
>> be
>> able to shed some light on this: Do you see an easy way to modify
>> rbd-mirror
>> in such a fashion that instead of mirroring to a pool on a different
>> cluster
>> (having the same pool name as the original), mirroring would be to a
>> pool on
>> the *same* cluster, (obviously having a pool different name)?
>>
>> From the "rbd cppool" perspective, a one-shot mode of operation would
>> be
>> fully sufficient - but looking at the code, I have not even been able
>> to
>> identify the spots where we might "cut away" the networking part, so
>> that
>> rbd-mirror might do an intra-cluster job.
>>
>> Are you able to judge how much work would need to be done, in order to
>> create a one-shot, intra-cluster version of rbd-mirror? Might it even
>> be
>> something that could be a simple enhancement?
>
>
>
> You might be interested in the deep-copy feature that will be included
> in the Mimic release. By running "rbd deep-copy 
> ", it will fully copy the image, including snapshots and
> parentage, to a new image. There is also work-in-progress for online
> image migration [1] that will allow you to keep using the image while
> it's being migrated to a new destination image. Both of these are
> probably more suited to your needs than the heavy-weight RBD mirroring
> process -- especially if you are only interested in the first step
> since RBD mirroring now directly utilizes the deep-copy feature for
> the initial image sync.



 ...

Re: [ceph-users] Migrating to new pools

2018-02-19 Thread Eugen Block 

Hi,

I created a ticket for the rbd import issue:

https://tracker.ceph.com/issues/23038

Regards,
Eugen


Zitat von Jason Dillaman :


On Fri, Feb 16, 2018 at 11:20 AM, Eugen Block  wrote:

Hi Jason,


... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.



thanks for that information, this could be very valuable for us. I'll have
to test that intesively, but not before next week.

But a first quick test brought up a couple of issues which I'll have to
re-check before bringing them up here.

One issue is worth mentioning, though: After I exported (rbd export
--export-format ...) a glance image and imported it back to a different pool
(rbd import --export-format ...) its snapshot was copied, but not protected.
This prevented nova from cloning the base image and leaving that instance in
error state. Protecting the snapshot manually and launch another instance
enabled nova to clone the image successfully.

Could this be worth a bug report or is it rather something I did wrong or
missed?


Definitely deserves a bug tracker ticket opened. Thanks.


I wish you all a nice weekend!

Regards
Eugen


Zitat von Jason Dillaman :


On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman 
wrote:


On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:


Dear list, hello Jason,

you may have seen my message on the Ceph mailing list about RDB pool
migration - it's a common subject that pools were created in a
sub-optimum
fashion and i. e. pgnum is (not yet) reducible, so we're looking into
means
to "clone" an RBD pool into a new pool within the same cluster
(including
snapshots).

We had looked into creating a tool for this job, but soon noticed that
we're
duplicating basic functionality of rbd-mirror. So we tested the
following,
which worked out nicely:

- create a test cluster (Ceph cluster plus an Openstack cluster using an
RBD
pool) and some Openstack instances

- create a second Ceph test cluster

- stop Openstack

- use rbd-mirror to clone the RBD pool from the first to the second Ceph
cluster (IOW aborting rbd-mirror once the initial coping was done)

- recreate the RDB pool on the first cluster

- use rbd-mirror to clone the mirrored pool back to the (newly created)
pool
on the first cluster

- start Openstack and work with the (recreated) pool on the first
cluster

So using rbd-mirror, we could clone an RBD pool's content to a
differently
structured pool on the same cluster - by using an intermediate cluster.

@Jason: Looking at the commit history for rbd-mirror, it seems you might
be
able to shed some light on this: Do you see an easy way to modify
rbd-mirror
in such a fashion that instead of mirroring to a pool on a different
cluster
(having the same pool name as the original), mirroring would be to a
pool on
the *same* cluster, (obviously having a pool different name)?

From the "rbd cppool" perspective, a one-shot mode of operation would be
fully sufficient - but looking at the code, I have not even been able to
identify the spots where we might "cut away" the networking part, so
that
rbd-mirror might do an intra-cluster job.

Are you able to judge how much work would need to be done, in order to
create a one-shot, intra-cluster version of rbd-mirror? Might it even be
something that could be a simple enhancement?



You might be interested in the deep-copy feature that will be included
in the Mimic release. By running "rbd deep-copy 
", it will fully copy the image, including snapshots and
parentage, to a new image. There is also work-in-progress for online
image migration [1] that will allow you to keep using the image while
it's being migrated to a new destination image. Both of these are
probably more suited to your needs than the heavy-weight RBD mirroring
process -- especially if you are only interested in the first step
since RBD mirroring now directly utilizes the deep-copy feature for
the initial image sync.



... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.


Thank you for any information and / or opinion you care to share!

With regards,
Jens



[1] https://github.com/ceph/ceph/pull/15831

--
Jason





--
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com





--
Eugen Block voice   : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG  fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail  : ebl...@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen
  Sitz und Registergericht:

Re: [ceph-users] Migrating to new pools

2018-02-16 Thread Jason Dillaman 
On Fri, Feb 16, 2018 at 11:20 AM, Eugen Block  wrote:
> Hi Jason,
>
>> ... also forgot to mention "rbd export --export-format 2" / "rbd
>> import --export-format 2" that will also deeply export/import all
>> snapshots associated with an image and that feature is available in
>> the Luminous release.
>
>
> thanks for that information, this could be very valuable for us. I'll have
> to test that intesively, but not before next week.
>
> But a first quick test brought up a couple of issues which I'll have to
> re-check before bringing them up here.
>
> One issue is worth mentioning, though: After I exported (rbd export
> --export-format ...) a glance image and imported it back to a different pool
> (rbd import --export-format ...) its snapshot was copied, but not protected.
> This prevented nova from cloning the base image and leaving that instance in
> error state. Protecting the snapshot manually and launch another instance
> enabled nova to clone the image successfully.
>
> Could this be worth a bug report or is it rather something I did wrong or
> missed?

Definitely deserves a bug tracker ticket opened. Thanks.

> I wish you all a nice weekend!
>
> Regards
> Eugen
>
>
> Zitat von Jason Dillaman :
>
>> On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman 
>> wrote:
>>>
>>> On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:

 Dear list, hello Jason,

 you may have seen my message on the Ceph mailing list about RDB pool
 migration - it's a common subject that pools were created in a
 sub-optimum
 fashion and i. e. pgnum is (not yet) reducible, so we're looking into
 means
 to "clone" an RBD pool into a new pool within the same cluster
 (including
 snapshots).

 We had looked into creating a tool for this job, but soon noticed that
 we're
 duplicating basic functionality of rbd-mirror. So we tested the
 following,
 which worked out nicely:

 - create a test cluster (Ceph cluster plus an Openstack cluster using an
 RBD
 pool) and some Openstack instances

 - create a second Ceph test cluster

 - stop Openstack

 - use rbd-mirror to clone the RBD pool from the first to the second Ceph
 cluster (IOW aborting rbd-mirror once the initial coping was done)

 - recreate the RDB pool on the first cluster

 - use rbd-mirror to clone the mirrored pool back to the (newly created)
 pool
 on the first cluster

 - start Openstack and work with the (recreated) pool on the first
 cluster

 So using rbd-mirror, we could clone an RBD pool's content to a
 differently
 structured pool on the same cluster - by using an intermediate cluster.

 @Jason: Looking at the commit history for rbd-mirror, it seems you might
 be
 able to shed some light on this: Do you see an easy way to modify
 rbd-mirror
 in such a fashion that instead of mirroring to a pool on a different
 cluster
 (having the same pool name as the original), mirroring would be to a
 pool on
 the *same* cluster, (obviously having a pool different name)?

 From the "rbd cppool" perspective, a one-shot mode of operation would be
 fully sufficient - but looking at the code, I have not even been able to
 identify the spots where we might "cut away" the networking part, so
 that
 rbd-mirror might do an intra-cluster job.

 Are you able to judge how much work would need to be done, in order to
 create a one-shot, intra-cluster version of rbd-mirror? Might it even be
 something that could be a simple enhancement?
>>>
>>>
>>> You might be interested in the deep-copy feature that will be included
>>> in the Mimic release. By running "rbd deep-copy 
>>> ", it will fully copy the image, including snapshots and
>>> parentage, to a new image. There is also work-in-progress for online
>>> image migration [1] that will allow you to keep using the image while
>>> it's being migrated to a new destination image. Both of these are
>>> probably more suited to your needs than the heavy-weight RBD mirroring
>>> process -- especially if you are only interested in the first step
>>> since RBD mirroring now directly utilizes the deep-copy feature for
>>> the initial image sync.
>>
>>
>> ... also forgot to mention "rbd export --export-format 2" / "rbd
>> import --export-format 2" that will also deeply export/import all
>> snapshots associated with an image and that feature is available in
>> the Luminous release.
>>
 Thank you for any information and / or opinion you care to share!

 With regards,
 Jens

>>>
>>> [1] https://github.com/ceph/ceph/pull/15831
>>>
>>> --
>>> Jason
>>
>>
>>
>>
>> --
>> Jason
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>>

Re: [ceph-users] Migrating to new pools

2018-02-16 Thread Eugen Block 

Hi Jason,


... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.


thanks for that information, this could be very valuable for us. I'll  
have to test that intesively, but not before next week.


But a first quick test brought up a couple of issues which I'll have  
to re-check before bringing them up here.


One issue is worth mentioning, though: After I exported (rbd export  
--export-format ...) a glance image and imported it back to a  
different pool (rbd import --export-format ...) its snapshot was  
copied, but not protected. This prevented nova from cloning the base  
image and leaving that instance in error state. Protecting the  
snapshot manually and launch another instance enabled nova to clone  
the image successfully.


Could this be worth a bug report or is it rather something I did wrong  
or missed?


I wish you all a nice weekend!

Regards
Eugen


Zitat von Jason Dillaman :


On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman  wrote:

On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:

Dear list, hello Jason,

you may have seen my message on the Ceph mailing list about RDB pool
migration - it's a common subject that pools were created in a sub-optimum
fashion and i. e. pgnum is (not yet) reducible, so we're looking into means
to "clone" an RBD pool into a new pool within the same cluster (including
snapshots).

We had looked into creating a tool for this job, but soon noticed  
that we're

duplicating basic functionality of rbd-mirror. So we tested the following,
which worked out nicely:

- create a test cluster (Ceph cluster plus an Openstack cluster  
using an RBD

pool) and some Openstack instances

- create a second Ceph test cluster

- stop Openstack

- use rbd-mirror to clone the RBD pool from the first to the second Ceph
cluster (IOW aborting rbd-mirror once the initial coping was done)

- recreate the RDB pool on the first cluster

- use rbd-mirror to clone the mirrored pool back to the (newly  
created) pool

on the first cluster

- start Openstack and work with the (recreated) pool on the first cluster

So using rbd-mirror, we could clone an RBD pool's content to a differently
structured pool on the same cluster - by using an intermediate cluster.

@Jason: Looking at the commit history for rbd-mirror, it seems you might be
able to shed some light on this: Do you see an easy way to modify  
rbd-mirror
in such a fashion that instead of mirroring to a pool on a  
different cluster
(having the same pool name as the original), mirroring would be to  
a pool on

the *same* cluster, (obviously having a pool different name)?

From the "rbd cppool" perspective, a one-shot mode of operation would be
fully sufficient - but looking at the code, I have not even been able to
identify the spots where we might "cut away" the networking part, so that
rbd-mirror might do an intra-cluster job.

Are you able to judge how much work would need to be done, in order to
create a one-shot, intra-cluster version of rbd-mirror? Might it even be
something that could be a simple enhancement?


You might be interested in the deep-copy feature that will be included
in the Mimic release. By running "rbd deep-copy 
", it will fully copy the image, including snapshots and
parentage, to a new image. There is also work-in-progress for online
image migration [1] that will allow you to keep using the image while
it's being migrated to a new destination image. Both of these are
probably more suited to your needs than the heavy-weight RBD mirroring
process -- especially if you are only interested in the first step
since RBD mirroring now directly utilizes the deep-copy feature for
the initial image sync.


... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.


Thank you for any information and / or opinion you care to share!

With regards,
Jens



[1] https://github.com/ceph/ceph/pull/15831

--
Jason




--
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




--
Eugen Block voice   : +49-40-559 51 75
NDE Netzdesign und -entwicklung AG  fax : +49-40-559 51 77
Postfach 61 03 15
D-22423 Hamburg e-mail  : ebl...@nde.ag

Vorsitzende des Aufsichtsrates: Angelika Mozdzen
  Sitz und Registergericht: Hamburg, HRB 90934
  Vorstand: Jens-U. Mozdzen
   USt-IdNr. DE 814 013 983

___
ceph-users mailing list
ceph-users@lists.ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-16 Thread Jason Dillaman 
On Fri, Feb 16, 2018 at 8:08 AM, Jason Dillaman  wrote:
> On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:
>> Dear list, hello Jason,
>>
>> you may have seen my message on the Ceph mailing list about RDB pool
>> migration - it's a common subject that pools were created in a sub-optimum
>> fashion and i. e. pgnum is (not yet) reducible, so we're looking into means
>> to "clone" an RBD pool into a new pool within the same cluster (including
>> snapshots).
>>
>> We had looked into creating a tool for this job, but soon noticed that we're
>> duplicating basic functionality of rbd-mirror. So we tested the following,
>> which worked out nicely:
>>
>> - create a test cluster (Ceph cluster plus an Openstack cluster using an RBD
>> pool) and some Openstack instances
>>
>> - create a second Ceph test cluster
>>
>> - stop Openstack
>>
>> - use rbd-mirror to clone the RBD pool from the first to the second Ceph
>> cluster (IOW aborting rbd-mirror once the initial coping was done)
>>
>> - recreate the RDB pool on the first cluster
>>
>> - use rbd-mirror to clone the mirrored pool back to the (newly created) pool
>> on the first cluster
>>
>> - start Openstack and work with the (recreated) pool on the first cluster
>>
>> So using rbd-mirror, we could clone an RBD pool's content to a differently
>> structured pool on the same cluster - by using an intermediate cluster.
>>
>> @Jason: Looking at the commit history for rbd-mirror, it seems you might be
>> able to shed some light on this: Do you see an easy way to modify rbd-mirror
>> in such a fashion that instead of mirroring to a pool on a different cluster
>> (having the same pool name as the original), mirroring would be to a pool on
>> the *same* cluster, (obviously having a pool different name)?
>>
>> From the "rbd cppool" perspective, a one-shot mode of operation would be
>> fully sufficient - but looking at the code, I have not even been able to
>> identify the spots where we might "cut away" the networking part, so that
>> rbd-mirror might do an intra-cluster job.
>>
>> Are you able to judge how much work would need to be done, in order to
>> create a one-shot, intra-cluster version of rbd-mirror? Might it even be
>> something that could be a simple enhancement?
>
> You might be interested in the deep-copy feature that will be included
> in the Mimic release. By running "rbd deep-copy 
> ", it will fully copy the image, including snapshots and
> parentage, to a new image. There is also work-in-progress for online
> image migration [1] that will allow you to keep using the image while
> it's being migrated to a new destination image. Both of these are
> probably more suited to your needs than the heavy-weight RBD mirroring
> process -- especially if you are only interested in the first step
> since RBD mirroring now directly utilizes the deep-copy feature for
> the initial image sync.

... also forgot to mention "rbd export --export-format 2" / "rbd
import --export-format 2" that will also deeply export/import all
snapshots associated with an image and that feature is available in
the Luminous release.

>> Thank you for any information and / or opinion you care to share!
>>
>> With regards,
>> Jens
>>
>
> [1] https://github.com/ceph/ceph/pull/15831
>
> --
> Jason



-- 
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-16 Thread Jason Dillaman 
On Fri, Feb 16, 2018 at 5:36 AM, Jens-U. Mozdzen  wrote:
> Dear list, hello Jason,
>
> you may have seen my message on the Ceph mailing list about RDB pool
> migration - it's a common subject that pools were created in a sub-optimum
> fashion and i. e. pgnum is (not yet) reducible, so we're looking into means
> to "clone" an RBD pool into a new pool within the same cluster (including
> snapshots).
>
> We had looked into creating a tool for this job, but soon noticed that we're
> duplicating basic functionality of rbd-mirror. So we tested the following,
> which worked out nicely:
>
> - create a test cluster (Ceph cluster plus an Openstack cluster using an RBD
> pool) and some Openstack instances
>
> - create a second Ceph test cluster
>
> - stop Openstack
>
> - use rbd-mirror to clone the RBD pool from the first to the second Ceph
> cluster (IOW aborting rbd-mirror once the initial coping was done)
>
> - recreate the RDB pool on the first cluster
>
> - use rbd-mirror to clone the mirrored pool back to the (newly created) pool
> on the first cluster
>
> - start Openstack and work with the (recreated) pool on the first cluster
>
> So using rbd-mirror, we could clone an RBD pool's content to a differently
> structured pool on the same cluster - by using an intermediate cluster.
>
> @Jason: Looking at the commit history for rbd-mirror, it seems you might be
> able to shed some light on this: Do you see an easy way to modify rbd-mirror
> in such a fashion that instead of mirroring to a pool on a different cluster
> (having the same pool name as the original), mirroring would be to a pool on
> the *same* cluster, (obviously having a pool different name)?
>
> From the "rbd cppool" perspective, a one-shot mode of operation would be
> fully sufficient - but looking at the code, I have not even been able to
> identify the spots where we might "cut away" the networking part, so that
> rbd-mirror might do an intra-cluster job.
>
> Are you able to judge how much work would need to be done, in order to
> create a one-shot, intra-cluster version of rbd-mirror? Might it even be
> something that could be a simple enhancement?

You might be interested in the deep-copy feature that will be included
in the Mimic release. By running "rbd deep-copy 
", it will fully copy the image, including snapshots and
parentage, to a new image. There is also work-in-progress for online
image migration [1] that will allow you to keep using the image while
it's being migrated to a new destination image. Both of these are
probably more suited to your needs than the heavy-weight RBD mirroring
process -- especially if you are only interested in the first step
since RBD mirroring now directly utilizes the deep-copy feature for
the initial image sync.

> Thank you for any information and / or opinion you care to share!
>
> With regards,
> Jens
>

[1] https://github.com/ceph/ceph/pull/15831

-- 
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-02-16 Thread Jens-U. Mozdzen 

Dear list, hello Jason,

you may have seen my message on the Ceph mailing list about RDB pool  
migration - it's a common subject that pools were created in a  
sub-optimum fashion and i. e. pgnum is (not yet) reducible, so we're  
looking into means to "clone" an RBD pool into a new pool within the  
same cluster (including snapshots).


We had looked into creating a tool for this job, but soon noticed that  
we're duplicating basic functionality of rbd-mirror. So we tested the  
following, which worked out nicely:


- create a test cluster (Ceph cluster plus an Openstack cluster using  
an RBD pool) and some Openstack instances


- create a second Ceph test cluster

- stop Openstack

- use rbd-mirror to clone the RBD pool from the first to the second  
Ceph cluster (IOW aborting rbd-mirror once the initial coping was done)


- recreate the RDB pool on the first cluster

- use rbd-mirror to clone the mirrored pool back to the (newly  
created) pool on the first cluster


- start Openstack and work with the (recreated) pool on the first cluster

So using rbd-mirror, we could clone an RBD pool's content to a  
differently structured pool on the same cluster - by using an  
intermediate cluster.


@Jason: Looking at the commit history for rbd-mirror, it seems you  
might be able to shed some light on this: Do you see an easy way to  
modify rbd-mirror in such a fashion that instead of mirroring to a  
pool on a different cluster (having the same pool name as the  
original), mirroring would be to a pool on the *same* cluster,  
(obviously having a pool different name)?


From the "rbd cppool" perspective, a one-shot mode of operation would  
be fully sufficient - but looking at the code, I have not even been  
able to identify the spots where we might "cut away" the networking  
part, so that rbd-mirror might do an intra-cluster job.


Are you able to judge how much work would need to be done, in order to  
create a one-shot, intra-cluster version of rbd-mirror? Might it even  
be something that could be a simple enhancement?


Thank you for any information and / or opinion you care to share!

With regards,
Jens

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools

2018-01-19 Thread Jens-U. Mozdzen 

Hi *,

Zitat von ceph-users-requ...@lists.ceph.com:

Hi *,

facing the problem to reduce the number of PGs for a pool, I've found
various information and suggestions, but no "definite guide" to handle
pool migration with Ceph 12.2.x. This seems to be a fairly common
problem when having to deal with "teen-age clusters", so consolidated
information would be a real help. I'm willing to start writing things
up, but don't want to duplicate information. So:

Are there any documented "operational procedures" on how to migrate

- an RBD pool (with snapshots created by Openstack)

- a CephFS data pool

- a CephFS metadata pool

to a different volume, in order to be able to utilize pool settings
that cannot be changed on an existing pool?

---

RBD pools: From what I've read, RBD snapshots are "broken" after using
"rados cppool" to move the content of an "RBD pool" to a new pool.


after having read up on "rbd-mirror" and having had a glimpse at its  
code, it seems that preserving clones, snapshots and their  
relationships has been solved for cluster-to-cluster migration.


Is this really correct? If so, it might be possible to extend the code  
in a fashion that will allow a one-shot, intracluster pool-to-pool  
migration as a spin-off to rbd-mirror.


I was thinking along the following lines:

- run rbd-mirror in a stand-alone fashion, just stating source and  
destination pool


- leave it to the cluster admin to take RDB "offline", so that the  
pool content does not change during the copy (no RBD journaling  
involved)


- check that the destination pool is empty. In a first version,  
cumulative migrates (joining multiple source pools with distinct image  
names) would complicate things ;)


- sync all content from source to destination pool, in a one-shot fashion

- done

Anyone out there who can judge the chances of that approach, better  
than me? I'd be willing to spend development time on this, but  
starting from scratch will be rather hard, so pointers at where to  
look at within the rbd-mirror code would be more than welcome.


Regards,
Jens

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools (RBD, CephFS)

2017-12-18 Thread Denes Dolhay 

Hi,


If the problem is not severe and you can wait, then according to this:

http://ceph.com/community/new-luminous-pg-overdose-protection/

there is a pg merge feature coming.


Regards,

Denes.


On 12/18/2017 02:18 PM, Jens-U. Mozdzen wrote:

Hi *,

facing the problem to reduce the number of PGs for a pool, I've found 
various information and suggestions, but no "definite guide" to handle 
pool migration with Ceph 12.2.x. This seems to be a fairly common 
problem when having to deal with "teen-age clusters", so consolidated 
information would be a real help. I'm willing to start writing things 
up, but don't want to duplicate information. So:


Are there any documented "operational procedures" on how to migrate

- an RBD pool (with snapshots created by Openstack)

- a CephFS data pool

- a CephFS metadata pool

to a different volume, in order to be able to utilize pool settings 
that cannot be changed on an existing pool?


---

RBD pools: From what I've read, RBD snapshots are "broken" after using 
"rados cppool" to move the content of an "RBD pool" to a new pool.


---

CephFS data pool: I know I can add additional pools to a CephFS 
instance ("ceph fs add_data_pool"), and have newly created files to be 
placed in the new pool ("file layouts"). But according to the docs, a 
small amount of metadata is kept in the primary data pool for all 
files, so I cannot remove the original pool.


I couldn't identify how CephFS (MDS) identifies it's current data pool 
(or "default data pool" in case of multiple pools - the one named in 
"ceph fs new"), so "rados cppool"-moving the data to a new pool and 
then reconfiguring CephFS to use the new pool (while MDS are stopped, 
of course) is not yet an option? And there might be references to the 
pool id hiding in CephFS metadata, too, invalidating this approach 
altogether.


Of course, dumping the current content of the CephFS to external 
storage and recreating the CephFS instance with new pools is a 
potential option, but may required a substantial amount of extra 
storage ;)


---

CephFS metadata pool: I've not seen any indication of a procedure to 
swap metadata pools.



I couldn't identify how CephFS (MDS) identifies it's current metadata 
pool, so "rados cppool"-moving the metadata to a new pool and then 
reconfiguring CephFS to use the new pool (while MDS are stopped, of 
course) is not yet an option?


Of course, dumping the current content of the CephFS to external 
storage and recreating the CephFS instance with new pools is a 
potential option, but may required a substantial amount of extra 
storage ;)


---

http://cephnotes.ksperis.com/blog/2015/04/15/ceph-pool-migration 
describes an interesting approach to migrate all pool contents by 
making the current pool a cache tier to the new pool and then migrate 
the "cache tier content" to the (new) base pool. But I'm not yet able 
to judge the approach and will have to conduct tests. Can anyone 
already make an educated guess if especially the "snapshot" problem 
for RBD pools will be circumvented this way and how CephFS will react 
to this approach? This "cache tier" approach, if feasible, would be a 
nice way to circumvent downtime and extra space requirements.


Thank you for any ideas, insight and experience you can share!

Regards,
J

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Migrating to new pools (RBD, CephFS)

2017-12-18 Thread Cary 
A possible option. They do not recommend using cppool.

http://lists.ceph.com/pipermail/ceph-users-ceph.com/2016-July/011460.html

**COMPLETELY UNTESTED AND DANGEROUS**

stop all MDS daemons
delete your filesystem (but leave the pools)
use "rados export" and "rados import" to do a full copy of the
metadata to a new pool (*not* cppool, it doesn't copy OMAP data)
use "ceph fs new" to create a new filesystem that uses your new metadata pool
use "ceph fs reset" to skip the creating phase of the new filesystem
start MDS daemons

**COMPLETELY UNTESTED AND DANGEROUS**


On Mon, Dec 18, 2017 at 1:18 PM, Jens-U. Mozdzen  wrote:
> Hi *,
>
> facing the problem to reduce the number of PGs for a pool, I've found
> various information and suggestions, but no "definite guide" to handle pool
> migration with Ceph 12.2.x. This seems to be a fairly common problem when
> having to deal with "teen-age clusters", so consolidated information would
> be a real help. I'm willing to start writing things up, but don't want to
> duplicate information. So:
>
> Are there any documented "operational procedures" on how to migrate
>
> - an RBD pool (with snapshots created by Openstack)
>
> - a CephFS data pool
>
> - a CephFS metadata pool
>
> to a different volume, in order to be able to utilize pool settings that
> cannot be changed on an existing pool?
>
> ---
>
> RBD pools: From what I've read, RBD snapshots are "broken" after using
> "rados cppool" to move the content of an "RBD pool" to a new pool.
>
> ---
>
> CephFS data pool: I know I can add additional pools to a CephFS instance
> ("ceph fs add_data_pool"), and have newly created files to be placed in the
> new pool ("file layouts"). But according to the docs, a small amount of
> metadata is kept in the primary data pool for all files, so I cannot remove
> the original pool.
>
> I couldn't identify how CephFS (MDS) identifies it's current data pool (or
> "default data pool" in case of multiple pools - the one named in "ceph fs
> new"), so "rados cppool"-moving the data to a new pool and then
> reconfiguring CephFS to use the new pool (while MDS are stopped, of course)
> is not yet an option? And there might be references to the pool id hiding in
> CephFS metadata, too, invalidating this approach altogether.
>
> Of course, dumping the current content of the CephFS to external storage and
> recreating the CephFS instance with new pools is a potential option, but may
> required a substantial amount of extra storage ;)
>
> ---
>
> CephFS metadata pool: I've not seen any indication of a procedure to swap
> metadata pools.
>
>
> I couldn't identify how CephFS (MDS) identifies it's current metadata pool,
> so "rados cppool"-moving the metadata to a new pool and then reconfiguring
> CephFS to use the new pool (while MDS are stopped, of course) is not yet an
> option?
>
> Of course, dumping the current content of the CephFS to external storage and
> recreating the CephFS instance with new pools is a potential option, but may
> required a substantial amount of extra storage ;)
>
> ---
>
> http://cephnotes.ksperis.com/blog/2015/04/15/ceph-pool-migration describes
> an interesting approach to migrate all pool contents by making the current
> pool a cache tier to the new pool and then migrate the "cache tier content"
> to the (new) base pool. But I'm not yet able to judge the approach and will
> have to conduct tests. Can anyone already make an educated guess if
> especially the "snapshot" problem for RBD pools will be circumvented this
> way and how CephFS will react to this approach? This "cache tier" approach,
> if feasible, would be a nice way to circumvent downtime and extra space
> requirements.
>
> Thank you for any ideas, insight and experience you can share!
>
> Regards,
> J
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Migrating to new pools (RBD, CephFS)

2017-12-18 Thread Jens-U. Mozdzen 

Hi *,

facing the problem to reduce the number of PGs for a pool, I've found  
various information and suggestions, but no "definite guide" to handle  
pool migration with Ceph 12.2.x. This seems to be a fairly common  
problem when having to deal with "teen-age clusters", so consolidated  
information would be a real help. I'm willing to start writing things  
up, but don't want to duplicate information. So:


Are there any documented "operational procedures" on how to migrate

- an RBD pool (with snapshots created by Openstack)

- a CephFS data pool

- a CephFS metadata pool

to a different volume, in order to be able to utilize pool settings  
that cannot be changed on an existing pool?


---

RBD pools: From what I've read, RBD snapshots are "broken" after using  
"rados cppool" to move the content of an "RBD pool" to a new pool.


---

CephFS data pool: I know I can add additional pools to a CephFS  
instance ("ceph fs add_data_pool"), and have newly created files to be  
placed in the new pool ("file layouts"). But according to the docs, a  
small amount of metadata is kept in the primary data pool for all  
files, so I cannot remove the original pool.


I couldn't identify how CephFS (MDS) identifies it's current data pool  
(or "default data pool" in case of multiple pools - the one named in  
"ceph fs new"), so "rados cppool"-moving the data to a new pool and  
then reconfiguring CephFS to use the new pool (while MDS are stopped,  
of course) is not yet an option? And there might be references to the  
pool id hiding in CephFS metadata, too, invalidating this approach  
altogether.


Of course, dumping the current content of the CephFS to external  
storage and recreating the CephFS instance with new pools is a  
potential option, but may required a substantial amount of extra  
storage ;)


---

CephFS metadata pool: I've not seen any indication of a procedure to  
swap metadata pools.



I couldn't identify how CephFS (MDS) identifies it's current metadata  
pool, so "rados cppool"-moving the metadata to a new pool and then  
reconfiguring CephFS to use the new pool (while MDS are stopped, of  
course) is not yet an option?


Of course, dumping the current content of the CephFS to external  
storage and recreating the CephFS instance with new pools is a  
potential option, but may required a substantial amount of extra  
storage ;)


---

http://cephnotes.ksperis.com/blog/2015/04/15/ceph-pool-migration  
describes an interesting approach to migrate all pool contents by  
making the current pool a cache tier to the new pool and then migrate  
the "cache tier content" to the (new) base pool. But I'm not yet able  
to judge the approach and will have to conduct tests. Can anyone  
already make an educated guess if especially the "snapshot" problem  
for RBD pools will be circumvented this way and how CephFS will react  
to this approach? This "cache tier" approach, if feasible, would be a  
nice way to circumvent downtime and extra space requirements.


Thank you for any ideas, insight and experience you can share!

Regards,
J

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com