[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Thomas Bennett]

A pleasure. Hope it helps :)

Happy to share if you need any more information Zac.

Cheers,
Tom

On Wed, 26 Apr 2023 at 18:14, Dan van der Ster 
wrote:

> Thanks Tom, this is a very useful post!
> I've added our docs guy Zac in cc: IMHO this would be useful in a
> "Tips & Tricks" section of the docs.
>
> -- dan
>
> __
> Clyso GmbH | https://www.clyso.com
>
>
>
>
> On Wed, Apr 26, 2023 at 7:46 AM Thomas Bennett  wrote:
> >
> > I would second Joachim's suggestion - this is exactly what we're in the
> > process of doing for a client, i.e migrating from Luminous to Quincy.
> > However below would also work if you're moving to Nautilus.
> >
> > The only catch with this plan would be if you plan to reuse any hardware
> -
> > i.e the hosts running rados gateways and mons, etc. If you have enough
> > hardware to spare this is a good plan.
> >
> > My process:
> >
> >1. Stand a new Quincy cluster and tune the cluster.
> >2. Migrate user information, secrets and access keys (using
> >radosg-admin in a script).
> >3. Using a combination of rclone and parallel to push data across from
> >the old cluster to the new cluster.
> >
> >
> > Below is a bash script I used to capture all the user information on the
> > old cluster and I ran it on the new cluster to create users and keep
> their
> > secrets and keys the same.
> >
> > #
> > for i in $(radosgw-admin user list | jq -r .[]); do
> > USER_INFO=$(radosgw-admin user info --uid=$i)
> > USER_ID=$(echo $USER_INFO | jq -r '.user_id')
> > DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name')
> > EMAIL=$(echo $USER_INFO | jq '.email')
> > MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)')
> > ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key')
> > SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key')
> > echo "radosgw-admin user create --uid=$USER_ID
> > --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS
> > --access-key=$ACCESS --secret-key=$SECRET" | tee -a
> > generated.radosgw-admin-user-create.sh
> > done
> > #
> >
> > Rclone is a really powerful tool! I lazily set up a backends for each
> user,
> > by appending below to the for loop in the above script. Below script is
> not
> > pretty but it does the job:
> > #
> > echo "" >> generated.rclone.conf
> > echo [old-cluster-$USER_ID] >> generated.rclone.conf
> > echo type = s3 >> generated.rclone.conf
> > echo provider = Ceph >> generated.rclone.conf
> > echo env_auth = false >> generated.rclone.conf
> > echo access_key_id = $ACCESS >> generated.rclone.conf
> > echo secret_access_key = $SECRET >> generated.rclone.conf
> > echo endpoint = http://xx.xx.xx.xx: >> generated.rclone.conf
> > echo acl = public-read >> generated.rclone.conf
> > echo "" >> generated.rclone.conf
> > echo [new-cluster-$USER_ID] >> generated.rclone.conf
> > echo type = s3 >> generated.rclone.conf
> > echo provider = Ceph >> generated.rclone.conf
> > echo env_auth = false >> generated.rclone.conf
> > echo access_key_id = $ACCESS >> generated.rclone.conf
> > echo secret_access_key = $SECRET >> generated.rclone.conf
> > echo endpoint = http://yy.yy.yy.yy: >> generated.rclone.conf
> > echo acl = public-read >> generated.rclone.conf
> > echo "" >> generated.rclone.conf
> > #
> >
> > Copy the generated.rclone.conf to the node that is going to act as the
> > transfer node (I just used the new rados gateway node) into
> > ~/.config/rclone/rclone.conf
> >
> > Now if you run rclone lsd old-cluser-{user}: (it even tab completes!)
> > you'll get a list of all the buckets for that user.
> >
> > You could even simply rclone sync old-cluser-{user}: new-cluser-{user}:
> and
> > it should sync all buckets for a user.
> >
> > Catches:
> >
> >- Use the scripts carefully - our buckets for this one user are set
> >public-read - you might want to check each line of the script if you
> use it.
> >- Quincy bucket naming convention is stricter than Luminous. I've had
> to
> >catch some '_' and upper cases and fix them in the command line I
> generate
> >for copying each bucket.
> >- Using rclone will take a long time.Feed a script into parallel sped
> >things up for me:
> >   - # parallel -j 10 < sync-script
> >- Watch out for lifecycling! Not sure how to handle this to make sure
> >it's captured correctly.
> >
> > Cheers,
> > Tom
> >
> > On Tue, 25 Apr 2023 at 22:36, Marc  wrote:
> >
> > >
> > > Maybe he is limited by the supported OS
> > >
> > >
> > > >
> > > > I would create a new cluster with Quincy and would migrate the data
> from
> > > > the old to the new cluster bucket by bucket. Nautilus is out of
> support
> > > > and
> > > > I would recommend at least to use a ceph version that is receiving
> > > > Backports.
> > > >
> > > > huxia...@horebdata.cn  schrieb am Di., 25.
> Apr.
> > > > 2023, 18:30:
> > > >
> > > > > Dear Ceph 

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Dan van der Ster]

Thanks Tom, this is a very useful post!
I've added our docs guy Zac in cc: IMHO this would be useful in a
"Tips & Tricks" section of the docs.

-- dan

__
Clyso GmbH | https://www.clyso.com




On Wed, Apr 26, 2023 at 7:46 AM Thomas Bennett  wrote:
>
> I would second Joachim's suggestion - this is exactly what we're in the
> process of doing for a client, i.e migrating from Luminous to Quincy.
> However below would also work if you're moving to Nautilus.
>
> The only catch with this plan would be if you plan to reuse any hardware -
> i.e the hosts running rados gateways and mons, etc. If you have enough
> hardware to spare this is a good plan.
>
> My process:
>
>1. Stand a new Quincy cluster and tune the cluster.
>2. Migrate user information, secrets and access keys (using
>radosg-admin in a script).
>3. Using a combination of rclone and parallel to push data across from
>the old cluster to the new cluster.
>
>
> Below is a bash script I used to capture all the user information on the
> old cluster and I ran it on the new cluster to create users and keep their
> secrets and keys the same.
>
> #
> for i in $(radosgw-admin user list | jq -r .[]); do
> USER_INFO=$(radosgw-admin user info --uid=$i)
> USER_ID=$(echo $USER_INFO | jq -r '.user_id')
> DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name')
> EMAIL=$(echo $USER_INFO | jq '.email')
> MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)')
> ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key')
> SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key')
> echo "radosgw-admin user create --uid=$USER_ID
> --display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS
> --access-key=$ACCESS --secret-key=$SECRET" | tee -a
> generated.radosgw-admin-user-create.sh
> done
> #
>
> Rclone is a really powerful tool! I lazily set up a backends for each user,
> by appending below to the for loop in the above script. Below script is not
> pretty but it does the job:
> #
> echo "" >> generated.rclone.conf
> echo [old-cluster-$USER_ID] >> generated.rclone.conf
> echo type = s3 >> generated.rclone.conf
> echo provider = Ceph >> generated.rclone.conf
> echo env_auth = false >> generated.rclone.conf
> echo access_key_id = $ACCESS >> generated.rclone.conf
> echo secret_access_key = $SECRET >> generated.rclone.conf
> echo endpoint = http://xx.xx.xx.xx: >> generated.rclone.conf
> echo acl = public-read >> generated.rclone.conf
> echo "" >> generated.rclone.conf
> echo [new-cluster-$USER_ID] >> generated.rclone.conf
> echo type = s3 >> generated.rclone.conf
> echo provider = Ceph >> generated.rclone.conf
> echo env_auth = false >> generated.rclone.conf
> echo access_key_id = $ACCESS >> generated.rclone.conf
> echo secret_access_key = $SECRET >> generated.rclone.conf
> echo endpoint = http://yy.yy.yy.yy: >> generated.rclone.conf
> echo acl = public-read >> generated.rclone.conf
> echo "" >> generated.rclone.conf
> #
>
> Copy the generated.rclone.conf to the node that is going to act as the
> transfer node (I just used the new rados gateway node) into
> ~/.config/rclone/rclone.conf
>
> Now if you run rclone lsd old-cluser-{user}: (it even tab completes!)
> you'll get a list of all the buckets for that user.
>
> You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: and
> it should sync all buckets for a user.
>
> Catches:
>
>- Use the scripts carefully - our buckets for this one user are set
>public-read - you might want to check each line of the script if you use 
> it.
>- Quincy bucket naming convention is stricter than Luminous. I've had to
>catch some '_' and upper cases and fix them in the command line I generate
>for copying each bucket.
>- Using rclone will take a long time.Feed a script into parallel sped
>things up for me:
>   - # parallel -j 10 < sync-script
>- Watch out for lifecycling! Not sure how to handle this to make sure
>it's captured correctly.
>
> Cheers,
> Tom
>
> On Tue, 25 Apr 2023 at 22:36, Marc  wrote:
>
> >
> > Maybe he is limited by the supported OS
> >
> >
> > >
> > > I would create a new cluster with Quincy and would migrate the data from
> > > the old to the new cluster bucket by bucket. Nautilus is out of support
> > > and
> > > I would recommend at least to use a ceph version that is receiving
> > > Backports.
> > >
> > > huxia...@horebdata.cn  schrieb am Di., 25. Apr.
> > > 2023, 18:30:
> > >
> > > > Dear Ceph folks,
> > > >
> > > > I would like to listen to your advice on the following topic: We have
> > > a
> > > > 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12,
> > > and
> > > > now will add 10 new nodes. Our plan is to phase out the old 6 nodes,
> > > and
> > > > run RGW Ceph cluster with the new 10 nodes on Nautilus version。
> > > >
> > > > I can think of two ways to achieve the above 

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Thomas Bennett]

I would second Joachim's suggestion - this is exactly what we're in the
process of doing for a client, i.e migrating from Luminous to Quincy.
However below would also work if you're moving to Nautilus.

The only catch with this plan would be if you plan to reuse any hardware -
i.e the hosts running rados gateways and mons, etc. If you have enough
hardware to spare this is a good plan.

My process:

   1. Stand a new Quincy cluster and tune the cluster.
   2. Migrate user information, secrets and access keys (using
   radosg-admin in a script).
   3. Using a combination of rclone and parallel to push data across from
   the old cluster to the new cluster.


Below is a bash script I used to capture all the user information on the
old cluster and I ran it on the new cluster to create users and keep their
secrets and keys the same.

#
for i in $(radosgw-admin user list | jq -r .[]); do
USER_INFO=$(radosgw-admin user info --uid=$i)
USER_ID=$(echo $USER_INFO | jq -r '.user_id')
DISPLAY_NAME=$(echo $USER_INFO | jq '.display_name')
EMAIL=$(echo $USER_INFO | jq '.email')
MAX_BUCKETS=$(echo $USER_INFO | jq -r '(.max_buckets|tostring)')
ACCESS=$(echo $USER_INFO | jq -r '.keys[].access_key')
SECRET=$(echo $USER_INFO | jq -r '.keys[].secret_key')
echo "radosgw-admin user create --uid=$USER_ID
--display-name=$DISPLAY_NAME --email=$EMAIL --max-buckets=$MAX_BUCKETS
--access-key=$ACCESS --secret-key=$SECRET" | tee -a
generated.radosgw-admin-user-create.sh
done
#

Rclone is a really powerful tool! I lazily set up a backends for each user,
by appending below to the for loop in the above script. Below script is not
pretty but it does the job:
#
echo "" >> generated.rclone.conf
echo [old-cluster-$USER_ID] >> generated.rclone.conf
echo type = s3 >> generated.rclone.conf
echo provider = Ceph >> generated.rclone.conf
echo env_auth = false >> generated.rclone.conf
echo access_key_id = $ACCESS >> generated.rclone.conf
echo secret_access_key = $SECRET >> generated.rclone.conf
echo endpoint = http://xx.xx.xx.xx: >> generated.rclone.conf
echo acl = public-read >> generated.rclone.conf
echo "" >> generated.rclone.conf
echo [new-cluster-$USER_ID] >> generated.rclone.conf
echo type = s3 >> generated.rclone.conf
echo provider = Ceph >> generated.rclone.conf
echo env_auth = false >> generated.rclone.conf
echo access_key_id = $ACCESS >> generated.rclone.conf
echo secret_access_key = $SECRET >> generated.rclone.conf
echo endpoint = http://yy.yy.yy.yy: >> generated.rclone.conf
echo acl = public-read >> generated.rclone.conf
echo "" >> generated.rclone.conf
#

Copy the generated.rclone.conf to the node that is going to act as the
transfer node (I just used the new rados gateway node) into
~/.config/rclone/rclone.conf

Now if you run rclone lsd old-cluser-{user}: (it even tab completes!)
you'll get a list of all the buckets for that user.

You could even simply rclone sync old-cluser-{user}: new-cluser-{user}: and
it should sync all buckets for a user.

Catches:

   - Use the scripts carefully - our buckets for this one user are set
   public-read - you might want to check each line of the script if you use it.
   - Quincy bucket naming convention is stricter than Luminous. I've had to
   catch some '_' and upper cases and fix them in the command line I generate
   for copying each bucket.
   - Using rclone will take a long time.Feed a script into parallel sped
   things up for me:
  - # parallel -j 10 < sync-script
   - Watch out for lifecycling! Not sure how to handle this to make sure
   it's captured correctly.

Cheers,
Tom

On Tue, 25 Apr 2023 at 22:36, Marc  wrote:

>
> Maybe he is limited by the supported OS
>
>
> >
> > I would create a new cluster with Quincy and would migrate the data from
> > the old to the new cluster bucket by bucket. Nautilus is out of support
> > and
> > I would recommend at least to use a ceph version that is receiving
> > Backports.
> >
> > huxia...@horebdata.cn  schrieb am Di., 25. Apr.
> > 2023, 18:30:
> >
> > > Dear Ceph folks,
> > >
> > > I would like to listen to your advice on the following topic: We have
> > a
> > > 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12,
> > and
> > > now will add 10 new nodes. Our plan is to phase out the old 6 nodes,
> > and
> > > run RGW Ceph cluster with the new 10 nodes on Nautilus version。
> > >
> > > I can think of two ways to achieve the above goal. The first method
> > would
> > > be:   1) Upgrade the current 6-node cluster from Luminous 12.2.12 to
> > > Nautilus 14.2.22;  2) Expand the cluster with the 10 new nodes, and
> > then
> > > re-balance;  3) After rebalance completes, remove the 6 old nodes from
> > the
> > > cluster
> > >
> > > The second method would get rid of the procedure to upgrade the old 6-
> > node
> > > from Luminous to Nautilus, because those 6 nodes will be phased out
> > anyway,
> > > but then we have to 

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Marc]

Maybe he is limited by the supported OS


> 
> I would create a new cluster with Quincy and would migrate the data from
> the old to the new cluster bucket by bucket. Nautilus is out of support
> and
> I would recommend at least to use a ceph version that is receiving
> Backports.
> 
> huxia...@horebdata.cn  schrieb am Di., 25. Apr.
> 2023, 18:30:
> 
> > Dear Ceph folks,
> >
> > I would like to listen to your advice on the following topic: We have
> a
> > 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12,
> and
> > now will add 10 new nodes. Our plan is to phase out the old 6 nodes,
> and
> > run RGW Ceph cluster with the new 10 nodes on Nautilus version。
> >
> > I can think of two ways to achieve the above goal. The first method
> would
> > be:   1) Upgrade the current 6-node cluster from Luminous 12.2.12 to
> > Nautilus 14.2.22;  2) Expand the cluster with the 10 new nodes, and
> then
> > re-balance;  3) After rebalance completes, remove the 6 old nodes from
> the
> > cluster
> >
> > The second method would get rid of the procedure to upgrade the old 6-
> node
> > from Luminous to Nautilus, because those 6 nodes will be phased out
> anyway,
> > but then we have to deal with a hybrid cluster with 6-node on Luminous
> > 12.2.12, and 10-node on Nautilus, and after re-balancing, we can
> remove the
> > 6 old nodes from the cluster.
> >
> > Any suggestions, advice, or best practice would be highly appreciated.
> >
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Joachim Kraftmayer]

I would create a new cluster with Quincy and would migrate the data from
the old to the new cluster bucket by bucket. Nautilus is out of support and
I would recommend at least to use a ceph version that is receiving
Backports.

huxia...@horebdata.cn  schrieb am Di., 25. Apr.
2023, 18:30:

> Dear Ceph folks,
>
> I would like to listen to your advice on the following topic: We have a
> 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and
> now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and
> run RGW Ceph cluster with the new 10 nodes on Nautilus version。
>
> I can think of two ways to achieve the above goal. The first method would
> be:   1) Upgrade the current 6-node cluster from Luminous 12.2.12 to
> Nautilus 14.2.22;  2) Expand the cluster with the 10 new nodes, and then
> re-balance;  3) After rebalance completes, remove the 6 old nodes from the
> cluster
>
> The second method would get rid of the procedure to upgrade the old 6-node
> from Luminous to Nautilus, because those 6 nodes will be phased out anyway,
> but then we have to deal with a hybrid cluster with 6-node on Luminous
> 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the
> 6 old nodes from the cluster.
>
> Any suggestions, advice, or best practice would be highly appreciated.
>
> best regards,
>
>
> Samuel
>
>
>
> huxia...@horebdata.cn
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[huxia...@horebdata.cn]

Thanks a lot for the valuable input, Wesley, Josh, and Anthony.

It seems the best practice would be upgrade first, and then expand, remove old 
nodes afterwards.

best regards,

Samuel



huxia...@horebdata.cn
 
From: Wesley Dillingham
Date: 2023-04-25 19:55
To: huxia...@horebdata.cn
CC: ceph-users
Subject: Re: [ceph-users] For suggestions and best practices on expanding Ceph 
cluster and removing old nodes
Get on nautilus first and (perhaps even go to pacific) before expansion. 
Primarily for the reason that starting  in nautilus degraded data recovery will 
be prioritized over remapped data recovery. As you phase out old hardware and 
phase in new hardware you will have a very large amount of backfill happening 
and if you get into a degraded state in the middle of this backfill it will 
take a much longer time for the degraded data to become clean again. 

Additionally, you will want to follow the best practice of updating your 
cluster in order. In short monitors then managers then osds then MDS and RGW 
then other clients. More details here: 
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-luminous

You dont want to run with a mixed software version cluster longer than a well 
coordinated upgrade takes. 

Respectfully,

Wes Dillingham
w...@wesdillingham.com
LinkedIn


On Tue, Apr 25, 2023 at 12:31 PM huxia...@horebdata.cn  
wrote:
Dear Ceph folks,

I would like to listen to your advice on the following topic: We have a 6-node 
Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and now will 
add 10 new nodes. Our plan is to phase out the old 6 nodes, and run RGW Ceph 
cluster with the new 10 nodes on Nautilus version。 

I can think of two ways to achieve the above goal. The first method would be:   
1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 
14.2.22;  2) Expand the cluster with the 10 new nodes, and then re-balance;  3) 
After rebalance completes, remove the 6 old nodes from the cluster

The second method would get rid of the procedure to upgrade the old 6-node from 
Luminous to Nautilus, because those 6 nodes will be phased out anyway, but then 
we have to deal with a hybrid cluster with 6-node on Luminous 12.2.12, and 
10-node on Nautilus, and after re-balancing, we can remove the 6 old nodes from 
the cluster.

Any suggestions, advice, or best practice would be highly appreciated.

best regards,


Samuel 



huxia...@horebdata.cn
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Wesley Dillingham]

Get on nautilus first and (perhaps even go to pacific) before expansion.
Primarily for the reason that starting  in nautilus degraded data recovery
will be prioritized over remapped data recovery. As you phase out old
hardware and phase in new hardware you will have a very large amount of
backfill happening and if you get into a degraded state in the middle of
this backfill it will take a much longer time for the degraded data to
become clean again.

Additionally, you will want to follow the best practice of updating your
cluster in order. In short monitors then managers then osds then MDS and
RGW then other clients. More details here:
https://docs.ceph.com/en/latest/releases/nautilus/#upgrading-from-mimic-or-luminous

You dont want to run with a mixed software version cluster longer than a
well coordinated upgrade takes.

Respectfully,

*Wes Dillingham*
w...@wesdillingham.com
LinkedIn 


On Tue, Apr 25, 2023 at 12:31 PM huxia...@horebdata.cn <
huxia...@horebdata.cn> wrote:

> Dear Ceph folks,
>
> I would like to listen to your advice on the following topic: We have a
> 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and
> now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and
> run RGW Ceph cluster with the new 10 nodes on Nautilus version。
>
> I can think of two ways to achieve the above goal. The first method would
> be:   1) Upgrade the current 6-node cluster from Luminous 12.2.12 to
> Nautilus 14.2.22;  2) Expand the cluster with the 10 new nodes, and then
> re-balance;  3) After rebalance completes, remove the 6 old nodes from the
> cluster
>
> The second method would get rid of the procedure to upgrade the old 6-node
> from Luminous to Nautilus, because those 6 nodes will be phased out anyway,
> but then we have to deal with a hybrid cluster with 6-node on Luminous
> 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the
> 6 old nodes from the cluster.
>
> Any suggestions, advice, or best practice would be highly appreciated.
>
> best regards,
>
>
> Samuel
>
>
>
> huxia...@horebdata.cn
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: For suggestions and best practices on expanding Ceph cluster and removing old nodes

[Josh Baergen]

Hi Samuel,

While the second method would probably work fine in the happy path, if
something goes wrong I think you'll be happier having a uniform
release installed. In general, we've found the backfill experience to
be better on Nautilus than Luminous, so my vote would be for the first
method. Given that your usage is RGW, just note that the OMAP format
change that happens between Luminous and Nautilus can sometimes take a
while.

Josh

On Tue, Apr 25, 2023 at 10:31 AM huxia...@horebdata.cn
 wrote:
>
> Dear Ceph folks,
>
> I would like to listen to your advice on the following topic: We have a 
> 6-node Ceph cluster (for RGW usage only ) running on Luminous 12.2.12, and 
> now will add 10 new nodes. Our plan is to phase out the old 6 nodes, and run 
> RGW Ceph cluster with the new 10 nodes on Nautilus version。
>
> I can think of two ways to achieve the above goal. The first method would be: 
>   1) Upgrade the current 6-node cluster from Luminous 12.2.12 to Nautilus 
> 14.2.22;  2) Expand the cluster with the 10 new nodes, and then re-balance;  
> 3) After rebalance completes, remove the 6 old nodes from the cluster
>
> The second method would get rid of the procedure to upgrade the old 6-node 
> from Luminous to Nautilus, because those 6 nodes will be phased out anyway, 
> but then we have to deal with a hybrid cluster with 6-node on Luminous 
> 12.2.12, and 10-node on Nautilus, and after re-balancing, we can remove the 6 
> old nodes from the cluster.
>
> Any suggestions, advice, or best practice would be highly appreciated.
>
> best regards,
>
>
> Samuel
>
>
>
> huxia...@horebdata.cn
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io