Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-12 Thread Orit Wasserman 
On Mon, Dec 11, 2017 at 5:44 PM, Sam Wouters  wrote:
> On 11-12-17 16:23, Orit Wasserman wrote:
>> On Mon, Dec 11, 2017 at 4:58 PM, Sam Wouters  wrote:
>>> Hi Orrit,
>>>
>>>
>>> On 04-12-17 18:57, Orit Wasserman wrote:
 Hi Andreas,

 On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
  wrote:
> Hello,
> With release 12.2.2 dynamic resharding bucket index has been disabled
> when running a multisite environment
> (http://tracker.ceph.com/issues/21725). Does this mean that resharding
> of bucket indexes shouldn't be done at all, manually, while running
> multisite as there's a risk of corruption?
>
 You will need to stop the sync on the bucket before doing the
 resharding and start it again after the resharding completes.
 It will start a full sync on the bucket (it doesn't mean we copy the
 objects but we go over on all of them to check if the need to be
 synced).
 We will automate this as part of the reshard admin command in the next
 Luminous release.
>>> Does this also apply to Jewel? Stop sync and restart after resharding.
>>> (I don't know if there is any way to disable sync for a specific bucket.)
>>>
>> In Jewel we only support offline bucket resharding, you have to stop
>> both zones gateways before resharding.
>> Do:
>> Execute the resharding radosgw-admin command.
>> Run full sync on the bucket using: radosgw-admin bucket sync init on the 
>> bucket.
>> Start the gateways.
>>
>> This should work but I have not tried it ...
>> Regards,
>> Orit
> Is it necessary to really stop the gateways? We tend to block all
> traffic to the bucket being resharded with the use of ACLs in the
> haproxy in front, to avoid downtime for non related buckets.
>

You need to stop the sync process for the bucket and the most simple
way is to stop the gateways.
In Luminous we added the capability to stop the sync per bucket but
you don't have it in Jewel.

> Would a:
>
> - restart gws with sync thread disabled
> - block traffic to bucket
> - reshard
> - unblock traffic
> - bucket sync init
> - restart gws with sync enabled
>
> work as well?
>
> r,
> Sam
>
>>> r,
>>> Sam
> Also, as dynamic bucket resharding was/is the main motivator moving to
> Luminous (for me at least) is dynamic reshardning something that is
> planned to be fixed for multisite environments later in the Luminous
> life-cycle or will it be left disabled forever?
>
 We are planning to enable it in Luminous time.

 Regards,
 Orit

> Thanks!
> /andreas
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-11 Thread Sam Wouters 
On 11-12-17 16:23, Orit Wasserman wrote:
> On Mon, Dec 11, 2017 at 4:58 PM, Sam Wouters  wrote:
>> Hi Orrit,
>>
>>
>> On 04-12-17 18:57, Orit Wasserman wrote:
>>> Hi Andreas,
>>>
>>> On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
>>>  wrote:
 Hello,
 With release 12.2.2 dynamic resharding bucket index has been disabled
 when running a multisite environment
 (http://tracker.ceph.com/issues/21725). Does this mean that resharding
 of bucket indexes shouldn't be done at all, manually, while running
 multisite as there's a risk of corruption?

>>> You will need to stop the sync on the bucket before doing the
>>> resharding and start it again after the resharding completes.
>>> It will start a full sync on the bucket (it doesn't mean we copy the
>>> objects but we go over on all of them to check if the need to be
>>> synced).
>>> We will automate this as part of the reshard admin command in the next
>>> Luminous release.
>> Does this also apply to Jewel? Stop sync and restart after resharding.
>> (I don't know if there is any way to disable sync for a specific bucket.)
>>
> In Jewel we only support offline bucket resharding, you have to stop
> both zones gateways before resharding.
> Do:
> Execute the resharding radosgw-admin command.
> Run full sync on the bucket using: radosgw-admin bucket sync init on the 
> bucket.
> Start the gateways.
>
> This should work but I have not tried it ...
> Regards,
> Orit
Is it necessary to really stop the gateways? We tend to block all
traffic to the bucket being resharded with the use of ACLs in the
haproxy in front, to avoid downtime for non related buckets.

Would a:

- restart gws with sync thread disabled
- block traffic to bucket
- reshard
- unblock traffic
- bucket sync init
- restart gws with sync enabled

work as well?

r,
Sam

>> r,
>> Sam
 Also, as dynamic bucket resharding was/is the main motivator moving to
 Luminous (for me at least) is dynamic reshardning something that is
 planned to be fixed for multisite environments later in the Luminous
 life-cycle or will it be left disabled forever?

>>> We are planning to enable it in Luminous time.
>>>
>>> Regards,
>>> Orit
>>>
 Thanks!
 /andreas
 ___
 ceph-users mailing list
 ceph-users@lists.ceph.com
 http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-11 Thread Orit Wasserman 
On Mon, Dec 11, 2017 at 4:58 PM, Sam Wouters  wrote:
> Hi Orrit,
>
>
> On 04-12-17 18:57, Orit Wasserman wrote:
>> Hi Andreas,
>>
>> On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
>>  wrote:
>>> Hello,
>>> With release 12.2.2 dynamic resharding bucket index has been disabled
>>> when running a multisite environment
>>> (http://tracker.ceph.com/issues/21725). Does this mean that resharding
>>> of bucket indexes shouldn't be done at all, manually, while running
>>> multisite as there's a risk of corruption?
>>>
>> You will need to stop the sync on the bucket before doing the
>> resharding and start it again after the resharding completes.
>> It will start a full sync on the bucket (it doesn't mean we copy the
>> objects but we go over on all of them to check if the need to be
>> synced).
>> We will automate this as part of the reshard admin command in the next
>> Luminous release.
> Does this also apply to Jewel? Stop sync and restart after resharding.
> (I don't know if there is any way to disable sync for a specific bucket.)
>

In Jewel we only support offline bucket resharding, you have to stop
both zones gateways before resharding.
Do:
Execute the resharding radosgw-admin command.
Run full sync on the bucket using: radosgw-admin bucket sync init on the bucket.
Start the gateways.

This should work but I have not tried it ...
Regards,
Orit

> r,
> Sam
>>> Also, as dynamic bucket resharding was/is the main motivator moving to
>>> Luminous (for me at least) is dynamic reshardning something that is
>>> planned to be fixed for multisite environments later in the Luminous
>>> life-cycle or will it be left disabled forever?
>>>
>> We are planning to enable it in Luminous time.
>>
>> Regards,
>> Orit
>>
>>> Thanks!
>>> /andreas
>>> ___
>>> ceph-users mailing list
>>> ceph-users@lists.ceph.com
>>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-11 Thread Sam Wouters 
Hi Orrit,


On 04-12-17 18:57, Orit Wasserman wrote:
> Hi Andreas,
>
> On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
>  wrote:
>> Hello,
>> With release 12.2.2 dynamic resharding bucket index has been disabled
>> when running a multisite environment
>> (http://tracker.ceph.com/issues/21725). Does this mean that resharding
>> of bucket indexes shouldn't be done at all, manually, while running
>> multisite as there's a risk of corruption?
>>
> You will need to stop the sync on the bucket before doing the
> resharding and start it again after the resharding completes.
> It will start a full sync on the bucket (it doesn't mean we copy the
> objects but we go over on all of them to check if the need to be
> synced).
> We will automate this as part of the reshard admin command in the next
> Luminous release.
Does this also apply to Jewel? Stop sync and restart after resharding.
(I don't know if there is any way to disable sync for a specific bucket.)

r,
Sam
>> Also, as dynamic bucket resharding was/is the main motivator moving to
>> Luminous (for me at least) is dynamic reshardning something that is
>> planned to be fixed for multisite environments later in the Luminous
>> life-cycle or will it be left disabled forever?
>>
> We are planning to enable it in Luminous time.
>
> Regards,
> Orit
>
>> Thanks!
>> /andreas
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-04 Thread Andreas Calminder 
Thanks!
Is there anything in the bug tracker about the resharding issues that
I can check, just to follow progress?

Regards,
Andreas

On 4 December 2017 at 18:57, Orit Wasserman  wrote:
> Hi Andreas,
>
> On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
>  wrote:
>> Hello,
>> With release 12.2.2 dynamic resharding bucket index has been disabled
>> when running a multisite environment
>> (http://tracker.ceph.com/issues/21725). Does this mean that resharding
>> of bucket indexes shouldn't be done at all, manually, while running
>> multisite as there's a risk of corruption?
>>
>
> You will need to stop the sync on the bucket before doing the
> resharding and start it again after the resharding completes.
> It will start a full sync on the bucket (it doesn't mean we copy the
> objects but we go over on all of them to check if the need to be
> synced).
> We will automate this as part of the reshard admin command in the next
> Luminous release.
>
>> Also, as dynamic bucket resharding was/is the main motivator moving to
>> Luminous (for me at least) is dynamic reshardning something that is
>> planned to be fixed for multisite environments later in the Luminous
>> life-cycle or will it be left disabled forever?
>>
>
> We are planning to enable it in Luminous time.
>
> Regards,
> Orit
>
>> Thanks!
>> /andreas
>> ___
>> ceph-users mailing list
>> ceph-users@lists.ceph.com
>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Luminous, RGW bucket resharding

2017-12-04 Thread Orit Wasserman 
Hi Andreas,

On Mon, Dec 4, 2017 at 11:26 AM, Andreas Calminder
 wrote:
> Hello,
> With release 12.2.2 dynamic resharding bucket index has been disabled
> when running a multisite environment
> (http://tracker.ceph.com/issues/21725). Does this mean that resharding
> of bucket indexes shouldn't be done at all, manually, while running
> multisite as there's a risk of corruption?
>

You will need to stop the sync on the bucket before doing the
resharding and start it again after the resharding completes.
It will start a full sync on the bucket (it doesn't mean we copy the
objects but we go over on all of them to check if the need to be
synced).
We will automate this as part of the reshard admin command in the next
Luminous release.

> Also, as dynamic bucket resharding was/is the main motivator moving to
> Luminous (for me at least) is dynamic reshardning something that is
> planned to be fixed for multisite environments later in the Luminous
> life-cycle or will it be left disabled forever?
>

We are planning to enable it in Luminous time.

Regards,
Orit

> Thanks!
> /andreas
> ___
> ceph-users mailing list
> ceph-users@lists.ceph.com
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Luminous, RGW bucket resharding

2017-12-04 Thread Andreas Calminder 
Hello,
With release 12.2.2 dynamic resharding bucket index has been disabled
when running a multisite environment
(http://tracker.ceph.com/issues/21725). Does this mean that resharding
of bucket indexes shouldn't be done at all, manually, while running
multisite as there's a risk of corruption?

Also, as dynamic bucket resharding was/is the main motivator moving to
Luminous (for me at least) is dynamic reshardning something that is
planned to be fixed for multisite environments later in the Luminous
life-cycle or will it be left disabled forever?

Thanks!
/andreas
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com