Re: [ceph-users] Rados bucket issues, default.rgw.buckets.index growing every day

2018-04-05 Thread Mark Schouten 
On Wed, 2018-04-04 at 09:38 +0200, Mark Schouten wrote:
> I have some issues with my bucket index. As you can see in the
> attachment, everyday around 16:30, the amount of objects in the
> default.rgw.buckets.index increases. This happens since upgrading
> from
> 12.2.2 to 12.2.4.

I disabled dynamic resharding, which seems to have stopped the
increasing number of objects..

-- 
Kerio Operator in de Cloud? https://www.kerioindecloud.nl/
Mark Schouten  | Tuxis Internet Engineering
KvK: 61527076  | http://www.tuxis.nl/
T: 0318 200208 | i...@tuxis.nl

smime.p7s
Description: S/MIME cryptographic signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Rados bucket issues, default.rgw.buckets.index growing every day

2018-04-04 Thread Mark Schouten 
On Wed, 2018-04-04 at 09:38 +0200, Mark Schouten wrote:
> I have some issues with my bucket index. As you can see in the
> attachment, everyday around 16:30, the amount of objects in the
> default.rgw.buckets.index increases. This happens since upgrading
> from
> 12.2.2 to 12.2.4.

It seems there is still a resharding action running. Which I cannot
cancel:

root@osdnode03:~# radosgw-admin reshard cancel --
uid='DB0220$elasticsearch' --tenant=DB0220 --bucket=backups 
Error in getting bucket backups: (2) No such file or directory
2018-04-04 10:07:03.905049 7f105ee0fcc0 -1 ERROR: failed to get entry
from reshard log, oid=reshard.10 tenant= bucket=backups

I think tenant is empty because of https://github.com/oritwas/ceph/blob
/0a2142e83b58fa8e238bcb748d1cb97bdba674c5/src/rgw/rgw_admin.cc#L5755

What do you think?

-- 
Kerio Operator in de Cloud? https://www.kerioindecloud.nl/
Mark Schouten  | Tuxis Internet Engineering
KvK: 61527076  | http://www.tuxis.nl/
T: 0318 200208 | i...@tuxis.nl

smime.p7s
Description: S/MIME cryptographic signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[ceph-users] Rados bucket issues, default.rgw.buckets.index growing every day

2018-04-04 Thread Mark Schouten 
Hi,

I have some issues with my bucket index. As you can see in the
attachment, everyday around 16:30, the amount of objects in the
default.rgw.buckets.index increases. This happens since upgrading from
12.2.2 to 12.2.4.

There is no real activity at that moment, but the logs do show the
following:

rgw.log.1.gz:2018-04-03 16:28:24.594527 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.2.gz:2018-04-02 16:28:19.100159 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.3.gz:2018-04-01 16:28:09.051291 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.4.gz:2018-03-31 16:27:59.205853 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.5.gz:2018-03-30 16:27:42.812917 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.6.gz:2018-03-29 16:26:51.298205 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10
rgw.log.7.gz:2018-03-28 06:25:24.250243 7f646026a700  0
check_bucket_shards: resharding needed: stats.num_objects=134612 shard
max_objects=10
rgw.log.7.gz:2018-03-28 16:26:45.095042 7fad2e13d700  0
check_bucket_shards: resharding needed:
stats.num_objects=18446744073709551539 shard max_objects=10


I have dynamic bucketsharding enabled, and there is a reshard entry,
but because I use tenant and rados had a bug where it didn't add tenant
to the reshard-log, that entry is invalid. The bucket it mentions
doesn't even exist anymore.

So. How do I find out who is creating all these entries in the index,
and why. Secondly, how do I remove the reshardingentry for a bucket
that does not exist? I have a strong feeling that the two questions are
very much related, because of https://tracker.ceph.com/issues/22046

Thanks,

Mark

-- 
Kerio Operator in de Cloud? https://www.kerioindecloud.nl/
Mark Schouten  | Tuxis Internet Engineering
KvK: 61527076  | http://www.tuxis.nl/
T: 0318 200208 | i...@tuxis.nl

smime.p7s
Description: S/MIME cryptographic signature
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com