Re: [ceph-users] bucket indices: ssd-only or is a large fast block.db sufficient?

2018-11-20 Thread Mark Nelson
One consideration is that you may not be able to fit higher DB levels on the db partition and end up with a lot of waste (Nick Fisk recently saw this on his test cluster).  We've talked about potentially trying to pre-compute the hierarchy sizing so that we can align a level boundary to fit

Re: [ceph-users] bucket indices: ssd-only or is a large fast block.db sufficient?

2018-11-20 Thread Gregory Farnum
Looks like you’ve considered the essential points for bluestore OSDs, yep. :) My concern would just be the surprisingly-large block.db requirements for rgw workloads that have been brought up. (300+GB per OSD, I think someone saw/worked out?). -Greg On Tue, Nov 20, 2018 at 1:35 AM Dan van der

[ceph-users] bucket indices: ssd-only or is a large fast block.db sufficient?

2018-11-20 Thread Dan van der Ster
Hi ceph-users, Most of our servers have 24 hdds plus 4 ssds. Any experience how these should be configured to get the best rgw performance? We have two options: 1) All osds the same, with data on the hdd and block.db on a 40GB ssd partition 2) Two osd device types: hdd-only for the rgw