[ceph-users] Re: Recommended approach for hybrid devices

[Anthony D'Atri via ceph-users]

> I have a test Tentacle cluster that is 9 nodes each with 8x 20 TB HDDs and
> 2x 2 TB enterprise NVMes.

Ensure the firmware is up to date on each!

> My initial plan was to split the NVMes into 5x
> 300 GB partitions: 1 partition for data and 4 for db (to be paired with 4
> HDDs creating hybrid devices). My hope was that for performance, the one
> NVMe data partition could then be used for 4x replicated SSD-only
> metadata pools.

What SSD and HDD SKUs exactly please?

> The hybrid devices would be for 4+2 EC cephfs.

Depending on your workload, you might find that the WAL+DB offload doesn't get 
you much.

> 
> Previously I had tested this successfully on Squid using a series of manual
> device adds, but on Tentacle it is seeing pre-created LVM partitions as in
> use by the file system where Squid did not (ceph orch device ls ceph01).
> 
> My understanding is a manual device add approach, while possible, removes
> the benefits of orch (creating "unmanaged" devices) and should be avoided.
> But would it even be possible for orch to be fed a yaml with the NVMes
> operating as both a data and db device?

If you specify the paths explicitly, that should work.

> The obvious answer is to have a third NVMe per node operating as data for
> the 4x replicated metadata pool, but I am not sure if my budget permits
> that.

What's your file workload like?  You might consider making the OSD partition 
larger and the WAL+DB offload partitions smaller.
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io