Is Wal device missing? Do I need to run bluefs-bdev-new-db and Wal?
Greets,
Stefan
> Am 24.04.2020 um 11:32 schrieb Stefan Priebe - Profihost AG
> <s.pri...@profihost.ag>:
>
> Hi Igor,
>
> there must be a difference. I purged osd.0 and recreated it.
>
> Now it gives:
> ceph tell osd.0 bench
> {
> "bytes_written": 1073741824,
> "blocksize": 4194304,
> "elapsed_sec": 8.1554735639999993,
> "bytes_per_sec": 131659040.46819863,
> "iops": 31.389961354303033
> }
>
> What's wrong wiht adding a block.db device later?
>
> Stefan
>
>> Am 23.04.20 um 20:34 schrieb Stefan Priebe - Profihost AG:
>> Hi,
>> if the OSDs are idle the difference is even more worse:
>> # ceph tell osd.0 bench
>> {
>> "bytes_written": 1073741824,
>> "blocksize": 4194304,
>> "elapsed_sec": 15.396707875000001,
>> "bytes_per_sec": 69738403.346825853,
>> "iops": 16.626931034761871
>> }
>> # ceph tell osd.38 bench
>> {
>> "bytes_written": 1073741824,
>> "blocksize": 4194304,
>> "elapsed_sec": 6.8903985170000004,
>> "bytes_per_sec": 155831599.77624846,
>> "iops": 37.153148597776521
>> }
>> Stefan
>>> Am 23.04.20 um 14:39 schrieb Stefan Priebe - Profihost AG:
>>> Hi,
>>> Am 23.04.20 um 14:06 schrieb Igor Fedotov:
>>>> I don't recall any additional tuning to be applied to new DB volume. And
>>>> assume the hardware is pretty the same...
>>>>
>>>> Do you still have any significant amount of data spilled over for these
>>>> updated OSDs? If not I don't have any valid explanation for the phenomena.
>>>
>>> just the 64k from here:
>>> https://tracker.ceph.com/issues/44509
>>>
>>>> You might want to try "ceph osd bench" to compare OSDs under pretty the
>>>> same load. Any difference observed
>>>
>>> Servers are the same HW. OSD Bench is:
>>> # ceph tell osd.0 bench
>>> {
>>> "bytes_written": 1073741824,
>>> "blocksize": 4194304,
>>> "elapsed_sec": 16.091414781000001,
>>> "bytes_per_sec": 66727620.822242722,
>>> "iops": 15.909104543266945
>>> }
>>>
>>> # ceph tell osd.36 bench
>>> {
>>> "bytes_written": 1073741824,
>>> "blocksize": 4194304,
>>> "elapsed_sec": 10.023828538,
>>> "bytes_per_sec": 107118933.6419194,
>>> "iops": 25.539143953780986
>>> }
>>>
>>>
>>> OSD 0 is a Toshiba MG07SCA12TA SAS 12G
>>> OSD 36 is a Seagate ST12000NM0008-2H SATA 6G
>>>
>>> SSDs are all the same like the rest of the HW. But both drives should give
>>> the same performance from their specs. The only other difference is that
>>> OSD 36 was directly created with the block.db device (Nautilus 14.2.7) and
>>> OSD 0 (14.2.8) does not.
>>>
>>> Stefan
>>>
>>>>
>>>> On 4/23/2020 8:35 AM, Stefan Priebe - Profihost AG wrote:
>>>>> Hello,
>>>>>
>>>>> is there anything else needed beside running:
>>>>> ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-${OSD}
>>>>> bluefs-bdev-new-db --dev-target /dev/vgroup/lvdb-1
>>>>>
>>>>> I did so some weeks ago and currently i'm seeing that all osds originally
>>>>> deployed with --block-db show 10-20% I/O waits while all those got
>>>>> converted using ceph-bluestore-tool show 80-100% I/O waits.
>>>>>
>>>>> Also is there some tuning available to use more of the SSD? The SSD
>>>>> (block-db) is only saturated at 0-2%.
>>>>>
>>>>> Greets,
>>>>> Stefan
>>>>> _______________________________________________
>>>>> ceph-users mailing list -- ceph-users@ceph.io
>>>>> To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
