No not a standalone Wal I wanted to ask whether bdev-new-db migrated dB and Wal from hdd to ssd.
Stefan > Am 24.04.2020 um 13:01 schrieb Igor Fedotov <ifedo...@suse.de>: > > > Unless you have 3 different types of disks beyond OSD (e.g. HDD, SSD, NVMe) > standalone WAL makes no sense. > > > > On 4/24/2020 1:58 PM, Stefan Priebe - Profihost AG wrote: >> Is Wal device missing? Do I need to run bluefs-bdev-new-db and Wal? >> >> Greets, >> Stefan >> >>> Am 24.04.2020 um 11:32 schrieb Stefan Priebe - Profihost AG >>> <s.pri...@profihost.ag>: >>> >>> Hi Igor, >>> >>> there must be a difference. I purged osd.0 and recreated it. >>> >>> Now it gives: >>> ceph tell osd.0 bench >>> { >>> "bytes_written": 1073741824, >>> "blocksize": 4194304, >>> "elapsed_sec": 8.1554735639999993, >>> "bytes_per_sec": 131659040.46819863, >>> "iops": 31.389961354303033 >>> } >>> >>> What's wrong wiht adding a block.db device later? >>> >>> Stefan >>> >>> Am 23.04.20 um 20:34 schrieb Stefan Priebe - Profihost AG: >>>> Hi, >>>> if the OSDs are idle the difference is even more worse: >>>> # ceph tell osd.0 bench >>>> { >>>> "bytes_written": 1073741824, >>>> "blocksize": 4194304, >>>> "elapsed_sec": 15.396707875000001, >>>> "bytes_per_sec": 69738403.346825853, >>>> "iops": 16.626931034761871 >>>> } >>>> # ceph tell osd.38 bench >>>> { >>>> "bytes_written": 1073741824, >>>> "blocksize": 4194304, >>>> "elapsed_sec": 6.8903985170000004, >>>> "bytes_per_sec": 155831599.77624846, >>>> "iops": 37.153148597776521 >>>> } >>>> Stefan >>>> Am 23.04.20 um 14:39 schrieb Stefan Priebe - Profihost AG: >>>>> Hi, >>>>>> Am 23.04.20 um 14:06 schrieb Igor Fedotov: >>>>>> I don't recall any additional tuning to be applied to new DB volume. And >>>>>> assume the hardware is pretty the same... >>>>>> >>>>>> Do you still have any significant amount of data spilled over for these >>>>>> updated OSDs? If not I don't have any valid explanation for the >>>>>> phenomena. >>>>> >>>>> just the 64k from here: >>>>> https://tracker.ceph.com/issues/44509 >>>>> >>>>>> You might want to try "ceph osd bench" to compare OSDs under pretty the >>>>>> same load. Any difference observed >>>>> >>>>> Servers are the same HW. OSD Bench is: >>>>> # ceph tell osd.0 bench >>>>> { >>>>> "bytes_written": 1073741824, >>>>> "blocksize": 4194304, >>>>> "elapsed_sec": 16.091414781000001, >>>>> "bytes_per_sec": 66727620.822242722, >>>>> "iops": 15.909104543266945 >>>>> } >>>>> >>>>> # ceph tell osd.36 bench >>>>> { >>>>> "bytes_written": 1073741824, >>>>> "blocksize": 4194304, >>>>> "elapsed_sec": 10.023828538, >>>>> "bytes_per_sec": 107118933.6419194, >>>>> "iops": 25.539143953780986 >>>>> } >>>>> >>>>> >>>>> OSD 0 is a Toshiba MG07SCA12TA SAS 12G >>>>> OSD 36 is a Seagate ST12000NM0008-2H SATA 6G >>>>> >>>>> SSDs are all the same like the rest of the HW. But both drives should >>>>> give the same performance from their specs. The only other difference is >>>>> that OSD 36 was directly created with the block.db device (Nautilus >>>>> 14.2.7) and OSD 0 (14.2.8) does not. >>>>> >>>>> Stefan >>>>> >>>>>> >>>>>>> On 4/23/2020 8:35 AM, Stefan Priebe - Profihost AG wrote: >>>>>>> Hello, >>>>>>> >>>>>>> is there anything else needed beside running: >>>>>>> ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-${OSD} >>>>>>> bluefs-bdev-new-db --dev-target /dev/vgroup/lvdb-1 >>>>>>> >>>>>>> I did so some weeks ago and currently i'm seeing that all osds >>>>>>> originally deployed with --block-db show 10-20% I/O waits while all >>>>>>> those got converted using ceph-bluestore-tool show 80-100% I/O waits. >>>>>>> >>>>>>> Also is there some tuning available to use more of the SSD? The SSD >>>>>>> (block-db) is only saturated at 0-2%. >>>>>>> >>>>>>> Greets, >>>>>>> Stefan >>>>>>> _______________________________________________ >>>>>>> ceph-users mailing list -- ceph-users@ceph.io >>>>>>> To unsubscribe send an email to ceph-users-le...@ceph.io _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io