Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-10 Thread Feng Zhang
Thanks for your reply, Janne! I was estimating the theoretical maximum recover speed for erasure coded pool. My calculation may not be accurate, hope it can be close to the correct one and the list can help here. As for hardware/software RAID, user can set the speed for rebuilding. Some hardware

Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-10 Thread Janne Johansson
Den tors 9 maj 2019 kl 17:46 skrev Feng Zhang : > Thanks, guys. > > I forgot the IOPS. So since I have 100disks, the total > IOPS=100X100=10K. For the 4+2 erasure, one disk fail, then it needs to > read 5 and write 1 objects.Then the whole 100 disks can do 10K/6 ~ 2K > rebuilding actions per secon

Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-09 Thread Feng Zhang
Thanks, guys. I forgot the IOPS. So since I have 100disks, the total IOPS=100X100=10K. For the 4+2 erasure, one disk fail, then it needs to read 5 and write 1 objects.Then the whole 100 disks can do 10K/6 ~ 2K rebuilding actions per seconds. While for the 100X6TB disks, suppose the object size is

Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-09 Thread Janne Johansson
Den tors 9 maj 2019 kl 16:17 skrev Marc Roos : > > > Fancy fast WAL/DB/Journals probably help a lot here, since they do > affect the "iops" > > you experience from your spin-drive OSDs. > > What difference can be expected if you have a 100 iops hdd and you start > using > wal/db/journals on ssd?

Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-09 Thread Marc Roos
to (estimating)? -Original Message- From: Janne Johansson [mailto:icepic...@gmail.com] Sent: donderdag 9 mei 2019 16:13 To: Feng Zhang Cc: ceph-users Subject: Re: [ceph-users] maximum rebuild speed for erasure coding pool Den tors 9 maj 2019 kl 15:46 skrev Feng Zhang : For erasure

Re: [ceph-users] maximum rebuild speed for erasure coding pool

2019-05-09 Thread Janne Johansson
Den tors 9 maj 2019 kl 15:46 skrev Feng Zhang : > > For erasure pool, suppose I have 10 nodes, each has 10 6TB drives, so > in total 100 drives. I make a 4+2 erasure pool, failure domain is > host/node. Then if one drive failed, (assume the 6TB is fully used), > what the maximum speed the recoveri

[ceph-users] maximum rebuild speed for erasure coding pool

2019-05-09 Thread Feng Zhang
Hello all, I have a naive question about the way and the maximum rebuild speed for erasure coding pool. I did some search, but could not find any formal and detailed information about this. For pool recovering, the way Ceph works(to my understanding) is: each active OSD scrubs the drive, and if i