Hi Loic, I am finally doing the benchmark tool and I found a bunch of wrong parameter checks which can make the whole thing SEGV.
All the RAID-6 codes have restrictions on the parameters but they are not correctly enforced for Liberation & Blaum-Roth codes in the CEPH wrapper class ... see text from PDF "Minimal Density RAID-6 codes are MDS codes based on binary matrices which satisfy a lower-bound on the number of non-zero entries. Unlike Cauchy coding, the bit-matrix elements do not correspond to elements in GF (2 w ). Instead, the bit-matrix itself has the proper MDS property. Minimal Density RAID-6 codes perform faster than Reed-Solomon and Cauchy Reed-Solomon codes for the same parameters. Liberation coding, Liber8tion coding, and Blaum-Roth coding are three examples of this kind of coding that are supported in jerasure. With each of these codes, m must be equal to two and k must be less than or equal to w. The value of w has restrictions based on the code: • With Liberation coding, w must be a prime number [Pla08b]. • With Blaum-Roth coding, w + 1 must be a prime number [BR99]. • With Liber8tion coding, w must equal 8 [Pla08a]. ... Do you add this fixes? For the benchmark suite it runs currently 308 different configurations for the 2 algorithm which make sense from the performance point of view and provides this output: # ----------------------------------------------------------------- # Erasure Coding Benchmark - (C) CERN 2013 - [email protected] # Ram-Size=12614856704 Allocation-Size=100000000 # ----------------------------------------------------------------- # [ -BENCH- ] [ ] technique=memcpy speed=5.408 [GB/s] latency=9.245 ms # [ -BENCH- ] [ ] technique=d=a^b^c-xor speed=4.377 [GB/s] latency=17.136 ms # [ -BENCH- ] [001/304] technique=cauchy_good:k=05:m=2:w=8:lp=0:packet=00064:size=50000000 speed=1.308 [GB/s] latency=038 [ms] size-overhead=40 [%] .. .. # [ -BENCH- ] [304/304] technique=liberation:k=24:m=2:w=29:lp=2:packet=65536:size=50000000 speed=0.083 [GB/s] latency=604 [ms] size-overhead=16 [%] # ----------------------------------------------------------------- # Erasure Code Performance Summary:: # ----------------------------------------------------------------- # RAM: 12.61 GB # Allocation-Size 0.10 GB # ----------------------------------------------------------------- # Byte Initialization: 29.35 MB/s # Memcpy: 5.41 GB/s # Triple-XOR: 4.38 GB/s # ----------------------------------------------------------------- # Fastest RAID6 2.72 GB/s liber8tion:k=06:m=2:w=8:lp=0:packet=04096:size=50000000 # Fastest Triple Failure 0.96 GB/s cauchy_good:k=06:m=3:w=8:lp=0:packet=04096:size=50000000 # Fastest Quadr. Failure 0.66 GB/s cauchy_good:k=06:m=4:w=8:lp=0:packet=04096:size=50000000 # ----------------------------------------------------------------- # ................................................................. # Top 1 RAID6 2.72 GB/s liber8tion:k=06:m=2:w=8:lp=0:packet=04096:size=50000000 # Top 2 RAID6 2.72 GB/s liber8tion:k=06:m=2:w=8:lp=0:packet=16384:size=50000000 # Top 3 RAID6 2.64 GB/s liber8tion:k=06:m=2:w=8:lp=0:packet=65536:size=50000000 # Top 4 RAID6 2.60 GB/s liberation:k=07:m=2:w=7:lp=0:packet=16384:size=50000000 # Top 5 RAID6 2.59 GB/s liberation:k=05:m=2:w=7:lp=0:packet=04096:size=50000000 # ................................................................. # Top 1 Triple 0.96 GB/s cauchy_good:k=06:m=3:w=8:lp=0:packet=04096:size=50000000 # Top 2 Triple 0.94 GB/s cauchy_good:k=06:m=3:w=8:lp=0:packet=16384:size=50000000 # Top 3 Triple 0.93 GB/s cauchy_good:k=06:m=3:w=8:lp=0:packet=65536:size=50000000 # Top 4 Triple 0.89 GB/s cauchy_good:k=07:m=3:w=8:lp=0:packet=04096:size=50000000 # Top 5 Triple 0.87 GB/s cauchy_good:k=05:m=3:w=8:lp=0:packet=04096:size=50000000 # ................................................................. # Top 1 Quadr. 0.66 GB/s cauchy_good:k=06:m=4:w=8:lp=0:packet=04096:size=50000000 # Top 2 Quadr. 0.65 GB/s cauchy_good:k=07:m=4:w=8:lp=0:packet=04096:size=50000000 # Top 3 Quadr. 0.64 GB/s cauchy_good:k=06:m=4:w=8:lp=0:packet=16384:size=50000000 # Top 4 Quadr. 0.64 GB/s cauchy_good:k=05:m=4:w=8:lp=0:packet=04096:size=50000000 # Top 5 Quadr. 0.64 GB/s cauchy_good:k=06:m=4:w=8:lp=0:packet=65536:size=50000000 # ................................................................. It takes around 30 second on my box. I will add a measurement how the XOR and the 3 top algorithms scale with the number of cores and make the object-size configurable from the command line. Anything else ? Shall I add the possiblity to test a single user specified configuration via command line arguments? Cheers Andreas. -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
