Re: [ceph-users] How to get Active set of OSD Map in serial order of osd index
Fundamentally, I wanted to know what chunks are allocated in which OSDs. This way I can preserve the array structure required for my Erasure Code. If all the chunks are placed in randomly ordered OSDs (like in Jerasure or ISA) then I loss that array structure required in the Encoding/Decoding algorithm of my Plugin. I'm trying to develop an Erasure Code plugin for RDP (or RAID-DP) kind of code. Thanks, Syed On Wed, Jul 27, 2016 at 4:12 AM, Samuel Just <sj...@redhat.com> wrote: > Why do you want them in serial increasing order? > -Sam > > On Tue, Jul 26, 2016 at 2:43 PM, Samuel Just <sj...@redhat.com> wrote: > >> How would such a code work if there were more than 24 osds? >> -Sam >> >> On Tue, Jul 26, 2016 at 2:37 PM, Syed Hussain <syed...@gmail.com> wrote: >> >>> Hi, >>> >>> I'm working to develop an Erasure Code plugin (variation of ISA) that >>> have typical requirement that the active set of the Erasure Coded pool in >>> serial order. >>> For example, >>> >>> >>> >ceph osd erasure-code-profile set reed_k16m8_isa k=16 m=8 plugin=isa >>> technique=reed_sol_van ruleset-failure-domain=osd >>> >ceph osd pool create reed_k16m8_isa_pool 128 128 erasure reed_k16m8_isa >>> >echo "ABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHI" | rados >>> --pool reed_k16m8_isa_pool put myobj16_8 - >>> >ceph osd map reed_k16m8_isa_pool myobj16_8 >>> osdmap e86 pool 'reed_k16m8_isa_pool' (1) object 'myobj16_8' -> pg >>> 1.cf6ec86f (1.6f) -> up >>> ([4,23,22,10,9,11,15,6,19,1,7,8,17,21,16,14,18,12,13,20,3,5,0,2], p4) >>> acting ([4,23,22,10,9,11,15,6,19,1,7,8,17,21,16,14,18,12,13,20,3,5,0,2], p4) >>> >>> >>> >>> That means the chunks 0, 1, 2, ...23 of the erasure coding are saved int >>> osd 4, 23, 22, 10, ...2 respectively as per the order given in the active >>> set. >>> >>> Now my question is how I'll be able to get the PG map for object >>> myobj16_8 having active set as: [0, 1, 2, ...23] so that the i-th chunk of >>> the Erasure Coded object saves into >>> i-th osd. >>> >>> Is there any option available in "ceph osd pool create" to do it? >>> Or there may be other way available to accomplish this case. >>> >>> Appreciate your suggestions.. >>> >>> Thanks, >>> Syed Hussain >>> NetWorld >>> >>> ___ >>> ceph-users mailing list >>> ceph-users@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >>> >>> >> > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] How to get Active set of OSD Map in serial order of osd index
Hi, I'm working to develop an Erasure Code plugin (variation of ISA) that have typical requirement that the active set of the Erasure Coded pool in serial order. For example, >ceph osd erasure-code-profile set reed_k16m8_isa k=16 m=8 plugin=isa technique=reed_sol_van ruleset-failure-domain=osd >ceph osd pool create reed_k16m8_isa_pool 128 128 erasure reed_k16m8_isa >echo "ABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHIABCDEFGHI" | rados --pool reed_k16m8_isa_pool put myobj16_8 - >ceph osd map reed_k16m8_isa_pool myobj16_8 osdmap e86 pool 'reed_k16m8_isa_pool' (1) object 'myobj16_8' -> pg 1.cf6ec86f (1.6f) -> up ([4,23,22,10,9,11,15,6,19,1,7,8,17,21,16,14,18,12,13,20,3,5,0,2], p4) acting ([4,23,22,10,9,11,15,6,19,1,7,8,17,21,16,14,18,12,13,20,3,5,0,2], p4) That means the chunks 0, 1, 2, ...23 of the erasure coding are saved int osd 4, 23, 22, 10, ...2 respectively as per the order given in the active set. Now my question is how I'll be able to get the PG map for object myobj16_8 having active set as: [0, 1, 2, ...23] so that the i-th chunk of the Erasure Coded object saves into i-th osd. Is there any option available in "ceph osd pool create" to do it? Or there may be other way available to accomplish this case. Appreciate your suggestions.. Thanks, Syed Hussain NetWorld ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] ISA and LRC profile doesn't load in freshly created CEPH cluster
Hi, The erasure coding libraries of both plugins ISA and LRC created in ~ceph-10.0.0/src/.libs. However, the command for creating pool is failing in CEPH cluster (with few OSDs, Montior,..) For example, = $ceph osd erasure-code-profile set LRCprofile rulesetfailure-domain=osd k=4 m=2 l=3 plugin=lrc directory=.libs --force *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** $~/ceph-10.0.0/src$ ceph osd pool create LRCpool 128 128 erasure LRCprofile *** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH *** Error EIO: load dlopen(.libs/libec_lrc.so): .libs/libec_lrc.so: cannot open shared object file: No such file or directoryfailed to load plugin using profile LRCprofile = Same error is occurring for plugin=isa There is latest version of yasm is installed and libec_isa.so is created as well in ~/src/.libs I guess I'm missing some early ceph compilation parameter. Could you Pl. point out how this issue can be solved. Thanks, Syed Abid Hussain NetApp ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Need help on benchmarking new erasure coding
Hi, I've been developing a new array type of erasure code. I'll be glad if you can send me few pointers for the two following items: (1) Required CRUSH map for Array code, e.g. RAID-DP or MSR erasure code. It is different than normal RS(n, k) or LRC code. For example, for RAID-DP or RDP (n, k) erasure code, the codeword has k*(k+m) blocks. Question is how to place these blocks in n device mapped as OSDs? Could you Pl. suggests... (2) How to measure the recovery time of a failed OSD in cluster. Is there any script available that will perform it and generate the final result? Thanks, Syed Abid Hussain NetApp India ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] Need help to develop CEPH EC Plugin for array type of Erasure Code
Thanks for the follow-up. Lets start with normal erasure coding plugin. Here we have k data node and m parity nodes. For RDP or RAID-DP or MSR/MBR kind of erasure code the codeword is two dimensional array. FOr example, in case of RDP Erasure code the cored is kx(k+m) array. The codeword has k rows and n columns (i.e. k + m) The first k columns represents the systematic data and the last m columns represents the parity data. Here the object to be divided into kX(k+m) symbols. The RDP Erasure code has one row parity and rests are diagonal parity. Pl. make google image for RDP Erasure Code diagram. I'm I making sense... Thanks, Syed On Mon, Feb 1, 2016 at 9:03 PM, Syed Hussain <syed...@gmail.com> wrote: > Hi, > > I've been working to develop a CEPH EC plgin for array type erasure code, > for example > RAID-DP (i.e RDP). Later I realized that I can't continue with (k+m) > format in CEPH as > like normal RS code if I want to save disk IO (or Read data). Since I > need to ready the individual symbols from each device rather that k > symbols. One "chunk" from OSD will contain k-symbols > in contiguous form. The same thing will be an issue for MSR(Minimum > Storage Regeneration) Erasure code plugin. > > As a solution I thought to consider k*k data chunk and k*m coding chunk > and use LRC kind of > layering to pass the info (in plugin) which is data chunk and coding chunk. > However, I've doubt if the decoding API will supply me the correct info on > lost chunks to get the info on lost erasure coding nodes. > > Does anybody have good solution to this issue using CRUSH map? > Could Pl. suggest any pointer or link that explain the above issue? > > Thanks, > Syed Abid Hussain > > ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] Need help to develop CEPH EC Plugin for array type of Erasure Code
Hi, I've been working to develop a CEPH EC plgin for array type erasure code, for example RAID-DP (i.e RDP). Later I realized that I can't continue with (k+m) format in CEPH as like normal RS code if I want to save disk IO (or Read data). Since I need to ready the individual symbols from each device rather that k symbols. One "chunk" from OSD will contain k-symbols in contiguous form. The same thing will be an issue for MSR(Minimum Storage Regeneration) Erasure code plugin. As a solution I thought to consider k*k data chunk and k*m coding chunk and use LRC kind of layering to pass the info (in plugin) which is data chunk and coding chunk. However, I've doubt if the decoding API will supply me the correct info on lost chunks to get the info on lost erasure coding nodes. Does anybody have good solution to this issue using CRUSH map? Could Pl. suggest any pointer or link that explain the above issue? Thanks, Syed Abid Hussain ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
