Re: [ceph-users] Need help to develop CEPH EC Plugin for array type of Erasure Code

[Gregory Farnum]

On Mon, Feb 1, 2016 at 7:33 AM, Syed Hussain  wrote:
> Hi,
>
> I've been working to develop a CEPH EC plgin for array type erasure code,
> for example
> RAID-DP (i.e RDP). Later I realized that I can't continue with (k+m) format
> in CEPH as
> like normal RS code if I want to save disk IO (or Read data).  Since I need
> to ready the individual symbols from each device rather that k symbols. One
> "chunk" from OSD will contain k-symbols
> in contiguous form. The same thing will be an issue for MSR(Minimum Storage
> Regeneration) Erasure code plugin.
>
> As a solution I thought to consider k*k data chunk and k*m coding chunk and
> use LRC kind of
> layering to pass the info (in plugin) which is data chunk and coding chunk.
> However, I've doubt if the decoding API will supply me the correct info on
> lost chunks to get the info on lost erasure coding nodes.
>
> Does anybody have good solution to this issue using CRUSH map?
> Could Pl. suggest any pointer or link that explain the above issue?

I'm not really familiar with some of the terminology you're using and
I suspect that's the case for most people on the list. Perhaps you
could expand on the exact requirements for the erasure code you're
trying to implement, and the issues with doing that through the
current EC plugin interface?
-Greg
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Need help to develop CEPH EC Plugin for array type of Erasure Code

[Syed Hussain]

Thanks for the follow-up.

Lets start with normal erasure coding plugin. Here we have k data node and
m parity nodes.
For RDP or RAID-DP or MSR/MBR kind of erasure code the codeword is two
dimensional array.

FOr example, in case of RDP Erasure code the cored is kx(k+m) array. The
codeword has k rows and n columns (i.e. k + m)
The first k columns represents the systematic data and the last m columns
represents the parity data.
Here the object to be divided into kX(k+m) symbols. The RDP Erasure code
has one row parity and rests are diagonal parity.
Pl. make google image for RDP Erasure Code diagram.

I'm I making sense...

Thanks,
Syed

On Mon, Feb 1, 2016 at 9:03 PM, Syed Hussain  wrote:

> Hi,
>
> I've been working to develop a CEPH EC plgin for array type erasure code,
> for example
> RAID-DP (i.e RDP). Later I realized that I can't continue with (k+m)
> format in CEPH as
> like normal RS code if I want to save disk IO (or Read data).  Since I
> need to ready the individual symbols from each device rather that k
> symbols. One "chunk" from OSD will contain k-symbols
> in contiguous form. The same thing will be an issue for MSR(Minimum
> Storage Regeneration) Erasure code plugin.
>
> As a solution I thought to consider k*k data chunk and k*m coding chunk
> and use LRC kind of
> layering to pass the info (in plugin) which is data chunk and coding chunk.
> However, I've doubt if the decoding API will supply me the correct info on
> lost chunks to get the info on lost erasure coding nodes.
>
> Does anybody have good solution to this issue using CRUSH map?
> Could Pl. suggest any pointer or link that explain the above issue?
>
> Thanks,
> Syed Abid Hussain
>
>
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com