Hi, Today Sam pointed out that the API for LRC ( Xorbas Hadoop Project Page, Locally Repairable Codes (LRC) http://smahesh.com/HadoopUSC/ for instance ) would need to be different from the one initialy proposed:
context(k, m, reed-solomon|...) => context* c
encode(context* c, void* data) => void* chunks[k+m]
decode(context* c, void* chunk[k+m], int* indices_of_erased_chunks) =>
void* data // erased chunks are
not used
repair(context* c, void* chunk[k+m], int* indices_of_erased_chunks) =>
void* chunks[k+m] // erased
chunks are rebuilt
The decode function must allow for partial read:
decode(context* c, int offset, int length, void* chunk[k+m], int*
indices_of_erased_chunks, int* missing_chunks) => void* data
If there are not enough chunks to recover the desired data range [offset,
offset+length) the function returns NULL and sets missing_chunks to the list of
chunks that must be retrieved in order to be able to read the desired data.
If decode is called to read just 1 chunk and it is missing, reed-solomon would
return on error and ask for all other chunks to repair. If the underlying
library implements LRC, it would ask for a subset of the chunks.
An implementation allowing only full reads and using jerasure ( which does not
do LRC ) requires that offset is always zero, length is the size of the object
and returns a copy of indices_of_erased_chunks if there are not enough chunks
to rebuild the missing ones.
Comments are welcome :-)
--
Loïc Dachary, Artisan Logiciel Libre
All that is necessary for the triumph of evil is that good people do nothing.
signature.asc
Description: OpenPGP digital signature
