Hi Andreas, On 17/11/2014 11:07, Andreas Joachim Peters wrote:> Hi, > > if you want to compute local parities within a plug-in library by adding rows > to the encoding matrix, you can still do that. If you can do LRC inside the > plug-in more efficienty, one should do it in the plug-in and skip the LRC > layer. I have a patch to the ISA plug-in which can do LRC like HDFS Xorbas > encoding, but i didn't manage to express that in Loic's generic LRC layer > ... I think it is hard to formulate a generic framework to define LRC as a > layer on top and doing the encoding/decoding only in the lowest layer. Same > problem for the shingled EC, which you cannot express in the simple LRC way > ...
Could you please remind us the URL of this draft ? > > I see the current LRC layering as a generic tool to use in multi-datacenter > setups honouring the locality of chunks because some of the LRC optimization > codes break locality by cross-talk between locations, they reduce the traffic > but do not restrict traffic within data centers. It certainly has its use but it would be nice to have a more specialized and better optimized version that implements most subtle algorithms ;-) Cheers > > Cheers Andreas. > ________________________________________ > From: Loic Dachary [[email protected]] > Sent: 17 November 2014 02:04 > To: Zhou, Yuan; Andreas Joachim Peters > Cc: [email protected] > Subject: Re: Question on Ceph LRC design > > Hi, > > I believe Andreas has a more elaborate answer on this topic. The current > implementation is not as good as what is described in the paper you mention, > this is correct. My incentive for chosing this path is that I was able to > understand it, mainly. It is not much more than stacking layers of erasure > coded chunks on top of each other ;-) > > Now that we have this plugin, it would be nice to have another implementation > that uses less space and possible even less network when reconstructing. > During the OpenStack summit we discussed this with Kevin Greenan and there > are promising directions. It would help a lot to have a sample code to study > so that it can be adapted to what we have currently in Ceph. Do you know of > such an implementation of LRC or other similar code designed to reduce the > network bandwidth during reconstruction ? > > Cheers > > On 17/11/2014 01:52, Zhou, Yuan wrote: >> Hi Loic/Anderas, >> >> >> >> I was trying to understand the LRC design in Ceph EC. Per my understanding, >> it seems Ceph was using a slightly different design with the Microsoft LRC: >> the local parities were calculated with the global parities included. Is >> there any special consideration on this change? >> >> I was asking because in a typical MS LRC design the global and local >> parities could be calculated at the same time actually(I mean inside the >> Erasure Code library). But with this new design, we lost this potential >> optimization. >> >> >> >> Thanks, -Yuan >> > > -- > Loïc Dachary, Artisan Logiciel Libre > -- Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP digital signature
