Hi Christopher, I would like to offer my help on this blueprint. In
http://wiki.ceph.com/01Planning/02Blueprints/Dumpling/Erasure_encoding_as_a_storage_backend you wrote "At this time, Annai is more than willing to help with this, but we don't have the resources (including ceph coders) to materially contribute code. " and I can work on the coding part. Cheers On 04/19/2013 02:47 AM, Christopher LILJENSTOLPE wrote: > Supposedly, on 2013-Apr-18, at 14.26 PDT(-0700), someone claiming to be Sage > Weil scribed: > >> On Thu, 18 Apr 2013, Noah Watkins wrote: >>> On Apr 18, 2013, at 2:08 PM, Josh Durgin <[email protected]> wrote: >>> >>>> I talked to some folks interested in doing a more limited form of this >>>> yesterday. They started a blueprint [1]. One of their ideas was to have >>>> erasure coding done by a separate process (or thread perhaps). It would >>>> use erasure coding on an object and then use librados to store the >>>> rasure-encoded pieces in a separate pool, and finally leave a marker in >>>> place of the original object in the first pool. >>> >>> This sounds at a high-level similar to work out of Microsoft: >>> >>> https://www.usenix.org/system/files/conference/atc12/atc12-final181_0.pdf >>> >>> The basic idea is to replicate first, then erasure code in the background. >> >> FWIW, I think a useful (and generic) concept to add to rados would be a >> redirect symlink sort of thing that says "oh, this object is over there is >> that other pool", such that client requests will be transparently >> redirected or proxied. This will enable generic tiering type operations, >> and probably simplify/enable migration without a lot of additional >> complexity on the client side. > > More to come, but I'm starting to think of a union mount of a fuse > "re-directing" overlay. The quick idea. > > On the "hot" pool, the OSD's would write to the host FS as usual. However, > that FS is actually a light-weight fuse (at least for prototype) fs that > passes almost everything right down to the file system. As the OSD hits a > capacity HWM, a watcher (asynchronous process), starts "evicting" objects > from the OSD. It does that by using a modified ceph client that calls zfec > and uses CRUSH to place the resulting shards in the "cool" pool. Once those > are committed, it replaces the object in the "hot" OSD with a special token. > This is repeated until a LWM is reached. When the OSD gets a read request > for that object, when the fuse shim sees the token, it knows to actually do a > modified client fetch from the "cool" pool. It returns the resulting object > to the original requester and (potentially) stores the object back in the > "hot" OSD (if you want a cache-like performance), replacing the token. If > necessary, some other object may get, in turn, evicted if the HWM is again > breached. > > We would also need to modify the repair mechanism for the deep scrub in the > "cool" pool to account for the repair being a re-constitution of an invalid > shard, rather than a copy (as there is only one copy of a given shard). > > I'll get a bit more of a write-up today, hopefully, in the wiki. > > Christopher > >> >> sage > > > -- > 李柯睿 > Check my PGP key here: https://www.asgaard.org/~cdl/cdl.asc > Current vCard here: https://www.asgaard.org/~cdl/cdl.vcf > Check my calendar availability: https://tungle.me/cdl -- Loïc Dachary, Artisan Logiciel Libre
signature.asc
Description: OpenPGP digital signature
