What I'm talking about is not specific to the Avro store. The issue is that state information can be lost during the mutation process. For example, one record has another record as a field. In this regard the sub-record represents a map. But deletion state in a record is not tracked; to have enough information to make sure you can go back and delete the kvs in the original store , you need to know what the original value was (depending on how the store does mappings) or do a range delete. Maps also do not retain enough information to be expressive in this regard; they maintain deleted state but do not describe in a granular fashion the original state of the object.
My current thinking is to strictly define four states for fields: clean, which means no mutation is pending for a record; dirty, which means a write is pending on a record; deleted, which means that a delete mutation is pending; and overwritten, which is equivalent to dirty and delete. Fields will be strictly separated into two categories: mutable (maps, arrays, and records) and immutable (bytes, strings, and other primitives). All non-immutable fields should have the original state of any mutated fields stored either via a tombstone object or dirty bits. Tombstone objects will be used to describe the original state of a mutable object that needs to be deleted, and dirty bits will be used to signal that the current state of the object is not yet persistent. Sent from my smartphone. Please excuse any typos or shorthand. On Jul 29, 2012 1:49 PM, "Lewis John Mcgibbney" <[email protected]> wrote: > Hi Ed, > > Yeah I actually noticed that deletes are not available/supported in > Avro store in trunk and in the your 84 patch. As I'm more or less > coming into the Avro stuff blind... does Avro do deletes or is it just > that we don't yet support in Gora? > > Best > Lewis > > On Sun, Jul 29, 2012 at 6:17 PM, Ed Kohlwey <[email protected]> wrote: > > I've found the apparent semantics of deletes to be pretty inconsistent > > through my work on the Avro port. I don't think enough state information > is > > actually stored to implement it reliably. I'm currently working on adding > > this on top of my Gora 84 work. > > > > Sent from my smartphone. Please excuse any typos or shorthand. > > On Jul 29, 2012 11:27 AM, "Lewis John Mcgibbney" < > [email protected]> > > wrote: > > > >> Hi, > >> > >> What kind of conversation needs to be kicked off here? > >> Currently as it stands deletes (and some other operations) in Gora > >> seem to be shrouded in mystery... :0| > >> > >> Deletes seem to be implemented in gora-accumulo fine, maybe Keith can > >> confirm? Also some of the semantics about what Accumulo expects > >> deletes to be like and whether or not it is working OK for your use > >> case? > >> Ferdy provided important input into GORA-155 stating that there needs > >> to be more clarity w.r.t semantics for versions of operations on a > >> general level before we begin to implement functionality willy nilly > >> at datastore level. > >> > >> Through Hector we can do many alternative delete operations for > >> Cassandra and this is great but I think it is important for us to > >> establish some general rules about what we wish the Gora API to > >> offer/achieve. > >> > >> Any comments? > >> Thanks > >> Lewis > >> > >> -- > >> Lewis > >> > > > > -- > Lewis >
