Hey Emmanuel,
I just wanted to drop you a line, since i haven't seen any other replies (i
figure people are probably busy). I think this is a great idea.
The only concerns i have (and i'm not sure this is something that would be a
serious problem) is that what you're suggesting runs very much contrary to
the way that NoSQL datastores function, which, possessing the capability of
storing arbitrary attribute sets don't need blob based attribute
serialization. So what i'd do is probably recommend as a roadmap phase
(since most of the discussion you've jotted down appears to be RDBMS
oriented, which is a good thing, imo), you might note what sort of
architectural changes you would need to make your gem abstract enough that
someone down the way can use the native facilities that NoSQL stores
possess.
Actually upon further thought, paper_trail's design would probably work as
well for NoSQL stores. You might look at how people deal with this on
Google AppEngine's BigTable store too. :)
What i would want to avoid, is basically taking a Resource and turning it
into an entity-mapping storage system, which are incredibly slow for large
numbers of records or fields for each ruby object.
Another mechanism you could use is to attach version numbers to properties,
so that the when the class is loaded, it knows which properties are current,
and which ones are deprecated. That way you have a semantic for knowing
which fields to create for new objects (most recent record set), and the
ruby class is also aware of what deprecated properties it *might* encounter
on older records.
Then you've got 3 explicit semantics, unversioned properties (current/always
present), versioned properties (deprecated, but still around), and deleted
properties (once all older records are migrated up, or deleted or whatever).
DataMapper errs on the side of being explicit, and i think that's usually a
good thing. You can do some fun api stuff with this too:
class Taco
include DataMapper::Resource
property :foo, Serial
property :bar, String, :version => 4 # or maybe a range? 4..8
version do
property :baz, Integer, :default => 0, :version => 8 # from version 0..8
version 4 do
property :blerp, Integer, :version => 12 # from version 4..12
property :bleep, String # only in version 4
end
end
end
Hopefully some food for thought,
-Ted
On Tue, May 3, 2011 at 5:40 PM, Emmanuel Gomez <[email protected]>wrote:
> Hi,
>
> I have need of a more flexible versioning solution that dm-is-versioned as
> it exists at v1.1.0. Also it looks like dm-is-versioned hasn't been updated
> in a while, as it contains an duplicate (partial) implementation of dirty
> tracking. Since dirty tracking is in dm-core, I'm guessing dm-is-versioned
> could probably rely on that instead of its internal implementation.
>
> In short, I'd like to overhaul dm-is-versioned. Let me rephrase that; I'm
> *going* to overhaul it for the project I'm on right now. I'd like to make my
> efforts more broadly useful by getting feedback on my plans.
>
> In the past I've used the acts_as_audited and paper_trail gems with
> ActiveRecord. I've also glanced at vestal_versions, another AR versioning
> plugin. Here are a couple of observations about those plugins' design, and
> I'll follow with some observations about dm-is-versioned.
> - They all store a serialized blob alongside other (query-able) properties.
> The serialized blob is either of
> - full attributes or
> - changes since the previous version
> - all models store versions in one table
> - Some offer controller hooks for 'magically' recording the actor (current
> user) with the version (typically Thread-local storage)
>
> For comparison, dm-is-versioned:
> - Stores one property in the version table per property in the original
> model (no blob)
> - Hence dm-is-versioned stores all attributes of each version (as opposed
> to changes from the preceding version)
> - Hooks auto_migrate/auto_upgrade calls on the original model to also
> auto_migrate/auto_upgrade the versions table
> - each versioned model has its own versions table
> - Has no web framework / controller integration
>
> I think the blob approach is handy because if your versions table(s) is
> (are) around long enough, your original model schema is likely to change,
> and all past version data may not comply with the new original model
> schema(s). The downside is that a serialized blob is opaque for querying
> (typically). This can be mitigated by including in the Version whatever
> properties you need to query it by, but you may end up duplicating data that
> is also in the blob. As far as I can see, the blob approach is somewhat
> inefficient at worst, but the inefficiency provides durability against
> schema changes. Of course,
>
> Basically, I like the core of paper_trail's design: a version consists of
> metadata about its creation, and a blob of the Resource's attributes at the
> time. I also think that design makes a clear relationship between Versions
> of Resources and the Resources themselves; the Version contains metadata
> about the change (what changed, when, by whom... whatever you want to store
> now and query later), and the full state (attributes) of the Resource.
>
> Contrast with dm-is-versioned, where instances of Version try to act very
> much like an instance of the Resource they are a Version of. In paper_trail,
> Version has-a versioned Resource. in dm-is-versioned, Version is-a versioned
> Resource.
>
> My current thinking is that a Version should be (at minimum):
> - a foreign key to the versioned model
> - a creation timestamp
> - a serialized blob of full attributes
> - ParanoidBoolean/ParanoidDateTime for the versioned Resource to preserve
> foreign-key integrity with the versions table
> - provide an easy interface to 'reconstitute' a versioned Resource from a
> Version
> - paper_trail calls this Version#reify ('reify' is basically a fancy
> synonym for 'concretize')
> - the 'reconstituted' instance would have a PersistedState of either
> Immutable or Dirty (configured per versioned Resource)
> - Immutable would ensures that past versions are read-only
> - Dirty would let you overwrite the current 'live' attributes with the
> attributes of this version
> - each versioned model having its own versions table (for foreign-key
> integrity)
>
> OK, this has turned into a manifesto. Hopefully someone will actually read
> this far and give some feedback :).
>
> Thanks for any and all feedback,
> Emmanuel
>
> --
> You received this message because you are subscribed to the Google Groups
> "DataMapper" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/datamapper?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"DataMapper" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/datamapper?hl=en.
