On Tue, 2008-03-04 at 17:18 -0500, Ian Eslick wrote: > My fellow Elephants, > > I've recently added support for a full schema evolution infrastructure > on my local development branch. Every persistent class now has a > schema object associated with it and each store that has one or more > instances of that class has a corresponding database-specific schema > that includes, among other things, a unique schema id for that class > version and that store. The schema in the database is sufficient to > reproduce a basic defclass form (slot names and types, not accessors, > initargs, initforms).
Congratulations! Well done! > > The prior version of class indexing had a sophisticated mechanism for > synchronizing between the in-memory class definition and the in-store > index list. This doesn't seem to be used, and is too complicated to > be useful so I have nixed it. For the new schema-based notion of > synchronization, I have made some simplifying assumptions: > > MASTER SCHEMA: > The in-memory class definition is always the master schema for all > open stores that contain instances > of that class. > > A class redefinition or connecting to a store with a stale schema may > mean that the master and the store schemas are now different. This > means we need to upgrade the store schema and potentially store > instances to the new master schema. > > IN-PLACE EVOLUTION: > If only the indexed slots differ, then we simply add/delete > indices to accommodate. > This means that we have to keep track of the hierarchy so we can > remove subclasses, > or merge a set of subclass schemas if we move the index to a base > class. In-place > is fine because no data becomes irrecoverable and all changes are > at the class level. > > FULL SCHEMA EVOLUTION: > We have add/deleted slots or changed the type of a slot > (persistent->transient) > - We compute a diff function that adds/deletes the slot storage > from the store > - Any in-memory instances are upgraded > - Store instances are upgraded: > 1) A scan function is provided to upgrade all instances in the > store and delete the old schema versions > 2) Instances of prior schema versions still in the store can be > lazily upgraded on load > > SOME DESIGN CHOICES: > > - Do we delete data (dropped slots, indices) by default during a > schema evolution, or do we keep it around just in case? Personally, I think a new slot can be added safely, but to drop one requires a restart. That is, upon reattaching to the store or executing defclass so that a slot is invalidated, the user is presented with a choice of aborting or dropping the slot. I think type changes must be handled in the same way, although possible a type change could be done by unbinding each slot value and invoking initform, if there is one; but this should again be a conscious user decision. > > - What kind of warning/error conditions do we want to provide and when? > - When we load a new class def, connect to a store, and need to > make schema changes on the store? > - Class redefinition with one or more open stores > - etc... Adding is silent, because no information is loss. If there is a danger of losing information, the user must either offer a restart or have explicitly configured that away somehow. > > - How do we enable users to specify an upgrade function to move from > schema to schema; do we provide a way to specify a schema version and > an upgrade function and allow non-specified versions to just upgrade > automatically? The ultimate case is chaining. The user provides an upgrade function from version N to version N+1. These must all exist from N (where are now) to K (where we wish to be) in order to start upgrading. However, rules for defaulting can mean that one rarely has to specify all of this. This may seem over-blown, and can be conisidered an enhancement; but it makes sense if one imagines schema evolution as software development released to a third party. For example, I charge you $10K dollars and give you a schema that backs a Content Management System. While you fill up your schema with content, I create 30 schema improvements. Finally, my new system passes QA, and I offer you the upgrade of your Content Management System -- with the function that evolves you through 30 evolutions. > > - What happens when two lisp images are attached to the same store and > one updates its class definition? (Maybe you just get what you > deserve?) You get what you deserve. In Elephant, the in-memory store is the system of record. > > - What happens if a store has a schema for a class for which there is > no in-memory class object? You are alerted that the class is going to be deleted unless you specify otherwise. > > - Do we want the database schema to store initforms, initargs, and > accessor/reader/writer names? Not doing so, or doing so, is a matter of efficiency; the important thing is the consistency of the data after you detach from a store and reattach (or, alternatively, when there is some database failure of some kind.) > > - Given class schemas we could probably add persistent class slots... > > Thanks, > Ian > > > _______________________________________________ > elephant-devel site list > elephant-devel@common-lisp.net > http://common-lisp.net/mailman/listinfo/elephant-devel _______________________________________________ elephant-devel site list elephant-devel@common-lisp.net http://common-lisp.net/mailman/listinfo/elephant-devel
