Rickard Öberg wrote:
Hey,
Yes, versioning again. I have thought some more about versioning and
am coming back to that while the previous scheme was way too
complicated, and that it's an implementation detail, Qi4j could still
help somewhat to make it easier.
To do data migration in key-value stores it would be useful if each
stored entity had a version number of the entire application, rather
than the version of the entity-type that we played with in the SPI
some time ago. This way it becomes easier to specify migration rules
and ensure that data is in a consistent format. Importing old backups
would also be possible, as it could be upgraded on the fly.
Here's what I suggest: in Application we add a method version(), with
a corresponding method in ApplicationAssembly to set it. What to set
it to is up to the user. It could be on the "1.2.3" format, or it
could be revision nr (e.g. "345"). Just something that makes sense for
the application. Then, the default EntityStore implementations would
store this version in the data. When state is read the stored version
can be compared with the current version. If they are the same, all is
ok. If they are not the same, then a Migrator service should be invoked.
The Migrator service, and how to configure it, is not defined. Well,
the Migrator interface would be defined in the SPI, but there could be
many implementations, with different strategies. One would be to
declare rules using some Fluent API, like so:
migrator.
onVersion("1.2.1").
forEntities("SomeEntity").
renameProperty("oldName","newName").
addProperty("name","someDefaultValue").
removeAssociation("someName").
migrate(new CustomEntityMigrator());
and so on. This should allow pretty powerful possibilities for doing
both standard and custom migrations. The migration could either be
done lazily on read, or eagerly on startup. This could be specified as
well in such a fluent API:
migrator.onVersion("1.3.0").eagerMigrate();
All of this would be done in the assembly phase, btw.
Another implementation could take a JSON-export of the database and
serialize it to XML and use XSLT for the rules. And yet another one
could post it to some REST-resource that performs the transformation.
And so on.
But the point is that if we just add the version property in the
Application API, and a Migrator service that EntityStores can call,
then I think that would be enough.
I have quite urgent need for it, so unless this is an obviously bad
idea I would like to go ahead and add it.
What do you think?
/Rickard
_______________________________________________
qi4j-dev mailing list
[email protected]
http://lists.ops4j.org/mailman/listinfo/qi4j-dev
Hi Rickard,
as I understand your sample for the migrator API,
the onVersion(String) method takes the version of the entities,
which are to be migrated to the version defined by
ApplicationAssembly.version().
So if we have a bunch of old version lying around, we (may) have
migration rules for all this old versions to the actual
application version,e.g.
migrator.
onVersion("1.2.1").
...
migrator.
onVersion("1.3.0").
...
if we e.g. assume that the current version history is
1.2.1 -> 1.3.0 -> 1.4.0
where 1.4.0 is the current application version and '->' means 'has
successor'.
The problem with the API I see is, that the migration rules always
describe the *full* migration from each of the predecessors of
the current application version to it (or did I get something wrong?).
I.e. if we later have
a 1.5.0 as sucessor of 1.4.0, and the 'diff' between 1.4.0 and 1.5.0 is
something like addProperty("name","someDefaultValue"), we must not only
introduce a
migrator.
onVersion("1.4.0").
...
addProperty("name","someDefaultValue")
rule, but we also must touch the older rules and add the same statement.
Would'nt it be better to specify the migration rules as diffs (or better
patches) from one version to its direct successor, plus the version
history,
and let the migrator service determine which patches to apply in the
correct order?
Cheers,
Georg
_______________________________________________
qi4j-dev mailing list
[email protected]
http://lists.ops4j.org/mailman/listinfo/qi4j-dev
