Hi Maurits,
Maurits van Rees wrote:
Martin Aspeli, on 2007-05-11:
Maurits van Rees wrote:
Now you've set my mind racing... :) I've got a revised counter-proposal
below that I'm starting to quite like.... very interested in your thoughts!
I am glad to keep your brain busy. ;-)
I think the name SchemaManager is a bit misleading, though. Is it
managing an Archetypes schema? :)
I am not tied to the name 'SchemaManager'. Something like
'UpgradeManager' might be more to the point. What I am thinking about
when pondering an evolution step may be as simple as the addition of a
column or index to the portal_catalog.
As an example, when seen in the context of Plone, the upgrade from
beta 2 to beta 3 might mean going from generation 42 to 47.
That's what I'd like to avoid. A "generation" as an abstract number is
likely to be hard to manage, and a "this number must be the same as the
name of an extension profile" may be awkard and typo-prone.
In Plone, we let each upgrade step be between versions, but try to
ensure that you can re-run an upgrade step several times if you need to,
e.g. because you had used it before that version was actually released
and the code in it was finalised.
There might be made a case for one schema manager that handles
generations 0 through 12 for a product and a next for generations 13
through 19. But I cannot think of a good use for that really.
The other problem here is that users don't really think of
"generations", they think of versions, which are sequential things.
Generations are also sequential things. I am really thinking of 1, 2,
3, 4, etcetera. But yes: 'versions' could be a better name. But then
as a user I would expect to see a generation/version of 1.1, 3.0, etc.
And my current rather easy idea is to simply use 0, 1, 2, etc as
profile names. I like the simplicity of that.
I'm advocating versions as in what's in the name of the tarball.
In my theory these numbers are just names of GenericSetup profiles.
If after a while Plone has 531 of these profiles in one directory then
they can easily be moved to subdirectories, as long as they are
registered under their simple integer name.
But managing 531 tiny profiles is likely to be really complicated no
matter where you put them. :)
BTW, I added 'collective.generations' to the collective. (Feel free
to rename it.) But when I add this literally to my Products dir or my
lib/python dir as collective.generations I get import errors like
this:
ImportError: No module named collective.generations
Should I rename it to a 'dot-less' name when putting it in my Products
dir or can I do this in a different way?
You need to learn about eggs and things. :)
Sorry, don't have time to go into details right now, on my way out the
door... ask on the channel :)
We could solve this by being a bit more explicit:
class IVersionManager(Interface):
"""Keep track of the current version for a product
"""
current_version = schema.TextLine("The current version")
product_name = schema.TextLine("Describe the product")
migration_steps = schema.Tuple("An ordered list of migration steps")
My current idea (subject to change) is that a version manager can
claim to handle evolutions to a *maximum* number of generations. So
it would not keep a current version number itself. The current
version/generation could be stored by a more general tool or utility
or property sheet.
I can't see a reason to ever have more than one version/upgrade manager
for the same product.
At any rate /some/ code should know which version is currently
installed. Something like this IVersionManager could do the trick.
class IMigrationStep(Interface):
"""Describes a migration
"""
to_version = schema.ASCIILine("Migrates to version - displayed to
the user")
def __call__(portal):
"""Execute this migration step
"""
I think this can be handled by simply registering a GS profile with
'to_version' as the name..
Not if version == the real release version. You don't want to build an
algorithm that works out whether 2.0b1 is before 2.1rc3. :)
Again, I don't like the implictness of this, though. :)
class VersionManager(object):
implements(IVersionManager)
def __init__(self, current, name, steps=[]):
self.current_version = current
self.product_name = name
self.steps = steps
So an instance of this class would need to be initialized with a list
of steps? I think that might take too many lines of code.
See the example below. It's one line of code per version, and it's
explicit and obvious what's going on.
But maybe we can add a method somewhere like listStepsFor(context)
where context is some package name or maybe an interface, like
IMigratingPloneSite.
Sure. Or:
poi_versions = getUtility(IVersinManager, name="Products.Poi")
for step in poi_versions.steps:
print step.version
BTW, can you think of a reason to have more than one VersionManager
for one Plone Product? I do not think there are any reasons, but the
zope 3 implementation of generations does leave room for it, so I
wonder if there is a use case for that in Plone/CMF.
I can't, no. So let's not worry about it. :)
class GSMigrationStep(object):
implements(IMigrationStep)
def __init__(self, version, profile):
self.to_version = version
self.profile = profile
This leaves room for GSMigrationSteps and something like
PlainPythonMigrationSteps. This might be good or this might be giving
room for python migrations that are just as well rendered in the
importVarious step.
That was the idea.
I propose supporting only GS steps. Do you think that is too strict?
If doing an upgrade step just means to call the step, and the
default/basic step implementation's __call__() calls a GS profile, then
that's no more work, but leaves you with a lot more flexibility should
you need it (or, should you just *not* need GS... sometimes, it may be
overkill or too cumbersome to register a new profile).
... now, in e.g. Poi, I'd do:
version = VersionManager(
name=u"Poi",
steps=[
MigrationStep("1.1", "profile-Products.Poi:1.1"),
MigrationStep("1.2", "profile-Products.Poi:1.2"),
MigrationStep("1.3", "profile-Products.Poi:1.2"),
]
)
My proposal does not need this explicit registering. Registering
profile-Products.Poi:1 implicitly registers a migration step from
generation 0 to 1.
That's what I don't like about it. :) Trust me, when it comes to
migrations, having explicit control will help you hugely. There mere
availability of a GS profile shouldn't be enough to tell you that
there's an upgrade step to be run.
Like I said, we can infer the profile name as a default from product
name + migration step to-version, but I'd still leave the door open for
an override (and for non-GS steps).
provideUtility(version, name=u"Products.Poi") # do this with ZCML
I would then vote for a name like
Products.Poi.generations/migration/installation to make the goal of
the utility clearer. But that is a minor point.
that's not the way people normally name utilities. The purpose of the
utility is described by the interface.:
getUtility(IVersionManager, name="Products.Poi")
... I want the version manager for Products.Poi. That's clear. :)
So, we have one global utility, named after the product by convention.
We specify the name of the product to display in the migration UI.
In my proposal we would initialize a SchemaManager and have an argument
package_name that also is the product name by convention/necessity.
Yes, that'd be the same.
The migration steps are listed explicitly, in order. The "latest"
version is the last one in the list (remember, it specifies "versions
being migrated to"). That makes sense - it's the last thing you can
migrate to.
In my proposal we could probably do away with the minimum_generation
as required by the base zope 3 generations and only state the number
of the last generation.
Sure. You need that if you're implicitly looking for and running
generations. If you're stating a list of migration step versions, you
don't need this - the latest one is always the last one in the list.
This also helps make clear the case when there *is* to migration to the
latest version. If I release v 2.1, but the last to_version of a
migration step is 2.0, it's clear that there's no 2.0-2.1 step. As a
user, I know what I'm doing when I push the big migration button. I
can't know what's going on when I ask for "run to generation 47,
please". :)
We reference a profile by name (we could simplify by putting the product
name in the VersionManager, and/or defaulting the profile name to
profile=${product_name}:${version}, but I think it makes sense to be
able to specify one explicitly.
You can already specify it explicitly by registering some directory
under a specific name.
When we migrate current version to 1.3, say, it goes through the list
from the current/pre-migration version, and does each step up until and
including the last one.
I am inclined to have several migration steps between version 1.2 and
1.3. That may be more guided by a wish to make testing with a
production database during development easier though.
Like I said, in Plone, we tie migrations to versions and expect people
to be able to re-run/force upgrade. I think this is most practical,
because quite often, we discover later in a release cycle that you need
to tweak the order of migrations, say. The migration from one released
version to another is what users care about, and is what, imho, should
be the most fundamental unit of design and testing when it comes to
migration.
If you have a zope/plone site that uses the svn trunk of a
Product and the official version is 1.2 but there are some
Let's say you have a production site that runs on Plone 2.5.2, which
was out in January of this year. Meanwhile there are some migration
steps. Let's say there are two migration steps defined already (one
column added to the catalog and one skin directory change). With my
strategy you could perform those steps already. With the current
Plone migration strategy I think either you can perform those steps
now and cannot perform future steps to 2.5.3 easily or you cannot
perform the currently available steps.
That's not true. You run the migration steps, and if need be, you re-run
later. That requires that steps do not unnecessarily stomp on settings,
but they need to be like that anyway!
That is my idea of Plone migration at least: migrating from the latest
stable version of 2.5 to the subversion branch of 2.5 is a bit iffy.
I do it all the time. Running an svn version is what's iffy. :p
And I think your strategy is closer to the current plone strategy.
That may just be a definition of what constitutes a
generation/migration though: I do not think your strategy mandates the
evolution from one official version to the next but can also handle
intermediate changes.
It tries to tie migrations to released (or conceptual) versions.
Adding a new migration step means adding another element to that list,
which should be pretty simple.
Yes.
To me, this is easier to understand and manager than sequential ints
that have no tie to the actual version number I'm releasing, and will
make more sense to users.
You can register a GenericSetup profile in
migrations/v3_0/profiles/beta1_beta2 under the name 42 and still give
if a readable title and description.
But I still somewhere need to think about (as a developer) what 42 means.
Maybe we could define a SchemaManager for say profile numbers 42
through 47 that handles the upgrade between beta 1 and beta 2.
Eeep! I smell off-by-one typos. :)
I think the following should then suit you just fine:
profile-Products.Poi:0
profile-Products.Poi:1
profile-Products.Poi:2
profile-Products.Poi:3
profile-Products.Poi:4
profile-Products.Poi:5
No. They have no semantic meaning and seem completely arbitrary.
Anyway... read Hanno's post, much of this may just be hot air right now,
if Tres and Rob have solved it all already. :)
Martin
_______________________________________________
Product-Developers mailing list
[email protected]
http://lists.plone.org/mailman/listinfo/product-developers
