Re: [Zope-CMF] GenericSetup proposal: upgrade profiles
Hi Rob,
Thanks for the feedback!
Rob Miller, on 2009-06-24:
i'm not actively working on GenericSetup these days, so i don't know that my
opinion counts for much, but i'm -1 on this. to me it seems to introduce an
entirely new way of handling upgrades that is likely to confuse people.
GenericSetup is more than complex enough already, IMO.
Maurits van Rees wrote:
Use case
For our use case we assume that you have made an add-on product. It
has an EXTENSION profile. You have applied the profile in a website.
Now you want to add a catalog index. Your options are:
A. Add this in the current extension profile. This means that any
time you apply this profile this index will be removed and
recreated so you lose data; you need to reindex the index (manually
or in python code - not GenericSetup) and that takes time.
in my mind this is the core problem, that there is no way to
non-destructively apply an index to a catalog. this is a weakness
in GenericSetup, but developing an entirely new alternative
mechanism for handling upgrades seems to me the wrong solution.
In general, I think we could use an option like 'override' or
'only_once', next to the purge and remove options, like this:
property name=email_from_name override=False
type=stringSite Administrator/property
property name=email_charset override=True
type=stringutf-8/property
This would then mean:
- at first the properties do not exist yet
- apply this profile - properties do exist, with the values given
- change email_from_name to 'Acme Inc' and email_charset to 'ascii'
- reapply this profile
- email_from_name is untouched at 'Acme Inc' and email_charset has
been overridden to 'utf-8'
Problem may be that if you do this in one import handler (say for the
properties) users may be caught off guard when this option does
nothing in another (like for the catalog) and loose data/time that
way.
B. Add an upgrade step. Works fine, but this needs python code along
these lines:
indexes = catalog.indexes()
if 'new' not in indexes:
catalog.addIndex('new', 'some_meta_type')
catalog.manage_reindexIndex(ids=['new'])
But why add python code when there is already a perfectly fine
import handler that can read a catalog.xml? We are back to the
dark days before there was GenericSetup then. (Well, okay, writing
python code is not *that* bad. ;-))
C. Add it to the current extension profile. Do not apply this profile
ever again: only once at install time.
this confuses me. in my mind, extension profiles should be idempotent. IOW,
it should _always_ be safe to re-apply an extension profile to a site,
assuming that extension profile does indeed represent the final state that
you'd like the site to have.
In my mind, an extension profile is just a starting point, at least
partly. Like above, you set a property email_from_name so the code
that expects this property to exist actually works. Then your client
comes along and changes this property to something he likes. You do
not want to reset this to the default when reapplying the profile.
For another part, an extension profile can serve as a set of known
good settings and be reapplied to cleanup the mess that the client
made when clicking through the ZMI.
Instead add an upgrade step
that applies the catalog step from this profile. But this has
basically the same problem as option A: if you later add a second
catalog index you would need to create a new upgrade step to apply
the catalog step and then your first index would be removed and
recreated empty again.
again, this seems to me a weakness of the catalog index importing, and it
should be fixed there.
it is option (C) which i would recommend, so much so that there is even a
dedicated ZCML tag specific for this purpose: genericsetup:upgradeDepends.
here's an example from the GenericSetup tests:
... genericsetup:upgradeDepends
... title=Bar Upgrade dependency profile import steps
... description=Re-imports steps from the profile
... import_profile=profile-Products.CMFDefault:default
... import_steps=baz bat
... run_deps=True
... purge=True
... /
upgradeDepends lets you specify any arbitrary import steps from any arbitrary
profile, so that you don't have to repeat the boilerplate code of invoking
those import steps by hand whenever you need to write such an upgrade step.
This is definitely an improvement, and I am certainly going to use
that when GenericSetup 1.5 becomes available. But it does not help
against a catalog index importer that recreates an empty index; or
against a properties importer that resets properties to their factory
default.
D. Define a new EXTENSION profile that just has a catalog.xml. Either
go to the import tab
Re: [Zope-CMF] [GenericSetup] Extension profile re-installing base profile
Andreas Jung, on 2009-06-23: We are currently migration from Zope 2.8/CMF 1.X to Zope 2.11/CMF 2.1. I have a minimal base profile and two almost identical extension profiles for product A and B (both containing toolset.xml, skins.xml and import_steps.xml). The setup code is almost identical with the one from CMFDefault.factory.addConfiguredSiteForm(). So what's happening: - creating a CMF site with the base profile + extension profile for A works as expected - creating a CMF site with the base profile + extension profile for B works but the setuphandler code of the base profile is executed twice (triggered through the installation of the extension profile for B) - creating a CMF site with the base profile + both extension profiles for A + B fails with a 'Exisiting registration...' error General question: what might be the reason for extension profile B to trigger the reinstallation of the base profile (and its setuphandler)? You say here that the base profile including its setuphandler is applied twice. Above you said that only the setuphandler is run twice. Which is it? The setuphandler of the base profile should have some code like this to make sure it is executed only at the moment the base profile itself is applied: def setuphandler_from_base_profile(context): # Only run step if a flag file is present if context.readDataFile(flagfile_for_base_profile.txt') is None: return ... Do something useful here... See for example http://maurits.vanrees.org/weblog/archive/2007/06/discovering-genericsetup But if a check like that is missing, I would expect this setuphandler to be executed when applying profile A as well, which is not happening. Perhaps there is a check that somehow treats profile A and B differently? I would say go in with a pdb and see if the traceback tells you anything about why the setuphandler gets executed again. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
[Zope-CMF] GenericSetup proposal: upgrade profiles
and BASE profiles, so that list is not so cluttered
anymore. You now just create an upgrade step that applies the
profile.
Implementation
--
The biggest changes (which are not actually that big) in GenericSetup
code would be:
- in tool.py make sure that purge is only True for BASE profiles:
def _getImportContext(...):
...
#should_purge = (info.get('type') != EXTENSION)
should_purge = (info.get('type') == BASE)
def listContextInfos(self):
- In registry.py (and tool.py) allow listing only some types, for
example here:
def listProfiles( self, for_=None, types=None ):
if types is None:
# possibly add SNAPSHOT here.
types = [BASE, EXTENSION]
result = []
for profile_id in self._profile_ids:
info = self.getProfileInfo( profile_id )
if for_ is None or issubclass( for_, info['for'] ):
if info['type'] in types:
result.append( profile_id )
return tuple( result )
And then some template changes and of course tests.
As a bonus it may be handy to combine the zcml registration of the
upgrade step and the upgrade profile, but that will be technically
more difficult. Something like this:
genericsetup:upgradeStep
title=Add catalog index
description=
source=1
destination=2
directory=profiles/upgrade2
name=upgrade2
/
This would then register directory profiles/upgrade2 as an UPGRADE
profile with name 'upgrade2' and register an upgrade step that applies
that profile. But that is icing on the cake and I am not sure if we
need more zcml constructs here.
I could do these changes (on a branch of course).
Questions
-
Is this feature interesting for inclusion in GenericSetup?
Is it in scope for 1.5.x?
Is anyone already registering profiles using numbers other than 1
(BASE) or 2 (EXTENSION)?
Do you see a use case for UNINSTALL, TEST, or SNAPSHOT?
Other comments?
Cheers,
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] Best use of source numbers in GS upgrade steps?
yuppie, on 2009-04-17:
And this tells me that my way of specifying source=1.1.9 and dest=2.0
should still work. A snippet of those tests adapted to my numbers
gives this result:
1.1.9 (source) 1.1.2 (current) 1.2 (dest)
e.versionMatch('1.1.2')
False
e.isProposed(tool, '1.1.2')
False
bool(_extractStepInfo(tool, 'ID', e, '1.1.2'))
True
If you add that test to the trunk this behavior will become officially
supported...
Ah, I meant that this test is already in trunk. The versions there
are slightly different but not in a relevant way.
trunk:
1.1 1.0 1.2
me:
1.1.9 1.1.2 1.2
and those two comparisons are exactly the same in principle:
source current dest
So trunk works fine for me.
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] Best use of source numbers in GS upgrade steps?
Hello yuppie!
yuppie, on 2009-04-16:
Maurits van Rees wrote:
So my question is: is this a sane way of doing this? Is it alright to
specify a version (or really a profile revision) as source when that
version does not yet exist? It works fine as far as I can tell.
AFAICS this will not work with GenericSetup trunk. Maybe you want to
show the upgrade step for *all* versions before 1.2? In that case you
don't specify any source version.
Yes, that is what I want. Looking at the trunk code, setting '*' as
source should have the exact behaviour that I want. Sweet!
BTW: The relevant behavior is quite inconsistent in GenericSetup
1.4.
Right, in the 1.4 code I see for example that the destination number
is never actually used, apart from showing it on the upgrades tab.
I added several tests and cleaned up the behavior on the trunk:
http://svn.zope.org/*checkout*/Products.GenericSetup/trunk/Products/GenericSetup/tests/upgrade.txt
Please let me know if I did break useful behavior.
Ah, that looks much saner, thanks! Nothing breaks here AFAICT.
And this tells me that my way of specifying source=1.1.9 and dest=2.0
should still work. A snippet of those tests adapted to my numbers
gives this result:
1.1.9 (source) 1.1.2 (current) 1.2 (dest)
e.versionMatch('1.1.2')
False
e.isProposed(tool, '1.1.2')
False
bool(_extractStepInfo(tool, 'ID', e, '1.1.2'))
True
So the version does not match and the step is not proposed, but the
step info is extracted anyway, and as far as I can see this is what
matters in the end, as this is called in listUpgradeSteps.
BTW, do I understand correctly that when in this example we add a
checker that returns False the step will still be shown?
Specifying '*' instead of '1.1.9' as source is conceptually better and
works just as well on trunk. But on GS 1.4 this has the effect of
always listing that upgrade step, as the current version is never
compared to the destination.
To get the exact same behaviour on 1.4 as on trunk I guess I could
copy the versionMatch code from trunk and add that as a checker to my
upgrade step... Seems silly though. :-)
Okay, my conclusion: I will stick to specifying 1.1.9 as source in
this case. Alternatively I will use '*' as source and make very sure
that running those upgrade steps a second time has no adverse effects
and is fast. And for an upgrade step in a package that is meant for
GenericSetup/Plone trunk I will use source=*.
Since I made some notes while investigating, I might as well share
them. So here are some random observations for reference, with some
CMFPlone versions thrown in for good measure.
- GS 1.3.3 is used in Plone 3.0.6.
- GS 1.4.1 is used in Plone 3.1.7.
- GS 1.4.2.2 is used in Plone 3.2.2.
- GS 1.4.2.2 is used in Plone 3.3rc1.
- GS trunk (1.5) is used in Plone trunk (4.0).
- The upgrade.py file is exactly the same in GS 1.3.x and 1.4.x, so
upgrade step behaviour should be the same in Plone 3.0-3.3.
- Checkers:
- GS 1.3/1.4: if a step has a checker, then the source and
destination do not matter anymore: only the return value of the
checker matters.
- GS trunk (1.5): if a step has a checker, then its return value is
used together with checks on the source and destination numbers.
- Destination:
- GS 1.3/1.4: the version destination does not matter at all (!) as
it is not used anywhere apart from being shown in the UI...
- GS trunk (1.5): the destination matters, as it is compared to the
currently applied profile revision.
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMFDefault 2.1.2-beta egg does not install
Jens Vagelpohl, on 2008-09-11:
By the way, DCWorkflow needed a similar change to make that same
traceback disappear, which means Wichert was testing CMFCore in
isolation. If possible, could we make sure to test all CMF packages
together when making changes?
I tested the last changes with a Zope 2.11 instance, with only the CMF
Products, including GenericSetup, in the Products dir. The Products
dir is a checkout of this:
ssh://svn.zope.org/repos/main/CMF/branches/2.1
bin/zopectl test -s Products passes for both GS tag 1.3.3 and branch
1.4.
But I get errors when running the CMFDefault and GenericSetup tests
individually.
Running the tests for Products.CMFDefault gives one failure with both
GS versions:
File
/home/maurits/instances/cmf/zope211-cmf21/Products/CMFDefault/browser/document.txt,
line 23, in document.txt
Failed example:
browser.open('http://localhost/site/myDocument/@@edit.html')
Exception raised:
Traceback (most recent call last):
...
VocabularyRegistryError
GS 1.3.3. gives 4 errors like this:
ValueError: Index of type DateIndex not found
GS 1.4 gives 13 ComponentLookupErrors:
ComponentLookupError: ('Could not adapt', Folder at /site, InterfaceClass
zope.component.interfaces.IComponentLookup)
I'm using a simple buildout that pulls them all in:
http://svn.dataflake.org/svn/sandboxes/cmf-21/
http://svn.dataflake.org/svn/sandboxes/cmf-trunk/
Currently this buildout (cmf-21 at least) pulls in
Products.GenericSetup 1.4.1 from the cheese shop. I am curious: why
is it not in the src/ here?
The tests *do* pass there, also the individual ones for CMFDefault and
GS. I don't see what the causes the difference. With Zope 2.11.1 in
that buildout instead of 2.10.6 the test results were the same.
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMFDefault 2.1.2-beta egg does not install
Maurits van Rees, on 2008-09-11: The tests *do* pass there, also the individual ones for CMFDefault and GS. I don't see what the causes the difference. With Zope 2.11.1 in that buildout instead of 2.10.6 the test results were the same. Okay, I was using an old Zope 2.11.0 b1. Using 2.11.1 fixes the errors in CMFDefault. GS still gives the same errors though. But the instance seems a bit confused and tries to load some products from the old Zope still, even though it is a completely new instance. I am puzzled. Tests for all Products combined pass. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMFDefault 2.1.2-beta egg does not install
Jens Vagelpohl, on 2008-09-10:
There is a reason why originally I pinned the egg at 1.33, and later
relaxed the pin to allow anything 1.3.x: I see test failures with the
1.4.1 egg. Please run the tests with the GS 1.4.1 egg and let me know
what you see. I am seeing errors like this:
snip
raise ConfigurationError(Unknown directive, ns, n)
zope.configuration.xmlconfig.ZopeXMLConfigurationError: File /usr/
local/zope/cmf-21/eggs/Products.GenericSetup-1.4.1-py2.4.egg/Products/
GenericSetup/configure.zcml, line 62.2
ConfigurationError: ('Unknown directive',
u'http://namespaces.zope.org/genericsetup'
, u'exportStep')
/snip
I see the same. I suspect that this is only some test setup that is
missing, or a missed loading of the meta.zcml of GenericSetup, but my
testing Fu fails to point me in the right direction here.
GS introduces new features, but I am not aware of any backwards
incompatible changes. Am I correct in that?
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
Re: [Zope-CMF] CMFDefault 2.1.2-beta egg does not install
Wichert Akkerman, on 2008-09-08: While working on plonenext I get an error when trying to install the Products.CMFDefault egg. This is the buildout -vv output: (...) While: Installing instance. Getting distribution for 'Products.CMFDefault'. Error: Couldn't install: Products.CMFDefault 2.1.2-beta Which is not very informative. Does anyone know what might cause this? Not very informative indeed. bin/buildout -Nv in plonenext gives me this more informative output instead: While: Installing instance. Error: There is a version conflict. We already have: Products.GenericSetup 1.4.1 but Products.CMFActionIcons 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.CMFCalendar 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.CMFCore 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.CMFDefault 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.CMFTopic 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.CMFUid 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. but Products.DCWorkflow 2.1.2-beta requires 'Products.GenericSetup==1.3.3'. So the question is: why are those products depending on GS 1.3.3? GS 1.4 has been out for a while. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See https://bugs.launchpad.net/zope-cmf/ for bug reports and feature requests
[Zope-CMF] Locales in CMFDefault
Hi, In CMFDefault there is the locales/cmf_default.pot file but no .po files. Also, there is no domain mentioned in that pot file - it should be 'cmf_default', like at the bottom of utils.py. And the directory is not registered in zcml, though I guess that does not matter since there are no po files. Are any translations available anywhere that I have missed? Is there a wish to add po files for CMFDefault in a central place like PloneTranslations where lots of people can contribute? The reason I ask is that a colleague of mine found out (when a client pointed it out) that in Plone several messages in the registration procedure are not translated. For example these ones that you can see when resetting your password (lines taken from the .pot file): == #: CMFDefault/RegistrationTool.py:71 msgid Your password must contain at least 5 characters. msgstr #: CMFDefault/RegistrationTool.py:74 msgid Your password and confirmation did not match. Please try again. msgstr == It would be nice to have messages like that translated. :-) (Now why have I not noticed this before...?) -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Eggified CMF 2.1.1 now available
Rob Miller, on 2008-08-05: Maurits van Rees wrote: Come to think of it, when my username on my laptop would have been 'mauritsvanrees' this checkout probably would have worked, so there is probably an ssh config file somewhere that I can tweak. Does anyone know how to do that? yes, b/c i just had to figure it out to check out the cmf-coredev buildout at https://svn.plone.org/svn/plone/buildouts/cmf-coredev/trunk. you have to include something like this in your ~/.ssh/config file: Host svn.zope.org User [USERNAME] the Host directive means that all subsequent config options only apply to that host, at least until another Host directive is hit. wildcards are supported, so Host * will match all hosts. Ah yes, that is it. Thanks! -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Eggified CMF 2.1.1 now available
Tres Seaver, on 2008-08-03: I created satellite versions of the 2.1.1 eggs for the remaining CMF products this morning: - CMFDefault - CMFTopic - CMFActionIcons - CMFCalendar - CMFUid - DCWorkflow Source distributions for the 2.1.1 versions are now available on PyPI. Great, thanks! But I hit a problem when I try to make a checkout the CMFActionIcons tag: $ svn co svn://svn.zope.org/repos/main/Products.CMFActionIcons/tags/2.1.1 A2.1.1/Products A2.1.1/Products/EXTERNALS.txt A2.1.1/Products/__init__.py A2.1.1/setup.py A2.1.1/README.txt U 2.1.1 Fetching external item into '2.1.1/Products/CMFActionIcons' Permission denied (publickey,gssapi-with-mic). svn: Connection closed unexpectedly This is because EXTERNALS.txt points to: CMFActionIcons svn+ssh://svn.zope.org/repos/main/CMF/tags/2.1.1/CMFActionIcons Should that not be changed to just svn instead of svn+ssh? I *do* have an ssh account for svn.zope.org, so a checkout of svn+ssh://[EMAIL PROTECTED]/... works. Am I missing some trick so that svn+ssh://svn.zope.org/ works too, without having a username in there? Come to think of it, when my username on my laptop would have been 'mauritsvanrees' this checkout probably would have worked, so there is probably an ssh config file somewhere that I can tweak. Does anyone know how to do that? BTW, I noticed this when making a checkout of the core development buildout of Plone, branch 3.2: https://svn.plone.org/svn/plone/buildouts/plone-coredev/branches/3.2 Not everyone using that checkout will have ssh access to svn.zope.org. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: License file question
Raphael Ritz, on 2008-05-29: Not sure whether that's following best practice but here is how paster/zopeskel generate this at the moment (this is taken from a custom add-on I'm currently working on): [EMAIL PROTECTED]:~/dev/paster/incf.applications/trunk$ ls docs incf incf.applications.egg-info README.txt setup.cfg setup.py [EMAIL PROTECTED]:~/dev/paster/incf.applications/trunk$ ls docs HISTORY.txt INSTALL.txt LICENSE.GPL LICENSE.txt How do others handle this? I can understand putting the HISTORY in the toplevel docs/ directory of a package. But personally I like having it inside the main folder, so in your example above it would be incf.applications/incf/applications/HISTORY.txt That way when changing a file in the main directory and you want to change the history you do not need to descend three directories and go to docs/ to change that. And in bundles with svn externals the top level docs/ directory is not even visible because you only include the main folder. So you miss the history file (and the README.txt if you keep that in docs/ as well). I remember that at least once I changed something in a plone package inside a bundle, Wichert asked me to update the history file and I ended up wrongly updating the history file of CMFPlone because I never even saw the real history file belonging to that package. :-) -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: License file question
Wichert Akkerman, on 2008-05-29: Previously Philipp von Weitershausen wrote: But personally I like having it inside the main folder, so in your example above it would be incf.applications/incf/applications/HISTORY.txt There's some benefit to that because it'll be part of the egg. You probably want to use a MANIFEST.in anyway and that can easily be used to include everything in doc/ or other places. A 'python setup.py sdist' will include the docs directory, as long as it is in subversion (and has at least one file in it). Aha, when you create an sdist in a subversion export (instead of a checkout) then the docs dir does not get included. Probably because setup.py does not point to it. With 'bdist_egg' the docs dir is indeed never included, unless you have it in MANIFEST.in. But I'm trying to avoid creating binary eggs at the moment. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: License file question
Wichert Akkerman, on 2008-05-29: Previously Maurits van Rees wrote: Wichert Akkerman, on 2008-05-29: Previously Philipp von Weitershausen wrote: But personally I like having it inside the main folder, so in your example above it would be incf.applications/incf/applications/HISTORY.txt There's some benefit to that because it'll be part of the egg. You probably want to use a MANIFEST.in anyway and that can easily be used to include everything in doc/ or other places. A 'python setup.py sdist' will include the docs directory, as long as it is in subversion (and has at least one file in it). And when someone installs from that sdist and there is no MANIFEST.in you suddenly end up with installs which are missing data files, zcml files or other things. Is there a difference in that regard between easy installing an sdist or a bdist_egg? I would think/hope that the end result is the same. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: License file question
Wichert Akkerman, on 2008-05-29: Previously Maurits van Rees wrote: Wichert Akkerman, on 2008-05-29: Previously Maurits van Rees wrote: Wichert Akkerman, on 2008-05-29: Previously Philipp von Weitershausen wrote: But personally I like having it inside the main folder, so in your example above it would be incf.applications/incf/applications/HISTORY.txt There's some benefit to that because it'll be part of the egg. You probably want to use a MANIFEST.in anyway and that can easily be used to include everything in doc/ or other places. A 'python setup.py sdist' will include the docs directory, as long as it is in subversion (and has at least one file in it). And when someone installs from that sdist and there is no MANIFEST.in you suddenly end up with installs which are missing data files, zcml files or other things. Is there a difference in that regard between easy installing an sdist or a bdist_egg? I would think/hope that the end result is the same. Yes. If you make the bdist_egg from a svn checkout it includes all files that are in subversion. If you make an egg from something else like a sdist or a svn export that does not happen (since the svn information is not available). Without a MANIFEST.in, the egg resulting from an sdist that is easy_installed is the same as a bdist_egg, right? Indeed when I try that for plone.portlet.static this is the case. So having a MANIFEST.in makes no real difference in that regard. One difference is that an sdist created from an svn *export* may not always be correct when there is no MANIFEST.in: when easy_installing an sdist created from a plone.portlet.static export, I got a complaint about a missing docs/HISTORY.txt as setup.py want to include that in the long_description of the egg. A bdist_egg created from that same export is fine. My conclusion is: as long as you remember to create an sdist or bdist_egg from an svn checkout, there is no need for a MANIFEST.in. There is one caveat: this will not include any empty directories in your package even though they are under svn control. A manifest file may help there. For reference, in the ploneout trunk only Products.PortalTransforms and Products.ExternalEditor have a MANIFEST.in file. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: License file question
Wichert Akkerman, on 2008-05-29: I'm not sure what you mean. The basic algorihm is: - MANIFEST is used to determine what is installed, or - subversion workingcopy information is used to determine what is installed, or - a default ruleset is used This algorithm is used at the moment a 'binary installation' is made, which is either when you build an egg using setup.py bdist_egg or when you install an egg from source (ie from a sdist or an unpacked tree). I think we mean the same. :-) I was not sure whether you were implying that there is a difference between a bdist_egg and an sdist that is easy installed. Apparently you agree that there is no difference, with or without a MANIFEST.in. Well, there is the point of C extensions perhaps, where at least for Windows users a compiled, windows-specific binary egg is good to have. For all other use cases an sdist is fine. At least that seems to be the current train of throught in the Grokiverse. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: genericsetup:upgradeRerunImportStep (was Re: Multiple GS profiles per product)
Rob Miller, on 2008-05-20: it's maybe worth mentioning here that i _have_ recently started using GS's upgrade machinery, and have been quite happy with it. the one thing that i've missed, which i'd like to add to the GS core if there's no strong objection, is the ability to easily register re-running a particular import step as a part of an upgrade path. i propose introducing genericsetup:upgradeRerunImportStep, which could be used as a standalone tag or nested within the grouping defined by a genericsetup:upgradeSteps tag. it would function similarly to a regular upgrade step, except instead of running an arbitrary callable handler, it would rerun a registered import step within the context of the profile with which the step is registered. it would also support an optional boolean purge attribute, which would be passed in to the import step execution as the purge argument. a sample usage might look like this: configure xmlns=http://namespaces.zope.org/zope; xmlns:genericsetup=http://namespaces.zope.org/genericsetup; genericsetup:upgradeSteps profile=PACKAGE_NAME:default source=1.0 destination=1.1 sortkey=10 genericsetup:upgradeRerunImportStep stepid=typeinfo / genericsetup:upgradeRerunImportStep stepid=workflow purge=True / genericsetup:upgradeStep title=Do something special description=Does something special handler=.upgrades.do_something_special / /genericsetup:upgradeSteps /configure And where are the original typeinfo and workflow upgrade steps defined? Are they in some other zcml file of your product? Or would they be general upgradesteps defined by GenericSetup, available for any products? I *could* imagine for example a general workflow upgrade step that reapplies the security settings based on some new workflow settings that your product has just defined. Although really that could be done like this, without needing to define a new zcml directive: genericsetup:upgradeStep title=Workflow upgrade for my product description=Same here handler=Products.GenericSetup.upgrades.apply_security_settings / So GenericSetup would then define some default upgrade handlers, just like it defines some export/import handlers. Would that suit your use case or are you thinking of something else? -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: genericsetup:upgradeRerunImportStep (was Re: Multiple GS profiles per product)
Rob Miller, on 2008-05-20: Maurits van Rees wrote: And where are the original typeinfo and workflow upgrade steps defined? Are they in some other zcml file of your product? Or would they be general upgradesteps defined by GenericSetup, available for any products? they're expected to be import steps that are already registered with the portal_setup tool. Ah, I was confusing upgrade steps and import steps. Makes more sense now. that's not quite the case i'm talking about. i'm talking about a situation where you've edited the my_workflow.xml file in your GS profile, and you want to express (using GenericSetup's upgrade step registry) that the 'workflow' import step needs to be re-applied to the site. this is extremely common... i'm always having to keep track of which import steps i need to re-run for a particular upgrade. in theory, all upgrade code should always be idempotent, so you _could_ argue that you should always just re-import the entire profile. but sometimes people make mistakes and a particular step is not idempotent. and re-importing the entire profile might be a much heavier operation than simply re-applying a fairly trivial config change. i much prefer to mark specific steps as needing to be re-run. In Plone land I am used to just reapplying the entire profile. You do not want that for e.g. catalog.xml when that defines an index, which is why I have switched to defining indexes in python code again. That could be a reason for having better upgrade support like you propose. it CAN be done w/o a custom tag... i'm currently using a 'rerun_import_steps' handler, you can see the implementation here: http://tinyurl.com/5d69my with a specific usage here: http://tinyurl.com/6m7sjm (-- ZCML) http://tinyurl.com/6zhtzw (-- handler) but this is still too much boiler-plate for me. i have to stub out my own data structure to store the steps that need to be run for a given upgrade path, including whether or not the step should be run in purge mode. all of this can be expressed in a new ZCML tag very easily, eliminating the need for boiler-plate code. as an added bonus, it makes your upgrade registration ZCML more informative, you can see at a glance everything that needs to happen for a given upgrade path. Okay, makes sense. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Multiple GS profiles per product
Thanks for the comments, Tres and Hanno. Hanno Schlichting, on 2008-05-16: Hi. Maurits van Rees wrote: This is on Plone 3.0 with CMFQuickInstaller 2.0.4. I think you are on the wrong list here. QuickInstaller is a part of Plone and not CMF and should be discussed on plone-dev. I'll give some responses anyways ;) Ah, I was thinking It starts with CMF so I will mail the CMF list. But the CMF-QI is in the plone collective indeed. My bad. Is anyone in CMF land using the CMFQuickInstaller without using CMFPlone? There are no imports from CMFPlone so it should be possible. Question 1: can I influence which profile is picked here? Should we add some code to the QuickInstaller.getInstallProfile(s) methods to for instance prefer a profile with a name like productname:default? Picking the first profile from the arbitrarily sorted list of profiles is obviously a shortcoming of QI. The main problem here is that QI uses the product name as a primary key for all its operations and thus can only really handle one installation record for one product. The whole use of extension profiles as installation procedures is a bit of a hack. What should really happen and which I'll do for Plone 4.0 is to remove the support for Extensions/Install.py and give up the one-to-one relationship between products and installation records. What happens in the end is that you apply configurations to a site - that can be as many as you want with extension profiles. I just don't see a way on how to move forward with this without a clear cut. I'm interested in this as well. If you need a hand in thinking/coding/testing ping me and I'll see what I can do. Now, I tested with eXtremeManagement 1.5.2, the latest stable release, in case anyone wants to try it out (remember to add a Poi 1.1 bundle too). That release also has Extensions/(App)Install.py files. I moved those out of the way and restarted. Why did you remove Extensions/Install.py? This one is supposed to take precedence over extensions profiles. In your case having one, which installs the GS profile you want internally should work just fine. I removed it for testing, just to see what would happen; no commits were done, no bits in the collective were hurt in the process. ;-) It was actually removed for real a while back, which still worked fine for Plone 3.0/GS 1.3, but Jean-Paul added it back for other reasons. Question 2: I am used to having a profiles directory in a product and a subdirectory inside it named 'default'. eXtremeManagement is the only product I know that has a second profile next to it. Are others using more than one profile? Well, CMFPlone does a few things here. Multiple profiles are common. I think I made the profiles/default thingy the default value, when you don't provide one in ZCML, but that's all the magic there is and should be. Question 3: Should we encourage programmers to only use one profile, presumably simply in a directory named 'profile' by default? No. :) OK. :) In the case of eXtremeManagement, the day is saved because it still has an Extensions/Install.py. That is the installer that is actually executed and it has some code to run the correct profile, including a dependency. The only hickup so far is that with the newer QI the name of the other profile is listed instead of the default profile. That is a bug. I think someone added this code of taking the title from the profile, shortly before the final and I missed to review it properly. We should just revert those changes. If you have an Extensions/Install.py nothing should be read from the profile database. Can you add a ticket for this last issue? Yes: http://dev.plone.org/plone/ticket/8142 I did not see a component that it would really fit for; I chose 'Upgrade/Migration' as in my mind that was the best fit. I was so bold as to assign the issue to you. :-) -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Multiple GS profiles per product
Tres Seaver, on 2008-05-19: Can you add a ticket for this last issue? I'm assuming that you mean against QI? Because I see nothing which needs changing in GS here. Correct. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Multiple GS profiles per product
Hi,
Some observations here with a few questions sprinkled through.
For Plone we at Zest Software made eXtremeManagement (affectionately
called xm), a project management tool. See
http://plone.org/products/extreme-management-tool
For upgrading from Plone 2.5 to 3.0 I made an upgrade profile:
basically just a regular extension profile that is not meant to be run
directly from the portal_quickinstaller or portal_setup but that is
only hooked up to an upgrade step like this:
def from_plone25_to_30(context):
context.runAllImportStepsFromProfile(
'profile-Products.eXtremeManagement:eXtremeManagement-30-types',
purge_old=False)
It removes some actions from some portal types, which seemed handy to
do in a GS profile instead of manually coding some python. Works fine
actually. This is on Plone 3.0 with CMFQuickInstaller 2.0.4.
Note that for GS upgrade steps you do not really need upgrade
profiles. That was just my understanding/expectation of how to do
upgrade steps at the time.
But: along comes Plone 3.1.1 with CMFQuickInstaller 2.1.4. This has
slightly different handling of GS profiles. In this case when
visiting the portal_quickinstaller in the ZMI it does not seem to care
much for the Extensions/Install.py, it finds the two profiles (the
upgrade profile and the default profile) and picks the first one it
finds and print its name in the list of installed/installable
products. So the QI now gives eXtremeManagement the dopey title of
eXtremeManagement types fix for Plone 3.0, which is the title of the
upgrade profile.
Question 1: can I influence which profile is picked here? Should we
add some code to the QuickInstaller.getInstallProfile(s) methods to
for instance prefer a profile with a name like productname:default?
For reference, this is info from the two profiles I now have:
[{'description': u'eXtremeManagement types actions fix for Plone 3.0.',
'for': InterfaceClass Products.CMFPlone.interfaces.siteroot.IPloneSiteRoot,
'id': u'Products.eXtremeManagement:eXtremeManagement-30-types',
'path':
u'/home/maurits/buildout/test/products/eXtremeManagement/profiles/30',
'product': 'Products.eXtremeManagement',
'title': u'eXtremeManagement types fix for Plone 3.0',
'type': 2,
'version': '1.5.2'},
{'description': 'Profile for Extreme Management',
'for': InterfaceClass Products.CMFPlone.interfaces.siteroot.IPloneSiteRoot,
'id': 'eXtremeManagement:default',
'path': 'profiles/default',
'product': 'eXtremeManagement',
'title': 'Extreme Management',
'type': 2,
'version': '1.5.2'}]
The first one is the upgrade profile, only meant to be used via the
Upgrades tab in portal_setup. The second one is the default profile
that I want to have available in the quick installer.
Now, I tested with eXtremeManagement 1.5.2, the latest stable release,
in case anyone wants to try it out (remember to add a Poi 1.1 bundle
too). That release also has Extensions/(App)Install.py files. I
moved those out of the way and restarted. On Plone 3.0 everything
seems to go fine: the name in quick installer is 'Extreme Management'
and installing it does everything it needs to do, using the default
profile. This is because the upgrade profile is not found, due to a
slightly different implementation of the getInstallProfile method.
Now, on Plone 3.1.1 (QI 2.1.4) with a fresh zodb the QI lists that
strange name of the upgrade profile instead of the name of the default
profile. And indeed when installing it only that upgrade profile is
executed, not the default profile.
Question 2: I am used to having a profiles directory in a product and
a subdirectory inside it named 'default'. eXtremeManagement is the
only product I know that has a second profile next to it. Are others
using more than one profile? Well, CMFPlone does a few things here.
Question 3: Should we encourage programmers to only use one profile,
presumably simply in a directory named 'profile' by default?
In the case of eXtremeManagement, the day is saved because it still
has an Extensions/Install.py. That is the installer that is actually
executed and it has some code to run the correct profile, including a
dependency. The only hickup so far is that with the newer QI the name
of the other profile is listed instead of the default profile.
Meanwhile on trunk (and on the 1.6rc5 I released on the cheese shop
last night) I have removed the upgrade profile as I do not like the
side effects of having two profiles.
Hm, hiding profiles by using
Products.CMFQuickInstallerTool.interfaces.INonInstallable could be an
option too, now that I think of it. Should work.
Reactions are welcome.
--
Maurits van Rees | http://maurits.vanrees.org/
Work | http://zestsoftware.nl/
This is your day, don't let them take it away. [Barlow Girl]
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http
[Zope-CMF] Re: SVN: Products.GenericSetup/branches/1.3/Products/GenericSetup/ Merge 84270 from trunk: During object manager imports do not throw an
yuppie, on 2008-02-27: Tres Seaver wrote: Marits, the new test you wrote here is failing when run on the Zope 2.11 branch or the trunk[1]: can you please diagnose why, and remediate? [1] http://mail.zope.org/pipermail/cmf-tests/2008-February/008099.html Fixed: http://svn.zope.org/?rev=84353view=rev Cheers, Yuppie Ah, thanks! I only ran the tests in a plone buildout: http://svn.plone.org/svn/plone/ploneout/trunk and http://svn.plone.org/svn/plone/ploneout/branches/3.0 and they passed there. I have now managed to setup a buildout with CMF trunk and Zope 2.11 and there it indeed fails with my code and passes with yuppies code. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: removing objects
Maurits van Rees, on 2008-02-26: Hi, Say you have for example an actions.xml like this: object name=portal_actions meta_type=Plone Actions Tool object name=history remove=True/ /object So you want to remove the history object. When you run a GS extension profile with this file twice, you get an AttributeError: at the second run the importer cannot remove this object because it was already removed. Pretty lame. Does anyone mind if I fix that so the importer just continues when the to-be-removed object is already removed? That would go like this on GS trunk: Wichert told me to go ahead. :) Fixed on GS trunk in rev 84270 and on branch 1.3 in rev 84271. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: How to use upgradeStep?
Hi, I've been on vacation I took my time answering. Rob Miller, on 2007-09-06: okay, here's where things get off track. in GenericSetup, there is no such thing (yet, perhaps) as an upgrade profile. it's possible to define an extension profile and to USE it as part of an upgrade process (Plone has been doing this), but there's no special profile type that GS knows about. Understood. upgrade steps are not meant to represent simple profile edits. for those, you'd just change the profile and up the profile version number, no need for an upgrade step at all. How do you add a version number to a profile? You can add a version to an import step in import_steps.xml, but I do not see version info in the zcml of a profile. Or do you just mean the version.txt? The quick installer uses that of course. GenericSetup does not use that anywhere as far as I see. upgrade steps are needed for more complex situations, which cannot be represented as profile edits, and which require custom python code. My main gripe with GS is currently the handling of catalog.xml. See this issue (well, feature request): http://www.zope.org/Collectors/CMF/455 When you want to remove an index or column you can do that by editing the profile and adding remove=True to that index or column. So this upgrade can be represented as a profile edit. But applying the profile will empty the remaining indexes that are mentioned in that profile. So ideally I would want to apply the profile only once when installing and rely on upgrade steps to handle any further changes without applying that complete profile again. It seems the new upgrade steps do not really solve this particular use case then. (It can sure be handy for other things, no doubt about that.) But I had hopes to use them to work around this issue with catalog.xml. Apparently a workaround is no substitute for really solving the problem. ;-) Is anyone going to the sprints after the Plone conference who wants to take a shot at this with me? Preferably someone with commit rights. :-) [snip example] performing a full upgrade, then, would require reapplying the profile configuration and running the upgrade steps. reasonably the quickinstaller (or even the GS interface) could do this all as one step. Right. Or a warning could be displayed: This profile has upgrade steps available; do you want to run them? hopefully my above explanation helps clarify the intent of the upgradeSteps directive. Certainly. Thanks a lot. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: How to use upgradeStep?
I sent this message earlier this week via gmane but it seemed to have problems at the time and I did not see the message show up. So I try again. If anyone wants to react you can take your time as I will be on vacation the next two weeks. :) Maurits van Rees, on 2007-08-21: I am having problems understanding how the new upgradeStep functionality of GenericSetup works. Maybe the most important question: is there a product that already uses this, so I can look at its code as an example? I did not get any responses yet about this. I went ahead and added an upgradeStep to eXtremeManagement, a project management Product for Plone. The trunk there is meant for Plone 2.5 and 3.0 but some tabs are duplicated in 3.0 so I wanted to fix that. The upgrade step does this. The changeset is here: http://dev.plone.org/collective/changeset/48097 The central piece of code there is the handler that simply runs all steps from an extension profile that I added for this upgrade step: def from_plone25_to_30(context): # Right, context == portal_setup here. context.runAllImportStepsFromProfile( 'profile-Products.eXtremeManagement:eXtremeManagement-30-types', purge_old=False) Does this way of using upgrade steps look sane? Like I said in my previous post, there are two things that I do not quite like about this or that I at least did not expect: - I need to specify a handler even though I only want to run the import steps of a profile, which I now have to do by hand in that handler. - I need to register an upgrade step *and* an extension profile, which feels double to me and which adds another profile to the list of extension profiles, which gets longer and longer. I hope I do not sound too much as only complaining. These upgrade steps form a good foundation. I just feel some parts are missing or have room for improvement. If other people feel the same way I do not mind putting some code where my mouth is. :) Or if I misunderstood how best to use upgrade steps, feel free to enlighten me. Thanks, -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: How to use upgradeStep?
Maurits van Rees, on 2007-08-21: I am having problems understanding how the new upgradeStep functionality of GenericSetup works. Maybe the most important question: is there a product that already uses this, so I can look at its code as an example? I did not get any responses yet about this. I went ahead and added an upgradeStep to eXtremeManagement, a project management Product for Plone. The trunk there is meant for Plone 2.5 and 3.0 but some tabs are duplicated in 3.0 so I wanted to fix that. The upgrade step does this. The changeset is here: http://dev.plone.org/collective/changeset/48097 The central piece of code there is the handler that simply runs all steps from an extension profile that I added for this upgrade step: def from_plone25_to_30(context): # Right, context == portal_setup here. context.runAllImportStepsFromProfile( 'profile-Products.eXtremeManagement:eXtremeManagement-30-types', purge_old=False) Does this way of using upgrade steps look sane? Like I said in my previous post, there are two things that I do not quite like about this or that I at least did not expect: - I need to specify a handler even though I only want to run the import steps of a profile, which I now have to do by hand in that handler. - I need to register an upgrade step *and* an extension profile, which feels double to me and which adds another profile to the list of extension profiles, which gets longer and longer. I hope I do not sound too much as only complaining. These upgrade steps form a good foundation. I just feel some parts are missing or have room for improvement. If other people feel the same way I do not mind putting some code where my mouth is. :) Or if I misunderstood how best to use upgrade steps, feel free to enlighten me. Thanks, -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] GenericSetup: How to use upgradeStep?
Hi,
I am having problems understanding how the new upgradeStep
functionality of GenericSetup works. Maybe the most important
question: is there a product that already uses this, so I can look at
its code as an example?
Warning: I am quite lengthy below in describing my (mis)understanding
of upgradeSteps. Do not continue reading if you are already
sleepy. ;-)
I am testing it in a fresh Plone 3.0 site. So that makes it CMF-2.1.0
and GenericSetup-1.3.1.
Expectation
---
The way I expect/hope to use upgradeSteps in add-on products is:
- Have one default GS profile for first time installs.
- Have one or more upgradeSteps to go from an old to a new version.
Perhaps a new version of portal_quickinstaller (or a clever
Extensions/install.py of a product) could then apply the default
profile when installing the product and could apply the correct
upgradeSteps when _re_installing.
Example
---
- I release version 1.0 of my testupgrade product. In
profiles/default it has this catalog.xml (some lines omitted for
clarity):
index name=goodName meta_type=FieldIndex /
index name=wrongName meta_type=FieldIndex /
- Alex installs version 1.0 of testupgrade by applying the GS profile.
- I spot an error in my product: one of the indexes it adds is wrong.
In profiles/default I fix this:
index name=goodName meta_type=FieldIndex /
index name=betterName meta_type=FieldIndex /
- For people like Alex I supply an upgradeStep to fix this in their
site. This step consists of a profile in profiles/upgrade_10-15/
with this catalog.xml:
index name=secondField meta_type=FieldIndex remove=True/
index name=thirdField meta_type=FieldIndex /
- With these fixes I release version 1.5 of testupgrade.
- Alex fixes his catalog by applying the upgradeStep.
- Betty is a first time user and installs version 1.5 by applying the
install profile.
Is something like this possible with the new upgradeSteps?
Problems
In a zcml file my upgrade step would like this:
genericsetup:upgradeStep
title=Cleanup catalog
description=Remove secondField and add thirdField to portal_catalog
source=1.0
destination=1.5
handler=Products.testupgrade.upgrades.from_10_to_15
sortkey=1
profile=Products.testupgrade:testupgrade-15 /
If I do not supply a handler, on Zope startup I get:
ConfigurationError: ('Missing parameter:', 'handler')
If I do not supply a profile, on Zope startup I get:
ConfigurationError: ('Missing parameter:', 'profile')
I have two problems here:
1. Since this upgradeStep needs a profile I also need to register that
profile with another zcml snippet. I had hoped that I would not
need that profile. Or rather: I had hoped that the upgradeStep
snippet would automatically register a profile (maybe by using a
name attribute).
In the ZMI there is now a notable difference between base and
extension profiles. I had also hoped for such a distinction with
respect to upgrade profiles. As it is now, the upgradeStep
itself is nicely visible on the Upgrades tab, but it needs a base
or extension profile as well, which clutters those two lists.
The current Plone 3.0 already registers four extension profiles
using its own upgrade profiles (which I think were meant as
temporary measure before the GS upgradeSteps were in place). I
want to avoid that the list of extension profiles gets cluttered
with upgrade profiles from every add-on product.
But maybe I am missing the correct meaning of the profile in
relation to the upgradeStep, as I see a second problem.
2. I am a bit surprised that I need both a handler and a profile. As
I understand it, we are supposed to move away from setuphandlers in
python code as far as possible and instead do everything with
profiles. In fact, I have hopes that the upgradeSteps make this
better possible than in the past.
As it is, I see that the code of the python handler correctly gets
called. The profile is not applied though. I guess I could put
code in the handler that actually applies the profile. But I would
hope that the upgradeStep would handle that for me. So: is the
profile attribute of an upgradeStep meant for something else?
I hope someone can shed some light on my clouded vision of how the
upgradeSteps are supposed to work. Examples would be great. :)
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
Do not worry about your difficulties in computers,
I can assure you mine are still greater.
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup extension profile import step mis-feature
Tres Seaver, on 2007-07-31: That is by design. Once an import step is grafted into the baseline, it is part of the site's configuration for ever. Resetting the baseline profile (a drastic step) is the only way to wipe it out. Which is unfortunate as it means that any Product that defines its own import step (be it via import_steps.xml or e.g. in the case of CacheFu cache.xml) needs to stay in the Products directory forever. If you remove it (even after uninstalling it and restarting Zope and removing it from Control_Panel/Products) the step is still in the registry. So it will give an error when you explicitly run it or when you run all available steps, like this: 2007-07-31 19:43:34 ERROR Zope.SiteErrorLog http://localhost:8080/kantoor/portal_setup/manage_importSelectedSteps Traceback (innermost last): Module ZPublisher.Publish, line 115, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 41, in call_object Module Products.GenericSetup.tool, line 414, in manage_importSelectedSteps Module Products.GenericSetup.tool, line 235, in runImportStep Module Products.GenericSetup.tool, line 727, in _doRunImportStep - __traceback_info__: kantoorsetup_various Module Products.GenericSetup.registry, line 146, in getStep Module Products.GenericSetup.utils, line 103, in _resolveDottedName - __traceback_info__: Products.kantoorsetup.setuphandlers.importVarious AttributeError: 'module' object has no attribute 'kantoorsetup' I went in with a zopectl debug session a few weeks ago to remove such a step manually from the registry. That worked, but having a method that you can call in an uninstall method that does this for you, would be better. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup extension profile import step mis-feature
Martin Aspeli, on 2007-07-31: Of course, there may be a good reason to have cache.xml not be processed if CacheFu isn't installed, but it's still unpredictable. The only way I can make it predictable would be if I explicitly declared a dependency on this profile and/or I explicitly declared a dependency on the product that provided it. Is there anything wrong with declaring CacheFu a dependency when you actually depend on it? And on the other hand: if you have a cache.xml file in e.g. Poi and someone else uses that product but does not want to use CacheFu, then without declaring a dependency the cache.xml just does not get processed, which is exactly what the user wants then. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: [GenericSetup] One profile for different GS versions
Andreas Jung, on 2007-07-18: I have an extension profile that works with Plone 3.0 (basically modifying = portal_skins and portal_actions). For Plone 2.5 compatibility my=20 actions.xml and skins.xml need to be different because the XML structure is = different between. What is the best approach for maintaining XML files for=20 both Plone 2.5 and Plone 3.0 in one profile (at least within the same source tree)? There might be some support for conditions in some of the xml files, but I am not aware of any. Other than that, I would say you need more than one profile. profiles/25 and profiles/30 could be copies of each other, except for the minor differences. Any changes to one profile would have to be kept in synch with the other profile, which is not terribly nice, but can be done. Or profiles/default could have all the configuration that is valid for both Plones and the 25 and 30 profiles could have only actions.xml and skins.xml or maybe only the relevant parts of those files. It would then be up to the user to pick the right profile in portal_setup. Or you could manage it for them in an install.py. I need something like that for the eXtremeManagement product too I think. I have a few settings in the various types definitions that lead to multiple sharing tabs and a superfluous metadata tab on Plone 3.0. No big deals, but not very nice either. I thought about making a subversion branch once Plone 3 is out, but two profiles might do the trick too. Thanks for the tip. Other ideas anyone? -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Moving to browser views
Charlie Clark, on 2007-07-18: Okay, thanks! I'm nearly there. I think. %-? ;-) Because I'm only creating essentially a slightly different view of an event I've taken the ZCML for the standard view as a lead: browser:page for=Products.CMFCalendar.interfaces.IEvent layer=Products.CMFDefault.interfaces.ICMFDefaultSkin name=event_iCal_view class=.event.EventiCalView permission=zope2.View / is event_iCal_view my view class? No, that is the name. That means you can do tal:define=view context/@@event_iCal_view in a template that has an IEvent provider as context. The class is the EventiCalView class in event.py. So, let's say I need to turn my creation date into 20030127T09Z I can call this in my view class via self.startdate = self.context.CreationDate() and call this in my template via view/startdate? Yes. You need to do the tal:define above first though, unless there is some other magic (well, code) that hooks up this view to that specific template. I think that is done when you specify template=mytemplate in the zcml snippet. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: state of GenericSetup trunk and branches
Rob Miller, on 2007-06-25: it's a reasonable first pass, and it's much better than not having it there. a better approach would be that taken by Plone, where you have the import step check the profile for the existence of a file with a specific name. if the file exists, run the step, if not, don't. then you 'touch' that file so it exists in your profile. I switched to checking a file now. Thanks. (http://dev.plone.org/collective/changeset/44567) B. These importVarious steps that eXtremeManagement and other products define, and that cannot be mapped to other handlers, are they good practice? Or are they an abomination, bound to lead to problems? they're not exactly an abomination, but they do have a gigantic gotcha that you've hit on here. i'll restate it clearly: IT IS THE RESPONSIBILITY OF AN IMPORT STEP'S IMPLEMENTATION TO ENSURE THAT IT IS INDEED APPROPRIATE TO PERFORM ITS ACTIONS DURING ANY GIVEN INVOCATION OF THE STEP. all of the XML-based import steps already do this; they check for the existence of a specific XML file, and if they find it they perform the action. if they do not find the file, no problem, they do nothing. the so-called 'importVarious' steps, i.e. any step that uses a plain old python function as its handler (as opposed to building on the existing XML parsing infrastructure), must perform this check explicitly. you could restrict it to only running when the intended profile is the one being imported, or you could check for the existence of a specific file within the profile. i like the latter choice. i tried to say this in my tutorial on plone.org (http://plone.org/documentation/tutorial/genericsetup), but i wrote that some time ago, it could probably be updated to be more clear about this issue. I think I have read at least part of that a while ago before I started with GS profiles. I should probably read that again now I have more experience with GS. anyway, hope this clears things up. Sure does. Thanks. I gathered some things I learned about GS the last days and put it on my weblog, quoting you extensively. :) http://maurits.vanrees.org/weblog/archive/2007/06/discovering-genericsetup -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: state of GenericSetup trunk and branches
Rob Miller, on 2007-06-25: all of the steps for a profile will be loaded into the setup tool the first time you run any of the steps for that profile. what's missing is the option of loading all of the steps from a profile WITHOUT actually running them, so that you could run only the importVarious from your profile before running any of the other ones. i imagine a 'load import steps' button on the profile page, next to the 'import' button. The patch I linked to (http://paste.plone.org/15349) does that. Well, the button is labelled Register Selected Extensions there. i like the file check, myself. my second choice would be checking the active profile by profile id. the path check you're using would be my third choice. but it's better than nothing. I am using the file check now. Hm, I just selected my importVarious step and ran that step with its dependencies, but I did that with the base (Plone) profile. This step has three dependencies. These dependencies are imported (but with the xml files from Plone as their data) and then the importVarious step itself exits early because of the check I added: INFO GenericSetup.propertiestool Properties tool imported. INFO GenericSetup.catalog Catalog imported. INFO GenericSetup.workflow Workflow tool imported. INFO GenericSetup.eXtremeManagement Nothing to import: not in eXtremeManagement path So the dependencies are run, but then the code in the depending step realizes it does not actually need to run itself, so running the dependencies was not really needed. I understand what is happening, but it sounds a bit wrong to me. i agree that it's awkward, but it is actually the right behaviour. ideally the profile check would happen before the dependencies are run. On the other hand: this only happens when you have manually selected an import step that the selected profile does not have a file to import for. So then it is the user's own mistake. Ideally, the user would not make a mistake. ;-) -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Effective Date inconsistencies
Wichert Akkerman, on 2007-06-23:
Previously Maurits van Rees wrote:
Wichert Akkerman, on 2007-06-23:
I misread the interface; that only specified what should happen if no
timezone is given. The interface says that the method should return a
string which corresponds to the time or None if no time is set. So the
behaviour you see is expected and can not be changed without breaking
backwards compatibility.
Where are you reading that? Products.CMFCore.interfaces.IDublinCore
only has some comments:
Look at what interfaces/__init__.py does: it imports the interface from
_content.py. DublinCore.py is a bbb stub.
Ah, there. :)
But that interface says:
Result is a string, formatted '-MM-DD H24:MN:SS TZ', or None.
I parse that as:
Result is (a string, formatted '-MM-DD H24:MN:SS TZ',) or None.
and not as:
Result is a string, formatted ('-MM-DD H24:MN:SS TZ' or 'None').
Put simpler, I parse that as:
Result is a string or None.
which seems just what Limi wants.
In zope.dublincore that I quoted, the idea was that the result is a
formatted string or an empty string. In other words, there it is
always a string, though it may be empty. At least the boolean values
of the empty string and None are the same: False, as opposed to
bool('None') which is True.
I am not sure where I am going with this line of reasoning :) but it
seems to me the zope.dublincore interface is slightly preferable,
always returning a string and still giving a good boolean value.
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
Do not worry about your difficulties in computers,
I can assure you mine are still greater.
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: Effective Date inconsistencies
Wichert Akkerman, on 2007-06-23: Previously Wichert Akkerman wrote: Previously Alexander Limi wrote: Not so: - EffectiveDate returns a *string* 'None' when it has no value (wtf?) That is wrong: it should return 'the time in the system default timezone' as documented in T Products.CMFCore.interfaces.IDublinCore. I misread the interface; that only specified what should happen if no timezone is given. The interface says that the method should return a string which corresponds to the time or None if no time is set. So the behaviour you see is expected and can not be changed without breaking backwards compatibility. Where are you reading that? Products.CMFCore.interfaces.IDublinCore only has some comments: # BBB: module will be removed in CMF 2.2 # zope2 interfaces created on runtime: # - DublinCore # - CatalogableDublinCore # - MutableDublinCore In zope(.app).dublincore.interfaces I see class ICMFDublinCore(Interface): ... def EffectiveDate(): Return the effective date The value of the first Dublin Core `Date` element qualified by 'effective' is returned as a unicode string if a qualified element is defined, otherwise, an empty unicode string is returned. The string is formatted '-MM-DD H24:MN:SS TZ'. So: either it returns a nicely formatted string with a date or it returns an empty string, but not 'None'. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: state of GenericSetup trunk and branches
Maurits van Rees, on 2007-06-23:
2. What needs to happen on the import tab now on trunk? We want a
drop down that lists all extension profiles. When I select one of
those extension profiles, should I get a list of only those steps
for which this profile has an xml file?
This does not seem possible, unless we add more info about steps to
the registry: the name of the xml file that the handler of that step
will read.
Or should I just get a
list of all steps available (several dozen on Plone 3) and have the
extension profile as context when I click the correct button?
I added that. I have no commit rights, so for the total changes
(including the previous one for the Profiles tab), see here:
http://paste.plone.org/15349
Just select your favorite extension profile, perhaps select some steps
and then click the Import selected/all button.
The tests still pass, but I did not add extra tests. The current
manage_* methods from the setup tool are also not tested, so I do not
feel very guilty. ;-)
One thing that we may want to do differently: with this code it is
possible to create a new Zope and Plone site, go to portal_setup, go
to the Import tab *without* going to the Profiles tab first, select
the eXtremeManagement extension profile (if you have that in your
Products dir) and run all steps. Then all steps get run, except the
importVarious step, as that has not been registered yet. Wait, that
does not actually happen. The importVarious step gets run after all.
Maybe it now gets registered on the fly in some part of the code. Oh
well.
One other thing that strikes me as odd: I added a few lines in my
setuphandler like:
logger.info(Migrated schema of %s, contentType)
In the event log this correctly gives lines like this:
INFO GenericSetup.eXtremeManagement Migrated schema of eXtremeManagement.Task
But the message log on the import tab gives me lines like this:
eXtremeManagement: Migrated schema of %s
So the argument is lost somewhere.
As a test I added my own eXtremeManagement product in the Products
dir, which also has a GS extension profile with an importVarious
step. I registered this profile with the patch I linked to above.
Then I tried to import all steps. This meant that the
eXtremeManagement importVarious step was run, which gave problems
as it assumed some portal types to be available, which were not
there. I now fixed that by checking what the context is:
def importVarious(context):
if 'Products/eXtremeManagement/profiles/default' \
not in context._profile_path:
return
...
I now changed that to:
if not context._profile_path.startswith(os.path.split(__file__)[0]):
logger.info('Nothing to import: not in eXtremeManagement path')
return
I see that some other products (Plone, CMFEditions) use a different
strategy:
# Only run step if a flag file is present
if context.readDataFile('cmfeditions_various.txt') is None:
return
So they add a simple text file to their profile directory and if that
does not exist in the current import context, then the importVarious
function quits. Works too.
Any thoughts on which of the two strategies is better?
Hm, I just selected my importVarious step and ran that step with its
dependencies, but I did that with the base (Plone) profile. This step
has three dependencies. These dependencies are imported (but with the
xml files from Plone as their data) and then the importVarious step
itself exits early because of the check I added:
INFO GenericSetup.propertiestool Properties tool imported.
INFO GenericSetup.catalog Catalog imported.
INFO GenericSetup.workflow Workflow tool imported.
INFO GenericSetup.eXtremeManagement Nothing to import: not in eXtremeManagement
path
So the dependencies are run, but then the code in the depending step
realizes it does not actually need to run itself, so running the
dependencies was not really needed. I understand what is happening,
but it sounds a bit wrong to me.
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
Do not worry about your difficulties in computers,
I can assure you mine are still greater.
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: state of GenericSetup trunk and branches
Rob Miller, on 2007-06-21: Wichert Akkerman wrote: The new GS looks very nice, but from the ZMI UI I can no longer see how I can import only selected steps of an extension profile. I hope that is still possible; it can be extremely useful. It looks like the import and export tab only act on the base profiels now. hmm... i think that the aggregate of all of the steps from the various profiles that have ever been imported should show up on the import screen, so it should be possible to select ones that were added by a specific extension profile. but this doesn't help, of course, when you only want to re-import the portal types step from an extension profile. Indeed: currently you will just import the portal types as defined by the base profile. so you're right, the UI is missing a piece. two pieces, actually. on the 'profiles' tab, there should be another button, load import steps, which will register the steps w/o actually running them. (importing a profile does this for you.) then the import and export tabs can have a dropdown that lets you choose from any profiles that have had the steps loaded. i won't be able to get to that until next week, unfortunately. anyone else? I have some code that I think does the first part: adding a button on the Profiles tab and adding some code to support it: http://paste.plone.org/15348 During testing I noticed that my knowledge of GenericSetup might be flawed though, as I discovered some things that were new to me, so I have some questions that crept up. 1. In GenericSetup 1.2, if you select an extension profile on the Properties tab (now called Profiles), and then go to the Import tab and import all steps, is that extension profile then basically treated as a BASE profile? 2. What needs to happen on the import tab now on trunk? We want a drop down that lists all extension profiles. When I select one of those extension profiles, should I get a list of only those steps for which this profile has an xml file? Or should I just get a list of all steps available (several dozen on Plone 3) and have the extension profile as context when I click the correct button? 3. When I install Plone 3.0 with the new GenericSetup trunk, and go to the Import tab, there are about 37 steps listed. That is a lot. Especially there are a few importVarious steps of different products, at least when you add some third party products in the Products dir. I do not know how widespread and generally accepted those importVarious steps are for third party products. I do notice that in the current situation in Plone 3 when you go to that import tab and click Import All Steps, all registered steps are tried, including all third party importVarious steps. As a test I added my own eXtremeManagement product in the Products dir, which also has a GS extension profile with an importVarious step. I registered this profile with the patch I linked to above. Then I tried to import all steps. This meant that the eXtremeManagement importVarious step was run, which gave problems as it assumed some portal types to be available, which were not there. I now fixed that by checking what the context is: def importVarious(context): if 'Products/eXtremeManagement/profiles/default' \ not in context._profile_path: return ... In fact, when running all import steps, the profile path of the context is that of CMFPlone: u'/home/maurits/instances/xm30/Products/CMFPlone/profiles/default' Anyway, for extension profiles we want to give the users a means on that import tab to get the context right. I guess my question is: A. Is my change to eXtremeManagement in the importVarious function above logical? I committed it already, so it had better be. :) B. These importVarious steps that eXtremeManagement and other products define, and that cannot be mapped to other handlers, are they good practice? Or are they an abomination, bound to lead to problems? Thanks for your time, -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: remove columns from catalog.xml
Jens Vagelpohl, on 2007-05-15: The mailing list is a bad place for patches. True. Please file a collector issue in the CMF collector and attach your patches there. Done: http://www.zope.org/Collectors/CMF/483 -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] GenericSetup: bbq branch
Hi,
I took a look at the GS branch from the bbq sprint:
svn://svn.zope.org/repos/main/GenericSetup/branches/tseaver-bbq_sprint
or in the browser:
http://svn.zope.org/GenericSetup/branches/tseaver-bbq_sprint/
With this post I hope to spread a small beam of light on some failing
tests and get some motion on this branch again.
There are failing tests in the test_zcml.py file, along these lines:
$ bin/zopectl test -s Products.GenericSetup -m test_zcml
...
File
/home/maurits/instances/plone3/Products/GenericSetup/tests/test_zcml.py,
line 197, in Products.GenericSetup.tests.test_zcml.test_registerUpgradeSteps
Failed example:
step1.title
Expected:
u'Bar Upgrade Step 1'
Got:
u'Foo Upgrade Step 1'
Probably the main authors of that branch already know why this fails.
The problem there is that the upgrade steps are in the wrong order.
And the part of the code that is tested here does not do anything with
the sortkey. So the upgrade steps end up in the wrong order.
I do not know if this means that the tests need to be changed or if
the underlying code needs to be changed. If the tests simply need
changing, then this may be a start. Instead of this:
from Products.GenericSetup.upgrade import _upgrade_registry
profile_steps = _upgrade_registry.getUpgradeStepsForProfile('default')
do this:
from Products.GenericSetup.upgrade import _upgrade_registry
from Products.GenericSetup.upgrade import listUpgradeSteps
profile_steps = listUpgradeSteps(_upgrade_registry, 'default', '1.0')
The reason is that the listUpgradeSteps function looks to be the only
function that actually does anything with the sortkey. I might be
wrong.
I have diff here that does this and fixes up some more related tests:
http://paste.plone.org/14641
Or the most important part of that test file in new form:
http://paste.plone.org/14642
But that introduces new errors like this:
step1.handler
AttributeError: 'dict' object has no attribute 'handler'
These are unavoidable when you use the listUpgradeSteps function
though, as that returns a list with a list of dicts which indeed have
no 'handler' attribute.
I hope this report helps someone who has more clue. ;-) Maybe I can
take another look later. At any rate, the branch looks very useful
and I want it in trunk yesterday. :)
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
Do not worry about your difficulties in computers,
I can assure you mine are still greater.
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] GenericSetup: remove columns from catalog.xml
Hi,
A while ago I proposed to add the possibility to say in GenericSetup's
catalog.xml that an index should be removed. This has been added to
the code already.
Now I propose to do the same for metadata columns. So the following
should remove the column from the catalog:
column value=outdatedColumn remove=True /
I do most of my testing on Plone 2.5, so we are talking GenericSetup
branch 1.2 here. The code below ought to do it. I added test code
for this. And I added some comments to the test to clarify how the
testing is done, as I find myself wondering about that every time I
look at the current test code. :)
If this diff is good, can someone commit it? I have no rights.
[EMAIL PROTECTED]:~/svn/plone25-zope29/GenericSetup $ svn diff
Index: ZCatalog/tests/test_exportimport.py
===
--- ZCatalog/tests/test_exportimport.py (revision 75759)
+++ ZCatalog/tests/test_exportimport.py (working copy)
@@ -62,7 +62,7 @@
extra name=index_type value=Okapi BM25 Rank/
extra name=lexicon_id value=foo_plexicon/
/index
- column value=eggs/
+%s column value=eggs/
column value=spam/
/object
@@ -78,17 +78,33 @@
extra name=lexicon_id value=foo_plexicon/
/index
index name=non_existing remove=True/
+ column value=non_existing remove=True/
+ column value=bacon remove=True/
/object
+# START SITUATION
+#
+# The catalog starts out as the _CATALOG_BODY above with the following
+# xml snippets inserted.
+
+_VOCABULARY_XML = \
+ object name=foo_vocabulary meta_type=Vocabulary deprecated=True/
+
+
_TEXT_XML = \
index name=foo_text meta_type=TextIndex deprecated=True/
-_VOCABULARY_XML = \
- object name=foo_vocabulary meta_type=Vocabulary deprecated=True/
+_COLUMN_XML = \
+ column value=bacon/
+# END SITUATION
+#
+# The catalog ends as the _CATALOG_BODY above with the following
+# xml snippets and some empty strings inserted.
+
_ZCTEXT_XML = \
index name=foo_text meta_type=ZCTextIndex
indexed_attr value=foo_text/
@@ -97,7 +113,6 @@
/index
-
class ZCatalogXMLAdapterTests(BodyAdapterTestCase):
def _getTargetClass(self):
@@ -151,6 +166,7 @@
self._populate(self._obj)
obj._setObject('foo_vocabulary', Vocabulary('foo_vocabulary'))
obj.addIndex('foo_text', 'TextIndex')
+obj.addColumn('bacon')
def setUp(self):
import Products.GenericSetup.PluginIndexes
@@ -165,22 +181,24 @@
zcml.load_config('configure.zcml', Products.GenericSetup.ZCTextIndex)
self._obj = ZCatalog('foo_catalog')
-self._BODY = _CATALOG_BODY % ('', '')
+self._BODY = _CATALOG_BODY % ('', '', '')
def test_body_get_special(self):
+# Assert that the catalog starts out the way we expect it to.
self._populate_special(self._obj)
context = DummySetupEnviron()
adapted = getMultiAdapter((self._obj, context), IBody)
self.assertEqual(adapted.body,
- _CATALOG_BODY % (_VOCABULARY_XML, _TEXT_XML))
+ _CATALOG_BODY % (_VOCABULARY_XML, _TEXT_XML,
_COLUMN_XML))
def test_body_set_update(self):
+# Assert that the catalog ends up the way we expect it to.
self._populate_special(self._obj)
context = DummySetupEnviron()
context._should_purge = False
adapted = getMultiAdapter((self._obj, context), IBody)
adapted.body = _CATALOG_UPDATE_BODY
-self.assertEqual(adapted.body, _CATALOG_BODY % ('', _ZCTEXT_XML))
+self.assertEqual(adapted.body, _CATALOG_BODY % ('', _ZCTEXT_XML, ''))
def test_suite():
Index: ZCatalog/exportimport.py
===
--- ZCatalog/exportimport.py(revision 75759)
+++ ZCatalog/exportimport.py(working copy)
@@ -137,5 +137,10 @@
if child.nodeName != 'column':
continue
col = str(child.getAttribute('value'))
+if child.hasAttribute('remove'):
+# Remove the column if it is there
+if col in self.context.schema()[:]:
+self.context.delColumn(col)
+continue
if col not in self.context.schema()[:]:
self.context.addColumn(col)
--
Maurits van Rees | http://maurits.vanrees.org/ [NL]
Work | http://zestsoftware.nl/
Do not worry about your difficulties in computers,
I can assure you mine are still greater.
___
Zope-CMF maillist - Zope-CMF@lists.zope.org
http://mail.zope.org/mailman/listinfo/zope-cmf
See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: CMF Tests: 9 Failed, 2 Unknown
Maurits van Rees, on 2007-03-29: I see the same problem in a Plone Product of mine (eXtremeManagement) where bookings added after the DST get listed a day earlier in one page template. When I add a booking somewhere in November (I can choose the booking date) that one gets listed fine again. I am wondering what should be done. I *think* I can fix my own Python Script that is responsible for this now I know that DST is involved. I want to move to a browser view anyway. For the record: I just fixed it in my product. This is probably *very* specific to my product, but maybe it gives someone hints for the failing cmf tests. Here is the changeset: http://dev.plone.org/collective/changeset/39931/eXtremeManagement I was iterating through all days of the month, doing date+1 to go to the next day. That is apparently not the right thing to do in a DST context. I now fixed this by incrementing the day count and constructing a new DateTime object with date = DateTime(year, month, day). -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: GenericSetup: catalog.xml ideas
Hi yuppie, Sorry for reacting so late ro your swift reaction. yuppie, on 2007-03-02: Hi Maurits! Maurits van Rees wrote: So here are some points, with suggested changes to the _initIndexes function in exportimport.py. The tests even run after this, so it seems these changes do not break anything. But I have not *added* tests for this. To get your patches accepted it would be useful to have new tests that fail without the patches. I'll see if I can come up with something, if I can fit that into my schedule somewhere. There is always more to do. This requires an extra line, but the result should be the same: index name=getAssignees remove=True index name=getAssignees meta_type=KeywordIndex indexed_attr value=getAssignees/ /index Ah, that works and solves the most pressing part of my issue. Thanks. Or has this been debated to death already without me knowing it? :) Well. There is an open collector issue you should know: http://www.zope.org/Collectors/CMF/455 Ah, I forgot that I filed that one. :) I understand things a bit more now, thanks. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
