Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
Hi, On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. OK I took a stab at this here: - http://dev.plone.org/collective/changeset/237189/collective.transmogrifier/branches/aclark-mr-migrator-compat Which works OK with packages that declare entry points like the one here: - http://dev.plone.org/collective/changeset/237623/transmogrify.filesystem/branches/aclark-mr-migrator-compat This works because there is ZCML in that package and c.transmogrifier calls zcml.load on it. But I wonder how I can make it work for blueprints like this one: - http://dev.plone.org/collective/browser/collective.transmogrifier/trunk/src/collective/transmogrifier/sections/tests.py#L204 Where the code is loaded via provideUtility… Do I need to make a special exception to handle this case? Thanks for any pointers, Alex Wichert. -- Alex Clark · http://aclark.net/training ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 15/04/2011, at 3:09 AM, Alex Clark wrote: Hi, On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. OK I took a stab at this here: - http://dev.plone.org/collective/changeset/237189/collective.transmogrifier/branches/aclark-mr-migrator-compat Which works OK with packages that declare entry points like the one here: - http://dev.plone.org/collective/changeset/237623/transmogrify.filesystem/branches/aclark-mr-migrator-compat This works because there is ZCML in that package and c.transmogrifier calls zcml.load on it. It looks like you have two systems for registration which do the same thing. The autoinclude loads zcml and the special collective.transmogrifier entrypoint also loads zcml. Wasn't the idea of the special collective.transmogrifier entrypoint to bypass zcml alltogeather, ie be an alternate registration syntax by itself so zcml wasn't needed? ie somthing like for object in iter_entry_points(group='collective.transmogrifier', name=None): object.load() and object.load() is going to call a method the plugin author has provided to call register all they blueprints without zcml see http://wiki.pylonshq.com/display/pylonscookbook/Using+Entry+Points+to+Write+Plugins But I wonder how I can make it work for blueprints like this one: - http://dev.plone.org/collective/browser/collective.transmogrifier/trunk/src/collective/transmogrifier/sections/tests.py#L204 Where the code is loaded via provideUtility… Do I need to make a special exception to handle this case? if that one was meant to be used outside of those internal tests then it would have been registered with zcml. So it's not meant to be used by others. Thanks for any pointers, Alex Wichert. -- Alex Clark · http://aclark.net/training ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 4/14/11 4:45 PM, Dylan Jay wrote: On 15/04/2011, at 3:09 AM, Alex Clark wrote: Hi, On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. OK I took a stab at this here: - http://dev.plone.org/collective/changeset/237189/collective.transmogrifier/branches/aclark-mr-migrator-compat Which works OK with packages that declare entry points like the one here: - http://dev.plone.org/collective/changeset/237623/transmogrify.filesystem/branches/aclark-mr-migrator-compat This works because there is ZCML in that package and c.transmogrifier calls zcml.load on it. It looks like you have two systems for registration which do the same thing. The autoinclude loads zcml and the special collective.transmogrifier entrypoint also loads zcml. Wasn't the idea of the special collective.transmogrifier entrypoint to bypass zcml alltogeather, ie be an alternate registration syntax by itself so zcml wasn't needed? ie somthing like for object in iter_entry_points(group='collective.transmogrifier', name=None): object.load() and object.load() is going to call a method the plugin author has provided to call register all they blueprints without zcml OK that sounds reasonable. I'd consider this to be a first step in the right direction though. see http://wiki.pylonshq.com/display/pylonscookbook/Using+Entry+Points+to+Write+Plugins Yup, I've read that like 5000 times :-) The example is fairly simple, and though it helped me get this far, it doesn't go beyond a few lines of code for each of the moving parts (i.e. plugin system and plugin consumer). But I wonder how I can make it work for blueprints like this one: - http://dev.plone.org/collective/browser/collective.transmogrifier/trunk/src/collective/transmogrifier/sections/tests.py#L204 Where the code is loaded via provideUtility… Do I need to make a special exception to handle this case? if that one was meant to be used outside of those internal tests then it would have been registered with zcml. So it's not meant to be used by others. As MJ pointed out to me a package author can do: utility component=collective.transmogrifier.sections.tests.PrettyPrinter and off they go. But the goal here is to make this sort of thing easily configurable with mr.migrator IMHO. Alex Thanks for any pointers, Alex Wichert. -- Alex Clark · http://aclark.net/training ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers -- Alex Clark · http://aclark.net/training ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 26/03/2011, at 6:44 AM, Alex Clark wrote: On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. Why not indeed, that is one of the points of this thread… I have a branch waiting to be merged down, pending MJ's approval: http://svn.plone.org/svn/collective/collective.transmogrifier/branches/aclark-mr-migrator-compat/ But he expressed some concern about the overhead; so this is the part where blueprint authors chime in and say yes! that would be great! ;-) I think what Wichert is suggesting to register configurations and blueprints directly with entry_points Something like this entry_points = { 'transmogrifier.config' : ['funnelweb = funnelweb.registerConfig'], 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] } We could put some code into collective.transmogrifier which use these registrations directly as shown in http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html This would bypass the zcml. plugin authors then have a choice of which way they want to register, zcml or entry_point. except that if they register via zcml in which case mr.migrator would still need a zcml= option to support those plugins. Alex Wichert. -- Alex Clark · http://aclark.net/help-me-help-plone ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 27/03/2011, at 3:05 AM, Alex Clark wrote: On 3/26/11 10:37 AM, Dylan Jay wrote: On 26/03/2011, at 6:44 AM, Alex Clark wrote: On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. Why not indeed, that is one of the points of this thread… I have a branch waiting to be merged down, pending MJ's approval: http://svn.plone.org/svn/collective/collective.transmogrifier/branches/aclark-mr-migrator-compat/ But he expressed some concern about the overhead; so this is the part where blueprint authors chime in and say yes! that would be great! ;-) I think what Wichert is suggesting to register configurations and blueprints directly with entry_points Something like this entry_points = { 'transmogrifier.config' : ['funnelweb = funnelweb.registerConfig'], 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] } We could put some code into collective.transmogrifier which use these registrations directly as shown in http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html This would bypass the zcml. plugin authors then have a choice of which way they want to register, zcml or entry_point. except that if they register via zcml in which case mr.migrator would still need a zcml= option to support those plugins. In that case I would say huh? :-) OK this thread is going a bit off the rails (for me at least), but let me see if I understand. Option 1(which I suggested in the first email) == - includePlugins package=transmogrify in c.transmogrifier - blueprints target transmogrify - mr.migrator Just works (as would any other runner e.g. GS, browser view, etc.) because c.transmogrifier ensures all blueprints are available at runtime (via ZCML). Option 2(what Wichert suggested) - Use entrypoints instead (of ZCML, in particular instead of a zcml= in mr.migrator) entry_points = { 'transmogrifier.config' : ['funnelweb = funnelweb.registerConfig'], 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] } (that doesn't look like valid syntax, but i guess the point is in mr.migrator you specify… nope. i got nothing, sorry :-)) XXX Can someone fill in the rest here? FWIW, I'm not trying to give blueprint authors more options, I'm trying to give end users the ability to run c.transmogrifier-based migrations. If c.transmogrifier is a framework for building pipelines and a migration is a specific set of blueprints in a specific pipeline, then mr.migrator is a way for end users to mix and match those blueprints/pipelines. As it stands, AFAICT if I want to use a particular blueprint, I have to create a migration package and configure my migration inside the package (which I personally do not want to do). ok skip the part about registering pipelines for now. people who share blue prints have to either a) register them in zcml and then also include the autoinclude entry_point or b) just register them with an entry_point syntax such as 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] The advantage of b) is you no longer need zcml and there is no need for z3c.autoinclude. ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 3/26/11 1:29 PM, Dylan Jay wrote: On 27/03/2011, at 3:05 AM, Alex Clark wrote: On 3/26/11 10:37 AM, Dylan Jay wrote: On 26/03/2011, at 6:44 AM, Alex Clark wrote: On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. Why not indeed, that is one of the points of this thread… I have a branch waiting to be merged down, pending MJ's approval: http://svn.plone.org/svn/collective/collective.transmogrifier/branches/aclark-mr-migrator-compat/ But he expressed some concern about the overhead; so this is the part where blueprint authors chime in and say yes! that would be great! ;-) I think what Wichert is suggesting to register configurations and blueprints directly with entry_points Something like this entry_points = { 'transmogrifier.config' : ['funnelweb = funnelweb.registerConfig'], 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] } We could put some code into collective.transmogrifier which use these registrations directly as shown in http://reinout.vanrees.org/weblog/2010/01/06/zest-releaser-entry-points.html This would bypass the zcml. plugin authors then have a choice of which way they want to register, zcml or entry_point. except that if they register via zcml in which case mr.migrator would still need a zcml= option to support those plugins. In that case I would say huh? :-) OK this thread is going a bit off the rails (for me at least), but let me see if I understand. Option 1(which I suggested in the first email) == - includePlugins package=transmogrify in c.transmogrifier - blueprints target transmogrify - mr.migrator Just works (as would any other runner e.g. GS, browser view, etc.) because c.transmogrifier ensures all blueprints are available at runtime (via ZCML). Option 2(what Wichert suggested) - Use entrypoints instead (of ZCML, in particular instead of a zcml= in mr.migrator) entry_points = { 'transmogrifier.config' : ['funnelweb = funnelweb.registerConfig'], 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] } (that doesn't look like valid syntax, but i guess the point is in mr.migrator you specify… nope. i got nothing, sorry :-)) XXX Can someone fill in the rest here? FWIW, I'm not trying to give blueprint authors more options, I'm trying to give end users the ability to run c.transmogrifier-based migrations. If c.transmogrifier is a framework for building pipelines and a migration is a specific set of blueprints in a specific pipeline, then mr.migrator is a way for end users to mix and match those blueprints/pipelines. As it stands, AFAICT if I want to use a particular blueprint, I have to create a migration package and configure my migration inside the package (which I personally do not want to do). ok skip the part about registering pipelines for now. people who share blue prints have to either a) register them in zcml and then also include the autoinclude entry_point or b) just register them with an entry_point syntax such as 'transmogrifier.blueprint': ['transmogrify.xmlsource = transmogrify.xmlsource.xmlsource:XMLSource'] The advantage of b) is you no longer need zcml and there is no need for z3c.autoinclude. OK I get it, I think. While refreshing myself on entry points, I found: http://stackoverflow.com/questions/774824/explain-python-entry-points/774859#774859 which sounds like a pretty good description of what we might like c.transmogrifier to become, i.e. allowing one package do use plugins that are in another package. But I have no idea if this is something we can easily convince everyone is appropriate to live in c.transmogrifier… certainly includePlugins was not. Alex -- Alex Clark · http://aclark.net/help-me-help-plone ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
[Product-Developers] Calling all collective.transmogrifier blueprint authors
Hi all, I've been working with Dylan Jay on mr.migrator: a command line way to execute transmogrifier pipelines, and we now have a working demo: - https://github.com/aclark4life/mr_migrator_demo To run it yourself, try this: $ git clone g...@github.com:aclark4life/mr_migrator_demo.git $ cd mr_migrator_demo $ python2.6 bootstrap.py -d $ bin/buildout $ bin/plone start Now create a Plone site called Plone TTW, and then: $ bin/mr.migrator At which point you will see a bunch of stuff: --- ... WARNING:upload:Failuire while creating 'http://admin:admin@localhost:8080/platea' of type 'Folder' INFO:upload:/platea Created with type=Folder Level 1:logger:/platea WARNING:upload:Failuire while creating 'http://admin:admin@localhost:8080/tortor' of type 'Folder' INFO:upload:/tortor Created with type=Folder Level 1:logger:/tortor WARNING:upload:Failuire while creating 'http://admin:admin@localhost:8080/index.html' of type 'Document' INFO:upload:/index.html Created with type=Document Level 1:logger:/index.html WARNING:upload:Failuire while creating 'http://admin:admin@localhost:8080/ad/index.html' of type 'Document' INFO:upload:/ad/index.html Created with type=Document Level 1:logger:/ad/index.html ... --- And if you check the site again you'll see content in your site (more or less; I'm still working on it.) So, now I'm writing this email to: - Share the news of command-line driven transmogrification: it's coming! and: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer zcml = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer This only works now (without the zcml=) because I have created branches of collective.transmogrify and transmogrify.filesystem where I added the includePlugins ZCML: - http://svn.plone.org/svn/collective/collective.transmogrifier/branches/aclark-mr-migrator-compat/src/collective/transmogrifier/configure.zcml and the transmogrify entry point: - https://svn.plone.org/svn/collective/transmogrify.filesystem/branches/aclark-mr-migrator-compat/setup.py So what do you product developers think? Do you care at all? Personally, I don't want to: a. Enter zcml= in my buildout.cfgs b. Write the code to make it work in mr.migrator. But that's just me. As MJ has cautioned, we need to be careful about using z3c.autoinclude because it goes a bit nuts scanning files on the file system (which is why I have it tied to that zcml:condition currently). Thoughts appreciated! Thanks, Alex [1] https://github.com/aclark4life/mr_migrator_demo/blob/master/buildout.cfg -- Alex Clark · http://aclark.net/help-me-help-plone ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. Wichert. -- Wichert Akkerman wich...@wiggy.net It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 26/03/2011, at 2:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. good point how since they are already registered using zcml however that would mean authors would have to register each blueprint twice. Wichert. -- Wichert Akkerman wich...@wiggy.net It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers
Re: [Product-Developers] Calling all collective.transmogrifier blueprint authors
On 3/25/11 11:45 AM, Wichert Akkerman wrote: On 2011-3-25 16:03, Alex Clark wrote: - Ask for feedback on the issue of making collective.transmogrifier listen for a transmogrify target, coming from blueprints who add an entry point like: setup( ... entry_points= [z3c.autoinclude.plugin] target = transmogrify ) That would allow a buildout[1] like this to work: [mr.migrator] recipe = mr.migrator pipeline = pipeline.cfg eggs = transmogrify.filesystem transmogrify.ploneremote transmogrify.pathsorter transmogrify.printer Otherwise, we'll need to make mr.migrator accept the ZCML parameter, and end users will have to list their blueprints twice: Why not use entrypoints instead instead of relying on zcml in between? That way you don't need to list them twice, and you get discovery directly from setuptools. Why not indeed, that is one of the points of this thread… I have a branch waiting to be merged down, pending MJ's approval: http://svn.plone.org/svn/collective/collective.transmogrifier/branches/aclark-mr-migrator-compat/ But he expressed some concern about the overhead; so this is the part where blueprint authors chime in and say yes! that would be great! ;-) Alex Wichert. -- Alex Clark · http://aclark.net/help-me-help-plone ___ Product-Developers mailing list Product-Developers@lists.plone.org https://lists.plone.org/mailman/listinfo/product-developers