[Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
Andreas Jung wrote: --On 28. Februar 2008 20:35:09 +0100 Dieter Maurer [EMAIL PROTECTED] wrote: Andreas Jung wrote at 2008-2-28 07:13 +0100: --On 27. Februar 2008 21:59:58 + Maurits van Rees [EMAIL PROTECTED] wrote: greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. How about the following idea: - within the Zope core we define an _optional_ interface for indexes - something like: class IIndexConfiguration(Interface): def getConfiguration(): Returns a dict with index specific configuration parameters. - on the CMF/GS side we could register adapter for each index type we know (basically the Zope 2 core indexes, ExtendedPathIndex, TextIndexNG 3) and retrieve the related information I do not understand why something like this should be necessary. When the export handler is able to extract all relevant configuration parameters for an index, why should the import handler not be able to check the configuration parameters in a profile against an existing index and determine that it needs to do nothing? Huh? Because we're looking for a clean solution and not for a hack! Because this solution is extensible. An index can provide the introspection directly or another application could implement the functionality as needed through adaption. Having something hardcoded for each index type within the setuphandlers is a hacker solution. And the setuphandler should ideally not know about index internals. Respectfully, I'd say that if we can have a hacky solution today that doesn't wipe indexes on re-install, then that's very valuable. The set of standard index types is well known and doesn't change much. Of course, we should also provide a way to get an interface or something else describing the configuration for introspection purposes. Waiting one or two Zope versions for that to get a non-purging GS import handler when there's a works-90%-of-the-time solution (falling back on current behaviour) would be a shame though. Martin -- Author of `Professional Plone Development`, a book for developers who want to work with Plone. See http://martinaspeli.net/plone-book ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
--On 29. Februar 2008 08:39:05 + Martin Aspeli [EMAIL PROTECTED] wrote: Of course, we should also provide a way to get an interface or something else describing the configuration for introspection purposes. Waiting one or two Zope versions for that to get a non-purging GS import handler when there's a works-90%-of-the-time solution (falling back on current behaviour) would be a shame though. You don't have to wait for for new Zope versions. Defining the interface officially in Zope 2.10, 2.11 and trunk will raise no problems. I have no problems with putting this into the official release branches - or? Andreas pgpKFtun9qiGV.pgp Description: PGP signature ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
--On 29. Februar 2008 14:07:57 +0100 Jens Vagelpohl [EMAIL PROTECTED] wrote: On Feb 29, 2008, at 13:17 , Andreas Jung wrote: --On 29. Februar 2008 13:09:01 +0100 Wichert Akkerman [EMAIL PROTECTED] wrote: My personal opinion: I'd rather see the interface-based solution in a few weeks or a couple months (the next Zope release) than the, umh, less-than-professional solution that will stick around forever. As such solutions have a tendency to do. It works now absolves everyone from the task to come back later and improve the solution, so no one does. Sorry, I can't follow...what is the outcome? I volunteer to add the interface to the Zope 2.10-2.11 branches and trunk right now. This would be good enough for you for writing the related adapter. The related code can be moved already into the Zope core on the trunk (but not for any of the release branches). I misunderstood one thing here. You only talked about the interface, but I kept thinking implementation as well :-) So my desired outcome was implementation plus interface, so that everything is ready to be used with the next release. I gave you the interface and you'll put the implementations as adapters for all indexes you need into GS. With the interface alone we only help those indices that are not part of Zope itself, since the Zope core indices apparently won't be able to have a working implementation until Zope 2.12 comes around..? Sure - through adaptation with the GS core. You can of course depend at some point that the core indices implement the behavior on their own. But adapter approach allows you to deal with the GS problem right now and you don't have to wait until Zope 2.12. Andreas pgpKs1BoMytgq.pgp Description: PGP signature ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
--On 29. Februar 2008 16:45:19 +0100 yuppie [EMAIL PROTECTED] wrote: Andreas Jung wrote: --On 28. Februar 2008 09:38:31 +0100 yuppie y.2008-E2EsyBC0hj3+aS/[EMAIL PROTECTED] wrote: I'd prefer a IConfigurableIndex interface that also has a set method. I added the IIndexConfiguration to the Zope trunk. I don't think that a set method is a good idea. Removing and re-adding is possibly the cleanest solution. Some indexes might perform some configurations within their constructor. Calling clear() would not be enough for getting their configuration right. All the export/import adapters shipped with GenericSetup and the adapter shipped with TextIndexNG3 *modify* the indexes and call clear(). AFAICT this works fine. And with an official set method it would no longer be a hack. Switching to the remove-and-re-add pattern would not be easy. So just go ahead and define the interface as you need it :-) Andreas pgpsRvw2eJ63k.pgp Description: PGP signature ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
Andreas Jung wrote: --On 28. Februar 2008 09:38:31 +0100 yuppie y.2008-E2EsyBC0hj3+aS/[EMAIL PROTECTED] wrote: I'd prefer a IConfigurableIndex interface that also has a set method. I added the IIndexConfiguration to the Zope trunk. I don't think that a set method is a good idea. Removing and re-adding is possibly the cleanest solution. Some indexes might perform some configurations within their constructor. Calling clear() would not be enough for getting their configuration right. All the export/import adapters shipped with GenericSetup and the adapter shipped with TextIndexNG3 *modify* the indexes and call clear(). AFAICT this works fine. And with an official set method it would no longer be a hack. Switching to the remove-and-re-add pattern would not be easy. Cheers, Yuppie ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
Andreas Jung wrote at 2008-2-29 06:43 +0100: ... When the export handler is able to extract all relevant configuration parameters for an index, why should the import handler not be able to check the configuration parameters in a profile against an existing index and determine that it needs to do nothing? Huh? Because we're looking for a clean solution and not for a hack! Why is it a hack when the import handler uses as much detail about an object as the export handler does? As I understood Wichert, we can currently export indexes reliably but we cannot import it without removal and recreation of the index. I cannot understand this. -- Dieter ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
Andreas Jung wrote at 2008-2-28 07:13 +0100: --On 27. Februar 2008 21:59:58 + Maurits van Rees [EMAIL PROTECTED] wrote: greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. How about the following idea: - within the Zope core we define an _optional_ interface for indexes - something like: class IIndexConfiguration(Interface): def getConfiguration(): Returns a dict with index specific configuration parameters. - on the CMF/GS side we could register adapter for each index type we know (basically the Zope 2 core indexes, ExtendedPathIndex, TextIndexNG 3) and retrieve the related information I do not understand why something like this should be necessary. When the export handler is able to extract all relevant configuration parameters for an index, why should the import handler not be able to check the configuration parameters in a profile against an existing index and determine that it needs to do nothing? -- Dieter ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
--On 28. Februar 2008 20:35:09 +0100 Dieter Maurer [EMAIL PROTECTED] wrote: Andreas Jung wrote at 2008-2-28 07:13 +0100: --On 27. Februar 2008 21:59:58 + Maurits van Rees [EMAIL PROTECTED] wrote: greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. How about the following idea: - within the Zope core we define an _optional_ interface for indexes - something like: class IIndexConfiguration(Interface): def getConfiguration(): Returns a dict with index specific configuration parameters. - on the CMF/GS side we could register adapter for each index type we know (basically the Zope 2 core indexes, ExtendedPathIndex, TextIndexNG 3) and retrieve the related information I do not understand why something like this should be necessary. When the export handler is able to extract all relevant configuration parameters for an index, why should the import handler not be able to check the configuration parameters in a profile against an existing index and determine that it needs to do nothing? Huh? Because we're looking for a clean solution and not for a hack! Because this solution is extensible. An index can provide the introspection directly or another application could implement the functionality as needed through adaption. Having something hardcoded for each index type within the setuphandlers is a hacker solution. And the setuphandler should ideally not know about index internals. Andreas pgpjgcir8gdA7.pgp Description: PGP signature ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. -- Maurits van Rees | http://maurits.vanrees.org/ Work | http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
[Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
On Feb 28, 10:59 am, Maurits van Rees [EMAIL PROTECTED] wrote: greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. Can't that be wrapped in a try/except? The most common case is that the index will have the attributes that GS is trying to set. As for type, then again, given the prevalence of actions in GS that modify an assumed current state - then a flag 'recreate' would allow for a type change to be pushed through. -- Maurits van Rees |http://maurits.vanrees.org/ Work |http://zestsoftware.nl/ This is your day, don't let them take it away. [Barlow Girl] ___ Zope-CMF maillist - [EMAIL PROTECTED]://mail.zope.org/mailman/listinfo/zope-cmf Seehttp://collector.zope.org/CMFfor bug reports and feature requests ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
Re: [Zope-CMF] Re: What is the status of GS wiping catalog indexes on catalog.xml import?
--On 27. Februar 2008 21:59:58 + Maurits van Rees [EMAIL PROTECTED] wrote: greenman, on 2008-02-27: So, for the catalog.xml importer, why can't the trigger for reindexing an index be a flag on the catalog index declaration itself? Is it really generic setups role to determine if changes to an index invalidate the values it already holds? If you were to change properties of an index through code and not GS, then it would be up to you whether you reindexed all your objects or not. The problem is that GenericSetup does not know if your current index is of the same type and has the same settings/properties as the index that you specify in catalog.xml. Apparently it is hard/impossible to reliably compare the existing and the wanted index. So GenericSetup has no choice but to remove the existing index and make a new one. How about the following idea: - within the Zope core we define an _optional_ interface for indexes - something like: class IIndexConfiguration(Interface): def getConfiguration(): Returns a dict with index specific configuration parameters. - on the CMF/GS side we could register adapter for each index type we know (basically the Zope 2 core indexes, ExtendedPathIndex, TextIndexNG 3) and retrieve the related information - the related GS asks each index for its configuration and takes appropriate action based on the comparison of the values from the profile and the existing index. Adding the interface to Zope 2.11 or backporting it to Zope 2.10 would not be a problem. Since the Zope 2.11 branch is offically closed for new features, the index specific implementations of IIndexConfiguration should be implemented outside the Zope core but we might move the implementation into the Zope core with Zope 2.12. Sounds reasonable? Andreas pgpA8OQK4gGQ3.pgp Description: PGP signature ___ Zope-CMF maillist - Zope-CMF@lists.zope.org http://mail.zope.org/mailman/listinfo/zope-cmf See http://collector.zope.org/CMF for bug reports and feature requests
