Re: [ZODB-Dev] Proposal (version 2): cross database reference seat belt
Hi, On Tue, 2009-04-28 at 13:54 -0400, Jim Fulton wrote: Thanks again! (Note to everyone else, Shane and I discussed this on IRC, along with another alternative that I'll mention below.) I like version 2 better than version 1. I'd be inclined to simplify and it and skip the configuration flag and simply publish an event any time we see a cross-database reference when saving an object. Here's proposed solution 3. :) - We add a flag to disable new cross-database references unless they are explicitly registered. - We add a connection method to register a reference: def registerCrossDatabaseReference(from_, to): Register a new cross-database reference from from_ to to. - We arrange that connections can recognize old cross-database references. If someone accidentally creates a new reference and the flag is set, then transaction will be aborted. An interim step, if we're in a hurry to get 3.9 out, is to simply add the flag. This would disallow cross-database references in new applications. These applications could still support multiple databases by providing application-level traversal across databases. I think I'm reading something incorrectly: is there an emphasis on *new* applications? The flag would disallow the creation of cross-database references for a given DB -- independent of whether the app is new or old, right? Only depending on whether the application uses a ZODB that has the feature and has it enabled. Right? Otherwise, I'm +1. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 7 · fax +49 345 1229889 1 Zope and Plone consulting and development signature.asc Description: This is a digitally signed message part ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal (version 2): cross database reference seat belt
Christian Theune wrote: Hi, On Tue, 2009-04-28 at 13:54 -0400, Jim Fulton wrote: Thanks again! (Note to everyone else, Shane and I discussed this on IRC, along with another alternative that I'll mention below.) I like version 2 better than version 1. I'd be inclined to simplify and it and skip the configuration flag and simply publish an event any time we see a cross-database reference when saving an object. Here's proposed solution 3. :) - We add a flag to disable new cross-database references unless they are explicitly registered. - We add a connection method to register a reference: def registerCrossDatabaseReference(from_, to): Register a new cross-database reference from from_ to to. - We arrange that connections can recognize old cross-database references. If someone accidentally creates a new reference and the flag is set, then transaction will be aborted. An interim step, if we're in a hurry to get 3.9 out, is to simply add the flag. This would disallow cross-database references in new applications. These applications could still support multiple databases by providing application-level traversal across databases. I think I'm reading something incorrectly: is there an emphasis on *new* applications? The flag would disallow the creation of cross-database references for a given DB -- independent of whether the app is new or old, right? Only depending on whether the application uses a ZODB that has the feature and has it enabled. Right? I think the emphasis was on new versus existing cross-database references. Laurence ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal (version 2): cross database reference seat belt
On Apr 30, 2009, at 5:01 PM, Christian Theune wrote: Hi, On Tue, 2009-04-28 at 13:54 -0400, Jim Fulton wrote: Thanks again! (Note to everyone else, Shane and I discussed this on IRC, along with another alternative that I'll mention below.) I like version 2 better than version 1. I'd be inclined to simplify and it and skip the configuration flag and simply publish an event any time we see a cross-database reference when saving an object. Here's proposed solution 3. :) - We add a flag to disable new cross-database references unless they are explicitly registered. - We add a connection method to register a reference: def registerCrossDatabaseReference(from_, to): Register a new cross-database reference from from_ to to. - We arrange that connections can recognize old cross-database references. If someone accidentally creates a new reference and the flag is set, then transaction will be aborted. An interim step, if we're in a hurry to get 3.9 out, is to simply add the flag. This would disallow cross-database references in new applications. These applications could still support multiple databases by providing application-level traversal across databases. I think I'm reading something incorrectly: is there an emphasis on *new* applications? No The flag would disallow the creation of cross-database references for a given DB -- independent of whether the app is new or old, right? Right Only depending on whether the application uses a ZODB that has the feature and has it enabled. Right? This is not a sentence. :) The flag would disable implicit references (new or old). When an object is being saved, if it has implicit cross-references, the save will fail. The second and third bullets above provide a way to make explicit references. I've implemented the first bullet. There's a question of whether I need to implement the second 2 bullets for ZODB 3.9. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Proposal (version 2): cross database reference seat belt
Problem --- In multi-database configurations, ZODB applications can unintentionally create cross-database references. The causes include moving (rather than copying) an object between containers, storing an object in a session database and later moving it to the main database, and using a persistent object for a catalog index when the catalog is located in another database. Unintentional cross-database references can cause significant problems. For example, references from non-volatile objects to volatile session objects will break when the session expires, leading to application errors. In a project I am working on, my team decided that configuring our application to use a multi-database was too risky unless we had some way to prevent unintentional cross-database references. Proposed Solution - I propose an optional seat belt for cross-database references in ZODB. The seat belt, when enabled, gives applications an opportunity to veto the creation of cross-database references. Application policy will specify which cross-database references to allow. Proposed Mechanism -- The ZODB.DB.DB constructor will accept a new parameter, check_xrefs, that defaults to False. When check_xrefs is True, the cross-database reference seat belt is enabled. xref is short for cross-database reference. When the cross-database reference seat belt is enabled for a database, ZODB publishes events (using the zope.event package) every time an object that contains cross-database references changes. Event subscribers have the opportunity to veto the cross reference by generating an exception. The published event will conform to the following interface. class ICrossDatabaseReferenceEvent(Interface): source = Attribute(The object holding the reference) target = Attribute(The target of the reference) Each database in a multi-database has its own check_xrefs setting. The setting applies only to the objects contained in that database. This allows developers to specify, for example, that arbitrary references from the main database to the volatile session database are disallowed, while arbitrary references from the volatile session database to the main database are allowed. A possible risk of the proposed mechanism is that an event subscriber could wake up ghosts as a side effect. Implementers probably need to avoid that. Conclusion -- The proposed new feature is designed to help developers create more robust ZODB applications by enforcing cross-database reference policy. I hope it can be included in ZODB 3.9. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal (version 2): cross database reference seat belt
Thanks again! (Note to everyone else, Shane and I discussed this on IRC, along with another alternative that I'll mention below.) I like version 2 better than version 1. I'd be inclined to simplify and it and skip the configuration flag and simply publish an event any time we see a cross-database reference when saving an object. Here's proposed solution 3. :) - We add a flag to disable new cross-database references unless they are explicitly registered. - We add a connection method to register a reference: def registerCrossDatabaseReference(from_, to): Register a new cross-database reference from from_ to to. - We arrange that connections can recognize old cross-database references. If someone accidentally creates a new reference and the flag is set, then transaction will be aborted. An interim step, if we're in a hurry to get 3.9 out, is to simply add the flag. This would disallow cross-database references in new applications. These applications could still support multiple databases by providing application-level traversal across databases. Thoughts? If there are no objections, I'll pursue version 3. How soon do we want 3.9 beta? Jim On Apr 28, 2009, at 1:31 PM, Shane Hathaway wrote: Problem --- In multi-database configurations, ZODB applications can unintentionally create cross-database references. The causes include moving (rather than copying) an object between containers, storing an object in a session database and later moving it to the main database, and using a persistent object for a catalog index when the catalog is located in another database. Unintentional cross-database references can cause significant problems. For example, references from non-volatile objects to volatile session objects will break when the session expires, leading to application errors. In a project I am working on, my team decided that configuring our application to use a multi-database was too risky unless we had some way to prevent unintentional cross-database references. Proposed Solution - I propose an optional seat belt for cross-database references in ZODB. The seat belt, when enabled, gives applications an opportunity to veto the creation of cross-database references. Application policy will specify which cross-database references to allow. Proposed Mechanism -- The ZODB.DB.DB constructor will accept a new parameter, check_xrefs, that defaults to False. When check_xrefs is True, the cross-database reference seat belt is enabled. xref is short for cross-database reference. When the cross-database reference seat belt is enabled for a database, ZODB publishes events (using the zope.event package) every time an object that contains cross-database references changes. Event subscribers have the opportunity to veto the cross reference by generating an exception. The published event will conform to the following interface. class ICrossDatabaseReferenceEvent(Interface): source = Attribute(The object holding the reference) target = Attribute(The target of the reference) Each database in a multi-database has its own check_xrefs setting. The setting applies only to the objects contained in that database. This allows developers to specify, for example, that arbitrary references from the main database to the volatile session database are disallowed, while arbitrary references from the volatile session database to the main database are allowed. A possible risk of the proposed mechanism is that an event subscriber could wake up ghosts as a side effect. Implementers probably need to avoid that. Conclusion -- The proposed new feature is designed to help developers create more robust ZODB applications by enforcing cross-database reference policy. I hope it can be included in ZODB 3.9. Shane -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal (version 2): cross database reference seat belt
On 4/28/09 1:54 PM, Jim Fulton wrote: An interim step, if we're in a hurry to get 3.9 out, is to simply add the flag. This would disallow cross-database references in new applications. These applications could still support multiple databases by providing application-level traversal across databases. +1. It seems good to start simple like this. - C ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
