Re: [ZODB-Dev] Proposal: cross database reference seat belt
On Apr 28, 2009, at 11:36 AM, Shane Hathaway wrote: > Jim Fulton wrote: >> Thanks for writing this up! >> Do we really need per-object control? >> It would be simpler to simply allow or disallow (new) cross-data >> references at the database level. >> I think the most common use case for cross-database connection >> limitation is support for "mounts". With mounts, there are a >> small number of direct references between databases. This could >> be addressed by instantiating a database in a mode that allows >> cross- database references and using that instantiation to >> establish the mounts. All other instantiations would disallow new >> cross-database references. >> Do you have a specific use case that requires _p_check_xref? > > No. Good point. > > I am hoping that even after we configure the system to disallow new > cross-database references, we can still make other modifications to > the objects that contain cross-database references. Right. > For example, if I have a site object containing a catalog in > another database, I should still be able to modify the title > attribute of the site, as long as I don't try to change the catalog. Yup, although now that you mention it, I fear it will be hard to make this work. :) I need to think about it. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal: cross database reference seat belt
Jim Fulton wrote: > Thanks for writing this up! > > Do we really need per-object control? > > It would be simpler to simply allow or disallow (new) cross-data > references at the database level. > > I think the most common use case for cross-database connection > limitation is support for "mounts". With mounts, there are a small > number of direct references between databases. This could be > addressed by instantiating a database in a mode that allows cross- > database references and using that instantiation to establish the > mounts. All other instantiations would disallow new cross-database > references. > > Do you have a specific use case that requires _p_check_xref? No. Good point. I am hoping that even after we configure the system to disallow new cross-database references, we can still make other modifications to the objects that contain cross-database references. For example, if I have a site object containing a catalog in another database, I should still be able to modify the title attribute of the site, as long as I don't try to change the catalog. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Proposal: cross database reference seat belt
Thanks for writing this up! Do we really need per-object control? It would be simpler to simply allow or disallow (new) cross-data references at the database level. I think the most common use case for cross-database connection limitation is support for "mounts". With mounts, there are a small number of direct references between databases. This could be addressed by instantiating a database in a mode that allows cross- database references and using that instantiation to establish the mounts. All other instantiations would disallow new cross-database references. Do you have a specific use case that requires _p_check_xref? Jim On Apr 28, 2009, at 5:31 AM, Shane Hathaway wrote: > Problem > --- > > In multi-database configurations, ZODB applications can > unintentionally create cross-database references. The causes > include moving (rather than copying) an object between containers, > storing an object in a session database and later moving it to the > main database, and using a persistent object for a catalog index > when the catalog is located in another database. > > Unintentional cross-database references can cause significant > problems. For example, references from non-volatile objects to > volatile session objects will break when the session expires, > leading to application errors. > > In a project I am working on, my team decided that configuring our > application to use a multi-database was too risky unless we had some > way to prevent unintentional cross-database references. > > Proposed Solution > - > > I propose an optional "seat belt" for cross-database references in > ZODB. The seat belt, when enabled, will prevent most objects from > holding any cross-database references. Application policy will > specify which cross-database references to allow. When any cross- > database reference violates the policy, ZODB will raise an exception > to help application developers track down the policy violation. > > Proposed Mechanism > -- > > The ZODB.DB.DB constructor will accept a new parameter, > "check_xrefs", that defaults to False. When check_xrefs is True, > the cross-database reference seat belt is enabled. "xref" is short > for "cross-database reference". > > Applications will express cross-database reference policy through a > new method of persistent objects named "_p_check_xref". The > _p_check_xref method accepts a single parameter, the object to be > referenced in another database. If the reference should be allowed, > the _p_check_xref method returns True. If the _p_check_xref method > returns False, the object serialization machinery will raise an > InvalidObjectReference exception, leading to transaction abort. > > Furthermore, when the seat belt is enabled, ZODB will raise an > InvalidObjectReference exception when an object with no > _p_check_xref method attempts to hold a cross-database reference. > > Each database in a multi-database has its own check_xrefs setting. > The setting applies only to the objects contained in that database. > This allows developers to specify, for example, that arbitrary > references from the main database to the volatile session database > are disallowed, while arbitrary references from the volatile session > database to the main database are allowed. > > The proposed mechanism has been implemented in a branch of ZODB > named "shane-cross-database-seatbelt", checked in at svn.zope.org. > It does not change very many lines of code. > > Some possible risks of the proposed mechanism: > > * An implementation of the _p_check_xref method could wake up ghosts > as a side effect. Implementers probably need to avoid that. > > * People might want to express the application policy without > modifying persistent classes. I chose not to propose that kind of > solution, opting instead for a solution that follows established > ZODB patterns. > > Conclusion > -- > > The proposed new feature is designed to help developers create more > robust ZODB applications by enforcing cross-database reference > policy. I hope it can be included in ZODB 3.9. > > Shane > -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Proposal: cross database reference seat belt
Problem --- In multi-database configurations, ZODB applications can unintentionally create cross-database references. The causes include moving (rather than copying) an object between containers, storing an object in a session database and later moving it to the main database, and using a persistent object for a catalog index when the catalog is located in another database. Unintentional cross-database references can cause significant problems. For example, references from non-volatile objects to volatile session objects will break when the session expires, leading to application errors. In a project I am working on, my team decided that configuring our application to use a multi-database was too risky unless we had some way to prevent unintentional cross-database references. Proposed Solution - I propose an optional "seat belt" for cross-database references in ZODB. The seat belt, when enabled, will prevent most objects from holding any cross-database references. Application policy will specify which cross-database references to allow. When any cross-database reference violates the policy, ZODB will raise an exception to help application developers track down the policy violation. Proposed Mechanism -- The ZODB.DB.DB constructor will accept a new parameter, "check_xrefs", that defaults to False. When check_xrefs is True, the cross-database reference seat belt is enabled. "xref" is short for "cross-database reference". Applications will express cross-database reference policy through a new method of persistent objects named "_p_check_xref". The _p_check_xref method accepts a single parameter, the object to be referenced in another database. If the reference should be allowed, the _p_check_xref method returns True. If the _p_check_xref method returns False, the object serialization machinery will raise an InvalidObjectReference exception, leading to transaction abort. Furthermore, when the seat belt is enabled, ZODB will raise an InvalidObjectReference exception when an object with no _p_check_xref method attempts to hold a cross-database reference. Each database in a multi-database has its own check_xrefs setting. The setting applies only to the objects contained in that database. This allows developers to specify, for example, that arbitrary references from the main database to the volatile session database are disallowed, while arbitrary references from the volatile session database to the main database are allowed. The proposed mechanism has been implemented in a branch of ZODB named "shane-cross-database-seatbelt", checked in at svn.zope.org. It does not change very many lines of code. Some possible risks of the proposed mechanism: * An implementation of the _p_check_xref method could wake up ghosts as a side effect. Implementers probably need to avoid that. * People might want to express the application policy without modifying persistent classes. I chose not to propose that kind of solution, opting instead for a solution that follows established ZODB patterns. Conclusion -- The proposed new feature is designed to help developers create more robust ZODB applications by enforcing cross-database reference policy. I hope it can be included in ZODB 3.9. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
