[ZODB-Dev] Using a storage that is also handled by StorageServer
Hi, in ZEORaid we're using the storage (a ZEORaid instance) as it's being by ZEO for recovering data from the good storages into one we're recovering. For finishing the recovery exactly on transaction boundaries and to avoid transactions slipping in when we think we're done we acquire the commit lock for a short period of time. If a client tries to write in that period of time the transaction will be put into the StorageServer's waiting list and sit there either indefinitely or until another client comes around calling abort() or finish(). So this is basically an issue because StorageServer assumes that it or another instance of StorageServer is ever talking to that storage, so it implements a secret protocol (the waiting list) which is not officially visible to other code. I can hack around this for now by making the waiting list visible to my code and notify the waiting storages from there, but, I'd like to propose an extension to the storage API. We basically need the ability to register callbacks with a storage that will be called on transaction boundaries. With that we can redo the current _restart implementation so that it can be triggered from code that doesn't need to know that a storage is being wrapped in such a way. If you think we can go down that route in general, then I'll prepare a more detailed proposal including making a draft implementation. Christian -- Christian Theune · c...@gocept.com gocept gmbh co. kg · forsterstraße 29 · 06112 halle (saale) · germany http://gocept.com · tel +49 345 1229889 0 · fax +49 345 1229889 1 Zope and Plone consulting and development ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Using a storage that is also handled by StorageServer
On Thu, Nov 12, 2009 at 1:34 AM, Christian Theune c...@gocept.com wrote: Hi, in ZEORaid we're using the storage (a ZEORaid instance) as it's being by ZEO for recovering data from the good storages into one we're recovering. For finishing the recovery exactly on transaction boundaries and to avoid transactions slipping in when we think we're done we acquire the commit lock for a short period of time. If a client tries to write in that period of time the transaction will be put into the StorageServer's waiting list and sit there either indefinitely or until another client comes around calling abort() or finish(). So this is basically an issue because StorageServer assumes that it or another instance of StorageServer is ever talking to that storage, so it implements a secret protocol (the waiting list) which is not officially visible to other code. I can hack around this for now by making the waiting list visible to my code and notify the waiting storages from there, but, I'd like to propose an extension to the storage API. We basically need the ability to register callbacks with a storage that will be called on transaction boundaries. With that we can redo the current _restart implementation so that it can be triggered from code that doesn't need to know that a storage is being wrapped in such a way. If you think we can go down that route in general, then I'll prepare a more detailed proposal including making a draft implementation. I don't understand what you are trying to say above. Perhaps a more detailed proposal will make it clearer. BTW, I've refactored the waiting list code quite a bit on the jim-thready-zeo2 branch, which I plan to merge to trunk soon. Jim -- Jim Fulton ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
