Re: [ZODB-Dev] RelStorage breaks history tab - solution?
Hi Shane, On 23/05/2012 16:27, Chris Withers wrote: Okay, the issue appears to be that, in some circumstances, RelStorage is leaving the read connection with an open transaction that isn't rolled back. Upon investigation, this turns out to be the history method itself! :-) Switching to the following patch also solves the problem: diff --git a/relstorage/storage.py b/relstorage/storage.py index 2d592d3..3fa792b 100644 --- a/relstorage/storage.py +++ b/relstorage/storage.py @@ -1053,6 +1053,7 @@ class RelStorage( break return res finally: +self._load_conn.rollback() self._lock_release() def undo(self, transaction_id, transaction): However, that raised more questions for me: - How does RelStorage usually rollback the _load_conn in a non-write transaction? - Why doesn't this kick in when viewing the history method? My guess as to why ZODB.tests.HistoryStorage:HistoryStorage.checkSimpleHistory passes is that everything is done with the same thread/connection, and so a consistent view of the data is seen. More confusing: having dropped the zserver-threads down to one and from looking at the results of: select * from pg_catalog.pg_stat_activity where datname='myrelstorage'; ...it appears that there's: - one connection for reads - one connection for writes, which is opened when first used. Okay, as expected. - one connection for viewing history. I can't see where/why this final connection is opened... Shane, little help? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage breaks history tab
On 05/23/2012 09:27 AM, Chris Withers wrote: Okay, the issue appears to be that, in some circumstances, RelStorage is leaving the read connection with an open transaction that isn't rolled back. That is what RelStorage is designed to do when you set poll-interval. Does the bug go away when you set poll-interval to 0? Shane ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage breaks history tab
Hi Shane, On 22/05/2012 06:37, Chris Withers wrote: So, the problem is that, with a Zope 2 app server, any revisions made to an object (say, a Page Template) don't show in the history tab of that object *until* the app server is restarted. Okay, the issue appears to be that, in some circumstances, RelStorage is leaving the read connection with an open transaction that isn't rolled back. I couldn't find the source of this, but in the meantime, I less heavyweight hack is: diff --git a/relstorage/storage.py b/relstorage/storage.py index 2d592d3..459469b 100644 --- a/relstorage/storage.py +++ b/relstorage/storage.py @@ -1025,6 +1025,7 @@ class RelStorage( self._lock_acquire() try: self._before_load() +self._load_conn.rollback() cursor = self._load_cursor oid_int = u64(oid) try: However, that open transaction is likely to cause other problems (I'm fairly certain I've seen other effects of this) so do you have any idea where to look for it? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage breaks history tab
Hi All, I've been chatting with Shane about this off-list but wanted to see if anyone else has experience of this and has found a solution. So, the problem is that, with a Zope 2 app server, any revisions made to an object (say, a Page Template) don't show in the history tab of that object *until* the app server is restarted. I've verified this on both MySQL and Postgres with trunk of RelStorage as well as 1.5.2 and 1.4.2. It appears to be some kind of cursor re-use / relational database transaction isolation bug given that the following hack fixes the problem: --- a/relstorage/storage.py +++ b/relstorage/storage.py @@ -1025,7 +1025,7 @@ class RelStorage( self._lock_acquire() try: self._before_load() -cursor = self._load_cursor +conn, cursor = self._adapter.connmanager.open() oid_int = u64(oid) try: rows = self._adapter.dbiter.iter_object_history( @@ -1053,6 +1053,7 @@ class RelStorage( break return res finally: +self._adapter.connmanager.close(conn, cursor) self._lock_release() It seems a little heavyweight to have to open a whole new connection each time someone views a history tab, though :-/ A couple of surprising things: - ZODB.tests.HistoryStorage:HistoryStorage.checkSimpleHistory passes. I'd expect this to fail, so I wonder what it's doing differently? - The read connection is obviously getting the transaction's data, otherwise the wrong version of the object would be rendered, right? Any ideas very gratefully received... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage breaks History tab
Le vendredi 2 septembre 2011 09:26:30, Martijn Pieters a écrit : > Without looking, I'd say the history tab relies on the internal > structure of the stock FileStorage implementation, not on public ZODB > APIs. AFAICS OFS/History.py only relies on tids being 64bits integers, the rest looks standard API to me (or at least, Connection-level API, not storage- level). ...at least, we could get it to work in NEO ;) (easier than undo from performance point of view, but that's another story). -- Vincent Pelletier ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage breaks History tab
On 02/09/2011 08:26, Martijn Pieters wrote: > On Thu, Sep 1, 2011 at 22:56, Chris Withers wrote: >> I see the resulting transactions in both the root Undo tab and the Undo >> tab of the page template, but not in the History tab of the page template. > > Without looking, I'd say the history tab relies on the internal > structure of the stock FileStorage implementation, not on public ZODB > APIs. I haven't touched the archetypes History code in 5 years at > least though, so I am not sure about this. Nope, RelStorage has specific history hooks in dbiter.py, just doesn't look like they work for me ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] RelStorage breaks History tab
On Thu, Sep 1, 2011 at 22:56, Chris Withers wrote: > I see the resulting transactions in both the root Undo tab and the Undo > tab of the page template, but not in the History tab of the page template. Without looking, I'd say the history tab relies on the internal structure of the stock FileStorage implementation, not on public ZODB APIs. I haven't touched the archetypes History code in 5 years at least though, so I am not sure about this. -- Martijn Pieters ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] RelStorage breaks History tab
Hi Shane, It looks like RelStorage (1.4.2 in my case, connected to MySQL, ZODB 3.9.6 on Zope 2.12.7) breaks the history tab. To reproduce, I set up a stock ZODB with a history-preserving storage, went in and edited the index_html page a few times. I see the resulting transactions in both the root Undo tab and the Undo tab of the page template, but not in the History tab of the page template. Any ideas? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
