Re: [Zope-dev] many conflict errors
Answering to myself: we could get rid of the sessionrelated conflict errors, by removing all session = context.REQUEST.SESSION from our scripts. But we still have these conflict errors: how can I find out, what causes these conflicts. 2007-05-25T10:17:50 INFO ZPublisher.Conflict ConflictError at /VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/U431409/study_course/create_level: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2a44454dee 2007-05-25 09:14:16.000950, serial currently committed 0x036ddc2c21950377 2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup at Fri May 25 05:19:08 2007) -- and finally: -- 2007-05-25T10:19:01 ERROR Zope.SiteErrorLog http://uniben.waeup.org/campus/students/C172676/study_course/create_level Traceback (innermost last): Module Zope2.App.startup, line 173, in zpublisher_exception_hook Module ZPublisher.Publish, line 121, in publish Module Zope2.App.startup, line 240, in commit Module transaction._manager, line 96, in commit Module Products.CPSCompat.PatchZODBTransaction, line 175, in commit Module transaction._transaction, line 436, in _commitResources Module ZODB.Connection, line 665, in tpc_vote Module ZEO.ClientStorage, line 893, in tpc_vote Module ZEO.ClientStorage, line 877, in _check_serials ConflictError: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2b3e989fdd 2007-05-25 09:15:14.670982, serial currently committed 0x036ddc2dd48f4e33 2007-05-25 09:17:49.818700) -- Joachim Schmitz schrieb: Hi, in our Studentregistration portal we experience many conflict errors under heavy load many of them unresolved. the configuration ZEO-server running on one machine 4 Zope-Clients with CPS-3.4.2 with 4 threads each running on a second box, we are using the faster session product all running Zope 2.9.3, python 2.4.3 on Ubuntu 4.0.3 a typical case looks like this: -- Gruß Joachim ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Zope Tests: 5 OK
Summary of messages to the zope-tests list. Period Thu May 24 12:00:00 2007 UTC to Fri May 25 12:00:00 2007 UTC. There were 5 messages: 5 from Zope Unit Tests. Tests passed OK --- Subject: OK : Zope-2.7 Python-2.3.6 : Linux From: Zope Unit Tests Date: Thu May 24 20:59:43 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-May/007779.html Subject: OK : Zope-2.8 Python-2.3.6 : Linux From: Zope Unit Tests Date: Thu May 24 21:01:27 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-May/007780.html Subject: OK : Zope-2.9 Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu May 24 21:03:12 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-May/007781.html Subject: OK : Zope-2.10 Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu May 24 21:04:55 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-May/007782.html Subject: OK : Zope-trunk Python-2.4.4 : Linux From: Zope Unit Tests Date: Thu May 24 21:06:32 EDT 2007 URL: http://mail.zope.org/pipermail/zope-tests/2007-May/007783.html ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] many conflict errors
Perry napisał(a): Answering to myself: we could get rid of the sessionrelated conflict errors, by removing all session = context.REQUEST.SESSION from our scripts. But we still have these conflict errors: how can I find out, what causes these conflicts. 2007-05-25T10:17:50 INFO ZPublisher.Conflict ConflictError at /VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/U431409/study_course/create_level: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2a44454dee 2007-05-25 09:14:16.000950, serial currently committed 0x036ddc2c21950377 2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup at Fri May 25 05:19:08 2007) Try to change session resolution seconds (Timeout resolution (in seconds)) parameter to a larger value. Possibly you're ecountering same problem as I had some time ago. With long requests and small session resolution seconds I had a lot of conflict errors. I think it is because of way sessions work in zope. Every x seconds (where x is 'session resolution seconds'), when there is access to SESSION, session objects are moved in OOBTrees. It is used to handle session expiration. Not used session objects are moved to the end of the 'list' and finally removed after Session Timeout is reached. I don't know how this exactly goes but it is something like that. When this 'movement' happens during request (session object is moved when request that is bound to it is still being served) this causes conflicts. You may look at group archives for my posts month or two months ago... It is easy to reproduce these errors. -- Maciej Wisniowski ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] many conflict errors
Maciej Wisniowski schrieb: Perry napisał(a): Answering to myself: we could get rid of the sessionrelated conflict errors, by removing all session = context.REQUEST.SESSION from our scripts. But we still have these conflict errors: how can I find out, what causes these conflicts. 2007-05-25T10:17:50 INFO ZPublisher.Conflict ConflictError at /VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/U431409/study_course/create_level: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2a44454dee 2007-05-25 09:14:16.000950, serial currently committed 0x036ddc2c21950377 2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup at Fri May 25 05:19:08 2007) Try to change session resolution seconds (Timeout resolution (in seconds)) parameter to a larger value. Possibly you're ecountering same problem as I had some time ago. With long requests and small session resolution seconds I had a lot of conflict errors. I set it now to 9600/4800 (Timeout/Resolution) but it seams that it has no effect. I think it is because of way sessions work in zope. Every x seconds (where x is 'session resolution seconds'), when there is access to SESSION, session objects are moved in OOBTrees. It is used to handle session expiration. Not used session objects are moved to the end of the 'list' and finally removed after Session Timeout is reached. I don't know how this exactly goes but it is something like that. When this 'movement' happens during request (session object is moved when request that is bound to it is still being served) this causes conflicts. You may look at group archives for my posts month or two months ago... I read this already I am also using the faster-session product and patched it to use the AppendOnlyDict. -- Gruß Joachim ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] many conflict errors
I set it now to 9600/4800 (Timeout/Resolution) but it seams that it has no effect. Hm... where did you set this? At faster's preferences page in ZMI? can you verify that these settings are used? To change timeout/resolution to high values for standard session implementation in zope.conf I had to patch one of zope core files, otherwise it was not possible to start zope. One more thing may be that these errors appears because of large number of newly created session objects, eg. after zope restart. I mean that when you restart zope and then run 10 concurrent requests 10 new session objects will be created. This may (and possibly do) cause conflicts as these are all changes to OOBTree. When you run these 10 concurrent requests again from the same browsers (without restarting zope) then objects are already created so only session resolution may be taken into account I think. How do you check this? One more thing I wonder about is what number of threads your zope instance is running? I read this already I am also using the faster-session product and patched it to use the AppendOnlyDict. In this case it had no effect for me too. -- Maciej Wisniowski ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] many conflict errors
Maciej Wisniowski schrieb: I set it now to 9600/4800 (Timeout/Resolution) but it seams that it has no effect. Hm... where did you set this? At faster's preferences page in ZMI? Yes can you verify that these settings are used? don't know how. To change timeout/resolution to high values for standard session implementation in zope.conf I had to patch one of zope core files, otherwise it was not possible to start zope. One more thing may be that these errors appears because of large number of newly created session objects, eg. after zope restart. I mean that when you restart zope and then run 10 concurrent requests 10 new session objects will be created. This may (and possibly do) cause conflicts as these are all changes to OOBTree. When you run these 10 concurrent requests again from the same browsers (without restarting zope) then objects are already created so only session resolution may be taken into account I think. How do you check this? One more thing I wonder about is what number of threads your zope instance is running? we have 4 Zeo-Clients with 4 threads each accessing one ZEO-Server on a different machine. -- Gruß Joachim ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: many conflict errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Perry wrote: Answering to myself: we could get rid of the sessionrelated conflict errors, by removing all session = context.REQUEST.SESSION from our scripts. But we still have these conflict errors: how can I find out, what causes these conflicts. 2007-05-25T10:17:50 INFO ZPublisher.Conflict ConflictError at /VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/U431409/study_course/create_level: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2a44454dee 2007-05-25 09:14:16.000950, serial currently committed 0x036ddc2c21950377 2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup at Fri May 25 05:19:08 2007) -- and finally: -- 2007-05-25T10:19:01 ERROR Zope.SiteErrorLog http://uniben.waeup.org/campus/students/C172676/study_course/create_level Traceback (innermost last): Module Zope2.App.startup, line 173, in zpublisher_exception_hook Module ZPublisher.Publish, line 121, in publish Module Zope2.App.startup, line 240, in commit Module transaction._manager, line 96, in commit Module Products.CPSCompat.PatchZODBTransaction, line 175, in commit Module transaction._transaction, line 436, in _commitResources Module ZODB.Connection, line 665, in tpc_vote Module ZEO.ClientStorage, line 893, in tpc_vote Module ZEO.ClientStorage, line 877, in _check_serials ConflictError: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2b3e989fdd 2007-05-25 09:15:14.670982, serial currently committed 0x036ddc2dd48f4e33 2007-05-25 09:17:49.818700) Neither of those tracebacks look like they come out of the 'faster' session machinery itself; it uses an 'OOBTree' by default for storing the slices, not an 'IOBtree'. The primary consumer of 'IOBTree' is probably the catalog, where it is used to map RID - values in the 'unindex' of 'FieldIndex', etc. The original sessioning machinery uses 'IOBTree', too: are you sure you are using the faster machnery? Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGVyLe+gerLs4ltQ4RAoGOAJ96Hl1UkY+mbeaik2uaNszWQgh4oACfcZe6 mDWlIPGApco0/yrlv1SxOYs= =hejZ -END PGP SIGNATURE- ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Bug] ZODB invalidation processing (was: [Zope-dev] many conflict errors)
Perry wrote at 2007-5-25 13:16 +0200: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2a44454dee 2007-05-25 09:14:16.000950, serial currently committed 0x036ddc2c21950377 2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup at Fri May 25 05:19:08 2007) ... ConflictError: database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket, serial this txn started with 0x036ddc2b3e989fdd 2007-05-25 09:15:14.670982, serial currently committed 0x036ddc2dd48f4e33 2007-05-25 09:17:49.818700) These log entries indicate a bug in ZODB's invalidation processing. The first entry tells us that the object was read at 9:14:16 and the modification conflicts with a write from 9:16:07. The second entry tells us that the object was read at 9:15:14 *BUT* at the time this transaction has started, the ZODB should already have known the modification from 9:16:07 and the object read at 9:15:14 should have been invalidated. The new transaction should not have seen any state older than 9:16:07 (as it begins after this time). In older ZODB versions, there has been a bug in ZODB.Connection.Connection._setDB. It has forgotten to flush invalidations (which may lead to observations as the above). In our private Zope version, I have still a note like this: # DM 2005-08-22: always call '_flush_invalidations' as it does # more than cache handling only self._flush_invalidations() if self._reset_counter != global_reset_counter: # New code is in place. Start a new cache. self._resetCache() # DM 2005-08-22: always call '_flush_invalidations' ##else: ##self._flush_invalidations() The note indicates that the bug was fixed at least at 2005-08-22 (though the handling was not completely right in case the cache was reset). Maybe, the bug you see now affects only mounted connections? -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
