[Zope] strange database read conflict error
Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] strange database read conflict error
Peter, Than you for your response. Please find below a few things: - I give up using sessions in the website's front and years ago :) There is only one page left that is using them, when u subscribe to our newsletter but I don't think that this affects because is used rarely - I am using Yahoo Maps Ajax API to display maps; also some ajax calls for our build in chat (only for logged users) So if the things above are causing read conflicts why only for this particular object? There is nothing special or different with this one... Thanks again, Dragos On Fri, Aug 28, 2009 at 12:19 PM, Peter Bengtssonpete...@gmail.com wrote: The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] strange database read conflict error
The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] strange database read conflict error
2009/8/28 Dragos Chirila objectval...@gmail.com: Peter, Than you for your response. Please find below a few things: - I give up using sessions in the website's front and years ago :) There is only one page left that is using them, when u subscribe to our newsletter but I don't think that this affects because is used rarely It might be used anyway. It'll be empty (except the subscribe-to-newsletter page) all the time but it might still be in use. I think there is always a REQUEST.SESSION object. - I am using Yahoo Maps Ajax API to display maps; also some ajax calls for our build in chat (only for logged users) That's unrelated. Yahoo! isn't using ZODB. So if the things above are causing read conflicts why only for this particular object? There is nothing special or different with this one... It could be some other piece of code that tries to read from ZODB more than once and whatever it's trying to read might be something slow (slow for a computer). Have you tried running ZopeProfiler on your site to figure out if there are certain (ZO)DB related functions that are called either many many times or few but slow times. Thanks again, Dragos On Fri, Aug 28, 2009 at 12:19 PM, Peter Bengtssonpete...@gmail.com wrote: The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] strange database read conflict error
All objects as the same type of this one that generates conflicts are displayed using the same template that contains the same elements (js, images, etc.). I can access any object in the same folder (http://metropotam.ro/Opera/) and I will get no read conflicts (e.g. http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/). I have conflicts when I access this particular object http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/. If the problem is related with some piece of code I can't find any logical explanation for this behavior. ... or maybe I am looking in the wrong place. Also, told you before, I even erased the object from the Data.fs and then packed it. Conflicts stopped. After I have created a new object and rename it with the old id the conflicts showed up again... Thanks, Dragos On Fri, Aug 28, 2009 at 2:37 PM, Peter Bengtssonpete...@gmail.com wrote: 2009/8/28 Dragos Chirila objectval...@gmail.com: Peter, Than you for your response. Please find below a few things: - I give up using sessions in the website's front and years ago :) There is only one page left that is using them, when u subscribe to our newsletter but I don't think that this affects because is used rarely It might be used anyway. It'll be empty (except the subscribe-to-newsletter page) all the time but it might still be in use. I think there is always a REQUEST.SESSION object. - I am using Yahoo Maps Ajax API to display maps; also some ajax calls for our build in chat (only for logged users) That's unrelated. Yahoo! isn't using ZODB. So if the things above are causing read conflicts why only for this particular object? There is nothing special or different with this one... It could be some other piece of code that tries to read from ZODB more than once and whatever it's trying to read might be something slow (slow for a computer). Have you tried running ZopeProfiler on your site to figure out if there are certain (ZO)DB related functions that are called either many many times or few but slow times. Thanks again, Dragos On Fri, Aug 28, 2009 at 12:19 PM, Peter Bengtssonpete...@gmail.com wrote: The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___
Re: [Zope] strange database read conflict error
Ooops, paste it the wrong link below the URL of the object with conflicts is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman/ On Fri, Aug 28, 2009 at 2:47 PM, Dragos Chirilaobjectval...@gmail.com wrote: All objects as the same type of this one that generates conflicts are displayed using the same template that contains the same elements (js, images, etc.). I can access any object in the same folder (http://metropotam.ro/Opera/) and I will get no read conflicts (e.g. http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/). I have conflicts when I access this particular object http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/. If the problem is related with some piece of code I can't find any logical explanation for this behavior. ... or maybe I am looking in the wrong place. Also, told you before, I even erased the object from the Data.fs and then packed it. Conflicts stopped. After I have created a new object and rename it with the old id the conflicts showed up again... Thanks, Dragos On Fri, Aug 28, 2009 at 2:37 PM, Peter Bengtssonpete...@gmail.com wrote: 2009/8/28 Dragos Chirila objectval...@gmail.com: Peter, Than you for your response. Please find below a few things: - I give up using sessions in the website's front and years ago :) There is only one page left that is using them, when u subscribe to our newsletter but I don't think that this affects because is used rarely It might be used anyway. It'll be empty (except the subscribe-to-newsletter page) all the time but it might still be in use. I think there is always a REQUEST.SESSION object. - I am using Yahoo Maps Ajax API to display maps; also some ajax calls for our build in chat (only for logged users) That's unrelated. Yahoo! isn't using ZODB. So if the things above are causing read conflicts why only for this particular object? There is nothing special or different with this one... It could be some other piece of code that tries to read from ZODB more than once and whatever it's trying to read might be something slow (slow for a computer). Have you tried running ZopeProfiler on your site to figure out if there are certain (ZO)DB related functions that are called either many many times or few but slow times. Thanks again, Dragos On Fri, Aug 28, 2009 at 12:19 PM, Peter Bengtssonpete...@gmail.com wrote: The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] strange database read conflict error
It's very very hard to reproduce. You have to have really good test coverage but even in running tests it's hard because of the fact that they're single threaded. If I were you I'd look into something some stresstesting tool and bombard your localhost till you find where it's causing most of the conflict errors. NB. Conflict errors aren't errors. That's a sucky thing about ZODB. Deep down it's an error but since Zope, on a higher more human level, will just retry it should be considered a warning or info or something. 2009/8/28 Dragos Chirila objectval...@gmail.com: All objects as the same type of this one that generates conflicts are displayed using the same template that contains the same elements (js, images, etc.). I can access any object in the same folder (http://metropotam.ro/Opera/) and I will get no read conflicts (e.g. http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/). I have conflicts when I access this particular object http://metropotam.ro/Opera/loc8982879061-Sala-Palatului/. If the problem is related with some piece of code I can't find any logical explanation for this behavior. ... or maybe I am looking in the wrong place. Also, told you before, I even erased the object from the Data.fs and then packed it. Conflicts stopped. After I have created a new object and rename it with the old id the conflicts showed up again... Thanks, Dragos On Fri, Aug 28, 2009 at 2:37 PM, Peter Bengtssonpete...@gmail.com wrote: 2009/8/28 Dragos Chirila objectval...@gmail.com: Peter, Than you for your response. Please find below a few things: - I give up using sessions in the website's front and years ago :) There is only one page left that is using them, when u subscribe to our newsletter but I don't think that this affects because is used rarely It might be used anyway. It'll be empty (except the subscribe-to-newsletter page) all the time but it might still be in use. I think there is always a REQUEST.SESSION object. - I am using Yahoo Maps Ajax API to display maps; also some ajax calls for our build in chat (only for logged users) That's unrelated. Yahoo! isn't using ZODB. So if the things above are causing read conflicts why only for this particular object? There is nothing special or different with this one... It could be some other piece of code that tries to read from ZODB more than once and whatever it's trying to read might be something slow (slow for a computer). Have you tried running ZopeProfiler on your site to figure out if there are certain (ZO)DB related functions that are called either many many times or few but slow times. Thanks again, Dragos On Fri, Aug 28, 2009 at 12:19 PM, Peter Bengtssonpete...@gmail.com wrote: The fact that the class is Products.Transience.Transience.Increaser makes me suspect that you're using sessions and within the same session you make too many read requests to the ZODB. When the load is too high, after a certain amount of attempts Zope will say, piss off! ...for a while and come back later. So it comes back later and works fine. Are you using lots of AJAX requests? That can sometimes cause read conflict errors. 2009/8/28 Dragos Chirila objectval...@gmail.com: Hello everyone, I have a question regarding a database read conflict error. I would appreciate any thoughts on this issue. I am using Zope 2.8.9.1-final, Python 2.3.7 with a ZEO server and 4 clients for 6 months now. I have noticed that the event.log is filled with read conflict errors like the one below: 2009-08-28T10:29:26 INFO ZODB.Conflict database read conflict error (oid 0x09, class Products.Transience.Transience.Increaser) at /VirtualHostBase/http/metropotam.ro:80/metropotam/VirtualHostRoot/Opera/loc4769172835-Ateneul-Roman/ (21 conflicts, of which 0 were unresolved, since startup at Thu Aug 27 15:25:02 2009) This URL of the object is http://metropotam.ro/Opera/loc4769172835-Ateneul-Roman . We have around 2000 objects of the same type and this is *the only one* that generates read conflict errors. I have tried a few things: 1. Edit object and recatalog it 2. Rename it 3. Delete object, pack database, create a new object and then rename it to keep the URL alive 4. checked the data.fs and got no errors (fstest.py) None of the above solved the problem; I still get a lot of conflicts. Is there a way to fix this? I found it very strange that I get conflicts only for this object and for no other of the same type, even in the same folder... Thank you, Dragos Chirila ___ Zope maillist - z...@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev ) -- Peter Bengtsson, work www.fry-it.com home www.peterbe.com hobby www.issuetrackerproduct.com fun crosstips.org
Re: [Zope] strange database read conflict error
+---[ Peter Bengtsson ]-- | It's very very hard to reproduce. You have to have really good test | coverage but even in running tests it's hard because of the fact that | they're single threaded. | If I were you I'd look into something some stresstesting tool and | bombard your localhost till you find where it's causing most of the | conflict errors. | NB. Conflict errors aren't errors. Conflict Errors ARE errors, unless of course you don't care about data integrity, in which case probably nothing is an error. Later versions (than 2.8) have fewer Read Conflict problems Sean Hastings did a fairly good write up a number of years ago; https://mail.zope.org/pipermail/zope/2004-September/153704.html Might be a bit dated now, but, might help put things into perspective. -- Andrew Milton a...@theinternet.com.au ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
