Re: [Zope-dev] Conflict Errors
Jim Fulton wrote at 2008-6-5 10:46 -0400: On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote: On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote: I would like to know: - Can conflict errors appear when reading objects from the ZODB? Yes Gaaa. I meant no. Nevertheless, I think you have been right in the first place. Usually, MVCC prevents read conflict errors but TempStorage only maintains a very limited history -- occasionally to small to fully implement MVCC. Then read conflict errors will occur. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
On Jun 6, 2008, at 2:41 PM, Dieter Maurer wrote: Jim Fulton wrote at 2008-6-5 10:46 -0400: On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote: On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote: I would like to know: - Can conflict errors appear when reading objects from the ZODB? Yes Gaaa. I meant no. Nevertheless, I think you have been right in the first place. Usually, MVCC prevents read conflict errors but TempStorage only maintains a very limited history -- occasionally to small to fully implement MVCC. Then read conflict errors will occur. Sure. The right answer is generally, no. :) Jim -- Jim Fulton Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Previously David Otero Figueroa wrote: Hello everyone, I have a couple questions related to conflict errors. During our home page stress testing (20 concurrent users), we detected several conflict errors (see trace below), We thought these could ONLY appear when writing objects in the ZODB. That is correct. You need to audit your check to check for undesired writes. Wichert. -- Wichert Akkerman [EMAIL PROTECTED]It is simple to make things. http://www.wiggy.net/ It is hard to make things simple. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Conflict Errors
Hello everyone, I have a couple questions related to conflict errors. During our home page stress testing (20 concurrent users), we detected several conflict errors (see trace below), We thought these could ONLY appear when writing objects in the ZODB. 2008-05-27T18:57:53 INFO ZODB conflict error at /portal/ (26 conflicts since startup at 2008-05-27T13:37:19) I have this problems too in a not estress testing. In a simple access (2 concurrents users) I have this response: 2008-05-27T13:23:48 INFO ZODB conflict error at /portal/ (22 conflicts since startup at 2008-05-27T10:37:28) 2008-05-27T13:23:48 INFO ZODB conflict error at /portal/ (23 conflicts since startup at 2008-05-27T10:37:28) 2008-05-27T13:29:21 INFO ZODB conflict error at /portal/login_form_usuario_pwd/ (29 conflicts since startup at 2008-05-27T10:37:28) 2008-05-27T13:29:22 INFO ZODB conflict error at /portal/login_form_usuario_pwd/ (30 conflicts since startup at 2008-05-27T10:37:28) 2008-05-27T19:48:40 INFO ZODB conflict error at /portal/portal_css/Plone Default/ploneStyles5658.css (68 conflicts since startup at 2008-05-27T13:37:19) I would like to know: - Can conflict errors appear when reading objects from the ZODB? - Could conflict errors be the cause of a spinning ZOPE scenario? - Is there any situation where ZOPE could try to write objects when rendering an only read page? - How can they be solved? - Can I realize partial commits? How to? Our configuration is as follows: Apache + Pound + ZEO architecture (5 ZEO clients, 1 ZEO storage server - 650.000 ZODB objects) Thanks in advance. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote: On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote: I would like to know: - Can conflict errors appear when reading objects from the ZODB? Yes Gaaa. I meant no. Jim -- Jim Fulton Zope Corporation ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
Tim Peters wrote: I'm not really happy with that, because I have no theory for how ZEO client cache files _get_ out of synch. The most common one for me is connecting a ZEO client to one storage server (say a dev one) and then changing the config to connect to another storage server (say a staging one) without making sure the cache files are gone. That seems ot do it most time, and I'm sure I've got POSKeyErrors and all manner of other weirdness too. My workaround is never to use persistent client caches on clients that I'm likely to connect to different servers. This seems a shame, I wish there was some kind of safety check that ZEO could do to go wait, these caches are insane, they don't even apply to the server you're connecting to cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
Paul Winkler wrote at 2005-2-24 15:16 -0500: ... Twiddling around with some old suggestions from Dieter Maurer (see http://aspn.activestate.com/ASPN/Mail/Message/Zope-CMF/1978842 ), I added a _p_jar.sync() where he suggested - right before the final except: in initialize_product(). No joy. Note that the database is initialized in a separate connection -- in case it is empty. This may cause ConflictErrors in the primary connection. -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
Paul Winkler wrote at 2005-2-24 14:29 -0500: This is a topic that comes up occasionally, but since the symptom seems to be rare and hard to provoke, it never gets resolved. Well it's biting me again and I'm determined to fix it, but would like some input from the list. I know that ZopeProfiler can cause such ConflictErrors and (what is more) why it does this. ZopeProfiler (like some other products) wants to install itself in Control_Panel. Because, this requires a database connection and because App.ProductContext.ProductContest tries hard to hide its application object, ZopeProfiler opens a new one, modifies Control_Panel and commits the transaction and then closes its connection. Such modifications via a second ZODB connection can cause unexpected (and puzzling) ConflictError on startup. The next ZopeProfiler version will access ProductContext.__app instead (using the mangled name). -- Dieter ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
ZODB.POSException.ConflictError: database conflict error ( serial this txn started with 0x035b25f36751f988 2005-02-10 18:59:24.215675, serial currently committed 0x035b743c6d186822 2005-02-24 17:00:25.569220) [Dieter Maurer] Where is the oid in this ConflictError report? It is vital information to analyse which object causes the problem... I don't know. I reformatted Paul's traceback to line up the timestamps for easier reading, but there was no oid in the original either. There wouldn't be, either, if no oid was passed to the ConflictError constructor. ConflictError.__str__ only includes info for the stuff it was told about: def __str__(self): extras = [] if self.oid: extras.append(oid %s % oid_repr(self.oid)) if self.class_name: extras.append(class %s % self.class_name) if self.serials: current, old = self.serials extras.append(serial this txn started with %s % readable_tid_repr(old)) extras.append(serial currently committed %s % readable_tid_repr(current)) if extras: return %s (%s) % (self.message, , .join(extras)) else: return self.message In Paul's case, the ConflictError occurred on the ZEO server, but the traceback came from the ZEO client (just raising the Exception object the ZEO server passed to it). There's no way then to guess what the traceback might have looked like on the ZEO server side (b!). Perhaps looking at the ZEO server log would have helped. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 02:29:50PM -0500, Paul Winkler wrote: This is a topic that comes up occasionally, but since the symptom seems to be rare and hard to provoke, it never gets resolved. Well it's biting me again and I'm determined to fix it, ... And since my ConflictError turned out to be merely caused by a borked ZEO client cache, I've given up on the hunt ... again :-\ For all I know, ZEO cache may have been the cause of my previous run-ins with ConflictError on startup too. -PW -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Conflict errors on startup, revisited
This is a topic that comes up occasionally, but since the symptom
seems to be rare and hard to provoke, it never gets resolved.
Well it's biting me again and I'm determined to fix it, but
would like some input from the list.
Related collector issues:
http://www.zope.org/Collectors/Zope/300
http://www.zope.org/Collectors/Zope/1201
The issue:
On rare occasions I have seen ConflictErrors raised during
product installation.
Today, I can finally provoke them at will with a pretty minimal setup.
The setup:
Zope 2.7.3, using ZEO, with DirectoryStorage on the ZEO server.
Hardware: dual-processor PIII 800, some kinda scsi disk, ext3,
some old redhat flavor of linux.
To provoke it I need merely do
./bin/zopectl run foo.py
... where foo.py is empty.
Products/ now contains only one dummy product,
which I've named Conflictor, which contains only an empty __init__.py.
I get a ConflictError during registration of Conflictor.
Interestingly I can only provoke this ConflictError in this particular
INSTANCE_HOME. I have several other instances and am not seeing the
symptom there.
I started trivially hacking OFS/Application.py to add some debugging output
(a diff against 2.7.3 is attached), and here's what I get:
[EMAIL PROTECTED] PaulMiniInstanceHome]$ ls Products/
[EMAIL PROTECTED] PaulMiniInstanceHome]$ mkdir Products/Conflicter
[EMAIL PROTECTED] PaulMiniInstanceHome]$ touch Products/Conflicter/__in
it__.py
[EMAIL PROTECTED] PaulMiniInstanceHome]$ touch foo.py
[EMAIL PROTECTED] PaulMiniInstanceHome]$ ./bin/zopectl run foo.py
installing product PluginIndexes...OK!
installing product Conflicter... BZZT
Traceback (most recent call last):
File string, line 1, in ?
File /usr/local/lib/python2.3/site-packages/PIL/__init__.py, line 51, in app
File /usr/local/lib/python2.3/site-packages/PIL/__init__.py, line 47, in
startup
File /zope/Zope-2.7.3-SoftwareHome/lib/python/Zope/App/startup.py, line 96,
in startup
File /zope/ZopeSoftwareHome/lib/python/OFS/Application.py, line 279, in init
ialize
initializer.initialize()
File /zope/ZopeSoftwareHome/lib/python/OFS/Application.py, line 306, in init
ialize
self.install_products()
File /zope/ZopeSoftwareHome/lib/python/OFS/Application.py, line 561, in inst
all_products
return install_products(app)
File /zope/ZopeSoftwareHome/lib/python/OFS/Application.py, line 597, in inst
all_products
folder_permissions, raise_exc=debug_mode)
File /zope/ZopeSoftwareHome/lib/python/OFS/Application.py, line 803, in inst
all_product
get_transaction().commit()
File /zope/Zope-2.7.3-SoftwareHome/lib/python/ZODB/Transaction.py, line 252,
in commit
File /zope/Zope-2.7.3-SoftwareHome/lib/python/ZODB/Connection.py, line 738,
in tpc_vote
File /zope/Zope-2.7.3-SoftwareHome/lib/python/ZEO/ClientStorage.py, line 841
, in tpc_vote
File /zope/Zope-2.7.3-SoftwareHome/lib/python/ZEO/ClientStorage.py, line 825
, in _check_serials
ZODB.POSException.ConflictError: database conflict error (serial this txn starte
d with 0x035b25f36751f988 2005-02-10 18:59:24.215675, serial currently committed
0x035b743c6d186822 2005-02-24 17:00:25.569220)
I guess what I'm asking, before I hack more on OFS/Application.py,
is this:
* any ideas re. what on earth could be causing this ConflictError?
the product doesn't DO anything fer chrissakes.
(Maybe registration is finishing too fast??)
* Is the PIL stuff in the traceback a red herring or what?
--
Paul Winkler
http://www.slinkp.com
*** Application.py.ORIG Thu Feb 24 14:27:23 2005
--- Application.py Thu Feb 24 14:32:24 2005
***
*** 34,39
--- 34,40
from Acquisition import aq_base
from App.Product import doInstall
from App.config import getConfiguration
+ from ZODB.POSException import ConflictError
class Application(Globals.ApplicationDefaultPermissions,
ZDOM.Root, Folder.Folder,
***
*** 587,594
if done.has_key(product_name):
continue
done[product_name]=1
! install_product(app, product_dir, product_name, meta_types,
! folder_permissions, raise_exc=debug_mode)
Products.meta_types=Products.meta_types+tuple(meta_types)
Globals.default__class_init__(Folder.Folder)
--- 588,606
if done.has_key(product_name):
continue
done[product_name]=1
! # collector issue 300: handle ConflictErrors during installation
! sys.stderr.write('installing product %s' % product_name)
! try:
! import time; time.sleep(0.1)
! app._p_jar.sync() # DM's suggestion
! install_product(app, product_dir, product_name, meta_types,
! folder_permissions, raise_exc=debug_mode)
! sys.stderr.write('...OK!\n')
! except ConflictError:
! # somehow we should retry it. XXX look at zpublisher
! sys.stderr.write('... BZZT\n')
! raise
!
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 02:29:50PM -0500, Paul Winkler wrote: * Is the PIL stuff in the traceback a red herring or what? Note that PIL is not referred to by anything in my Products directory. (Heck, *nothing* is referred to by anything in Products.) That's what's so baffling about it appearing in the traceback. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote: Two words: *hurt system*. Please, let me kill it. Interesting. How on earth did you determine that? You think it has to do with registering help for PluginIndexes? I'm gonna go test that theory - i'll hack registerHelp() to no-op. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, 2005-02-24 at 14:57, Paul Winkler wrote: On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote: Two words: *hurt system*. Please, let me kill it. Interesting. How on earth did you determine that? Womanly intuition. You think it has to do with registering help for PluginIndexes? I'm gonna go test that theory - i'll hack registerHelp() to no-op. That's a good verifier. - C ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 02:58:01PM -0500, Chris McDonough wrote: On Thu, 2005-02-24 at 14:57, Paul Winkler wrote: On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote: Two words: *hurt system*. Please, let me kill it. Interesting. How on earth did you determine that? Womanly intuition. You think it has to do with registering help for PluginIndexes? I'm gonna go test that theory - i'll hack registerHelp() to no-op. That's a good verifier. Well, no joy. I hacked ProductContext.register* to return from the first line, so even registerClass() now does nothing. Barf, I still get the ConflictError. Twiddling around with some old suggestions from Dieter Maurer (see http://aspn.activestate.com/ASPN/Mail/Message/Zope-CMF/1978842 ), I added a _p_jar.sync() where he suggested - right before the final except: in initialize_product(). No joy. But - duh - my traceback clearly shows the ConflictError happens before that, in the call to get_transaction().commit(). SO I moved the call to app._p_jar.sync() to right before hte call to get_transaction().commit(). Aaaand... no more ConflictError. Weird weird weird. I'd be happy to check this in to cvs / svc, except that A) I have no clue how to write a test that exercises the bug, and B) so far nobody seems to understand what's really happening. If nobody objects, I wouldnt' mind checking the fix in anyway :-) -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
[Paul Winkler] ... ZODB.POSException.ConflictError: database conflict error ( serial this txn started with 0x035b25f36751f988 2005-02-10 18:59:24.215675, serial currently committed 0x035b743c6d186822 2005-02-24 17:00:25.569220) No time here for thought, but those timestamps are two weeks apart. Yikes! Just thought I'd mention that wink. A write conflict (which this is) happens when (and in this time order): 1. A transaction reads an object O. 2. A different transaction modifies O and commits its new state. 3. The original transaction also modifies O, and tries to commit its new state. #3 fails, because the original transaction has, in general, no way to know whether the changes it tried to make to O's state are consistent with the changes the other transaction made to its state in step #2. For your specific exception above, step #1 started by reading the state of the object as it existed on 2005-02-10. In step #2, someone else modified the same object, and committed its state around 2005-02-24 17:00:25. The original transaction then failed in step #3. Do you use a persistent ZEO client cache? If so, one thing to try is to physically remove all the cache files -- maybe they've gotten into an insane state. It's not impossible that you should have a 2-week difference in timestamps here, but it does look unusual. It's certainly possible (even expected) if two transactions try to modify the same object, and the last time that object was changed was in fact two weeks ago. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
[Paul Winkler] ... But - duh - my traceback clearly shows the ConflictError happens before that, in the call to get_transaction().commit(). SO I moved the call to app._p_jar.sync() to right before hte call to get_transaction().commit(). Aaaand... no more ConflictError. Well, that's certain: the first thing sync() does is abort the current transaction. Then of course commit() won't raise an exception: sync() threw away all the changes. ... If nobody objects, I wouldnt' mind checking the fix in anyway :-) It would be much clearer instead to change the get_transaction().commit(). to get_transaction().abort() That would get rid of the ConflictError too, and for the same reason. It has the advantage of being obviously insane, though wink. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 03:16:15PM -0500, Paul Winkler wrote: But - duh - my traceback clearly shows the ConflictError happens before that, in the call to get_transaction().commit(). SO I moved the call to app._p_jar.sync() to right before hte call to get_transaction().commit(). Aaaand... no more ConflictError. Never mind, this breaks existing unit tests. Duh, it prevents products from getting initialized at all :-\ now checking out tim's suggestion... -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
[Tim Peters] ... Do you use a persistent ZEO client cache? If so, one thing to try is to physically remove all the cache files -- maybe they've gotten into an insane state. Off list, Paul said bingo! to that one -- deleting his ZEO client cache files apparently made the problem go away. I'm not really happy with that, because I have no theory for how ZEO client cache files _get_ out of synch. I suspect they do, though, more often than stray gamma rays and pilot error could account for. Alas, I've got no time to give to that these days. If someone else wants to solve that, I'll pass on the cigar Paul offered to me wink. ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on startup, revisited
On Thu, Feb 24, 2005 at 04:11:35PM -0500, Tim Peters wrote: [Tim Peters] ... Do you use a persistent ZEO client cache? If so, one thing to try is to physically remove all the cache files -- maybe they've gotten into an insane state. Off list, Paul said bingo! to that one -- deleting his ZEO client cache files apparently made the problem go away. Yep, meant to mail the list, hit the wrong button. Thanks again Tim! I'm not really happy with that, because I have no theory for how ZEO client cache files _get_ out of synch. I suspect they do, though, more often than stray gamma rays and pilot error could account for. Alas, I've got no time to give to that these days. If someone else wants to solve that, I'll pass on the cigar Paul offered to me wink. Oh-sure-pretend-you-didn't-want-the-bag-of-ham-ly y'rs, -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Hi Chris, hi folks. thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse. we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them. what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request). this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session. so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment. i hope some of the community sees the importance of this. Regards Thomas On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote: Hi, whe are running portal unter zope2.7 with one zeo server and six frontend application servers. I have set up zeo to server two storages filestorage main path $INSTANCE_HOME/var/Data.fs /filestorage minimalstorage temp envdir $INSTANCE_HOME/var/session /minimalstorage temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often. BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Even though you'll need to pack frequently, you might want to try a FileStorage instead. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On 18/05/2004, at 6:57 AM, Gerhard Schmidt wrote: On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote: AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Perhaps you can get some relief by using pound as your load balancer. It is quite capable at sniffing out user sessions including watching the Cookie header. See http://www.apsis.ch/pound/ Michael ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Hi Dr. Wagner, thanks for the hint with minimal storage. however the same problem occurs with filestorage and is even worse. we know very well, that we cannot avoid the appearance of conflict errors in particular with zeo, and we understand what causes them. what wonders us is, that we find some contributions about this errors in the mailing lists, but no one seems to have a real problem with it. but we have! we are not worried about some errors appearing in a log, but we are using plone2 witch currently 30 concurrent authenticated users, so that this error occurs permanently, thus decreasing system performance due to several retries and moreover leading to very frequent nasty 'POS.Exception Errors' in the browser, whenever four successive tries fail (almost every 100. request). this is not acceptable, and i wonder wheter there are any plone sites with more than a handful of authenticated users that experience the same problem. we found that one (there are more) product responsible for it is the placeless translation service, which heavily writes to the session. Are you using ZEO for your session storage or are you using a local storage? (You hint at it above but you don't actually say one way or the other.) so we desperately need a solution for this problem, maybe uncouple the session of the locking mechanism or whatever, and this should be a problem that should be addressed by the core team. because this is a strong reason (and so far the only one) to avoid using zope/plone in a large user environment. I also need reliable sessioning for my own customers, so I am working towards fixing the issues I know about. But the issues I know about may not be the same issues as those that you want fixed. You can ensure your issues are addressed by devoting resources towards the problem, be that time+expertise or money or both. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Gerhard Schmidt wrote at 2004-5-17 21:01 +0200: ... However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports INSTANCE_HOME). Is that a bug or intetional. I expect it to be a bug. I plan to fix it soon -- but currently, I have lots of things to do. May take some months... -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, May 17, 2004 at 12:45:16PM -0400, Chris McDonough wrote: On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote: Hi, whe are running portal unter zope2.7 with one zeo server and six frontend application servers. I have set up zeo to server two storages filestorage main path $INSTANCE_HOME/var/Data.fs /filestorage minimalstorage temp envdir $INSTANCE_HOME/var/session /minimalstorage temp should provide an shared session. But when is set up the frontends to use temp as /temp_folder, we see many ZODB conflict errors. The performance decreases and PosErrors appear quite often. BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. Why isn´t there an implementation of temporary storage for ZEO I dont care about conflit resolution. Its only session data. I simply need a shared session storage. But as Far as i can see the is no way of doing this. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session Bye Estartu -- Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | [EMAIL PROTECTED] | auf Anfrage/ Tel: 08232 77 36 4 ||on request Fax: 08232 77 36 3 || pgpqXUCWUVwl0.pgp Description: PGP signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote: BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. When was that and who told you so? Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. OK. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level. Why isnt there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. I simply need a shared session storage. But as Far as i can see the is no way of doing this. I don't understand. You seem to have it working. Is performance the issue? Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
Gerhard Schmidt wrote at 2004-5-13 11:23 +0200: ... Is there any why to get a shared session working. You know that the ZODB is not designed for high frequency concurrent writes? When you want to reduce conflicts you will need to try hard to minimize concurrent writes. I have a (non-open) Transience implementation that writes much less then Zope's standard implementation. However, due to a ZEO weakness, its conflict resolution currently does not work (ZEO no longer supports INSTANCE_HOME). -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote: On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote: BDBMininalStorage is no longer maintained and doesn't do any conflict resolution. Why this. I was told some time ago we should use BDBMinimalStorage to fix the conflict problems. When was that and who told you so? Even though you'll need to pack frequently, you might want to try a FileStorage instead. There is no difference between BDB Storage and Filestorage regarding the conflicts. OK. Regardless, using ZEO will presumably always generate more conflict errors than using a local ZODB because the commit time is longer, and thus there's a better chance that two threads will be committing at the same time. The bottom line is that there is no way of getting an shared session running. It works doesn't it? You just get conflict errors, right? Conflict errors are normal at a certain level. No It doesn`t. At the Moment we run with single sessions on each frontend we have rewitten most ob the code using sessions to store ther information in cockies. But that´s not realy a good solution. Why isn´t there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I didn´t find a way to use Tempstorage in zeo. temporarystorage name temporary storage for sessioning /temporarystorage didn´t work. How do i Set up a temporarystorage in zeo. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage. I simply need a shared session storage. But as Far as i can see the is no way of doing this. I don't understand. You seem to have it working. Is performance the issue? The Performance decrase is there but the Problem is far deeper. If an read conflict isn´t resolved after three retries an Exception is raised an the request is aborted. And thats the real problem. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesn´t work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Bye Estartu -- Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | [EMAIL PROTECTED] | auf Anfrage/ Tel: 08232 77 36 4 ||on request Fax: 08232 77 36 3 || pgpsBoo3UooYb.pgp Description: PGP signature ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict errors on BDBMinimal storage
On Mon, 2004-05-17 at 14:57, Gerhard Schmidt wrote: Why isnt there an implementation of temporary storage for ZEO There is, just use the tempstorage package from Zope. I didnt find a way to use Tempstorage in zeo. temporarystorage name temporary storage for sessioning /temporarystorage didnt work. How do i Set up a temporarystorage in zeo. Good question. ;-) It involves changing the ZEO configuration machinery to recognize the tempstorage package. The steps to do this I couldn't tell you without actually doing it, however. I dont care about conflit resolution. Its only session data. For better or worse, because it is based on ZODB, the implementation makes you need to care. Than maybe the sessioncode should be redesigned not to use ZODB as storage but store ther information in some different system maybe a RDMB or some other system able to handle the kind of usage. Zope doesn't ship with a useful and maintained relational database. You can implement your own SQL-based TransientObjectContainer using the interfaces defined in Transience/Interfaces.py if you wish, or use Anthony Baxter's SQLSession product. It would be nice if there was an alternate implementation that used a relational database like this. I simply need a shared session storage. But as Far as i can see the is no way of doing this. Of course there is, it's just not built in. ;-) I don't understand. You seem to have it working. Is performance the issue? The Performance decrase is there but the Problem is far deeper. If an read conflict isnt resolved after three retries an Exception is raised an the request is aborted. And thats the real problem. I see. I'm not sure switching to TemporaryStorage will help much. I would first try using a FileStorage and make sure to put the Zope lib/python directory on the ZEO server's Python's PYTHONPATH (to allow for conflict resolution under ZEO, ZEO must be able to import the Products package and subpackages therein). I don't know if this will help very much, but it's worth a shot. Are we the only site out there running Plone2 (Placeless transaltion Service) at that scale. How do others handle there Session AFAIK Zope is the only appserver that even makes an attempt at allowing session data to be shared between servers like this. Most other appservers require you to set up session affinity in your load balancer and have a session database on each appserver. and nealy every system failed to do so. How do you implement this session affinity. Based on source IP. At the moment every large ISP uses multiple Proxyserver so every request comes from a different IP so the affinity doesnt work for them. The Proxyserver provided by our Computing Center is also split on at least 8 Hosts, so everyone using this proxy would lose session infomation, getting very strange effekts. Yes. It's a hard problem. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors; how to track them down?
Bjorn Stabell wrote: In any case, is there no way to find out more detailed information about why a conflict happened? There must be some oids somewhere? I don't know of a way. In a conflict, the transaction gets aborted, so the only data left is the log entry. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors; how to track them down?
Bjorn Stabell wrote at 2003-6-3 11:08 +0800:
Except for SARS, another problem that's been plagueing us for months are
seemingly random Conflict Errors. We see about 10 every day on our Zope
2.6.1, and they can happen on any page. In most cases they are not
related to pages that actually updated anything (AFAIK), although we do
use SESSIONS (but no frames). Any clue how to track down which objects
are having problems, what's causing this?
--
2003-06-03T09:08:10 INFO(0) ZODB conflict error at
/VirtualHostBase/http/www.beijingsammies.com:80/sammies/VirtualHostRoot/
(7 conflicts since startup at 2003-06-02T13:15:05)
The log message for conflict errors is far less instructive than
it could be (almost without any effort).
It should indicate whether it is a ReadConflictError or
a (Write)ConflictError.
It should tell which object caused the conflict.
The attached patch to Zope/App/startup.py provides this
additional information.
Please file a feature request to http://collector.zope.org/Zope.
Dieter
--- startup.py~ 2003-02-28 16:59:25.0 +0100
+++ startup.py 2003-03-17 11:13:28.0 +0100
@@ -140,9 +140,9 @@
global conflict_errors
conflict_errors = conflict_errors + 1
method_name = REQUEST.get('PATH_INFO', '')
-err = ('ZODB conflict error at %s '
+err = ('ZODB conflict error at %s: %s'
'(%s conflicts since startup at %s)')
-LOG(err % (method_name, conflict_errors, startup_time),
+LOG(err % (method_name, str(v), conflict_errors, startup_time),
INFO, '')
LOG('Conflict traceback', BLATHER, '', error=sys.exc_info())
raise ZPublisher.Retry(t, v, traceback)
Re: [Zope-dev] Conflict Errors; how to track them down?
Dieter Maurer wrote: The attached patch to Zope/App/startup.py provides this additional information. Where's the patch? cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors; how to track them down?
Chris Withers wrote: Dieter Maurer wrote: The attached patch to Zope/App/startup.py provides this additional information. Where's the patch? http://marc.theaimsgroup.com/?l=zope-devm=105466926610469q=p3 -- Jamie Heilman http://audible.transient.net/~jamie/ I was in love once -- a Sinclair ZX-81. People said, No, Holly, she's not for you. She was cheap, she was stupid and she wouldn't load -- well, not for me, anyway. -Holly ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Conflict Errors; how to track them down?
Hi all, Except for SARS, another problem that's been plagueing us for months are seemingly random Conflict Errors. We see about 10 every day on our Zope 2.6.1, and they can happen on any page. In most cases they are not related to pages that actually updated anything (AFAIK), although we do use SESSIONS (but no frames). Any clue how to track down which objects are having problems, what's causing this? -- 2003-06-03T09:08:10 INFO(0) ZODB conflict error at /VirtualHostBase/http/www.beijingsammies.com:80/sammies/VirtualHostRoot/ (7 conflicts since startup at 2003-06-02T13:15:05) -- I volounteer to write up a HOWTO document regarding this and the Bad reference to errors we're also getting (see previous post) if I can just find out what's wrong. Regards, -- Bjorn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors; how to track them down?
On 06/02/2003 11:08 PM, Bjorn Stabell wrote: Hi all, Except for SARS, another problem that's been plagueing us for months are seemingly random Conflict Errors. We see about 10 every day on our Zope 2.6.1, and they can happen on any page. In most cases they are not related to pages that actually updated anything (AFAIK), although we do use SESSIONS (but no frames). Any clue how to track down which objects are having problems, what's causing this? Look at your undo log. If the conflicts don't correspond with entries in the undo log, and you're not mounting other databases, the conflicts are almost certainly due to sessions. However, conflicts are a normal occurrence and rarely reach the user. Zope automatically retries the request and delivers the successful page. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
RE: [Zope-dev] Conflict Errors; how to track them down?
On 06/02/2003 11:08 PM, Bjorn Stabell wrote: Except for SARS, another problem that's been plagueing us for months are seemingly random Conflict Errors. We see about 10 every day on our Zope 2.6.1, and they can happen on any page. In most cases they are not related to pages that actually updated anything (AFAIK), although we do use SESSIONS (but no frames). Any clue how to track down which objects are having problems, what's causing this? Shane wrote: Look at your undo log. If the conflicts don't correspond with entries in the undo log, and you're not mounting other databases, the conflicts are almost certainly due to sessions. However, conflicts are a normal occurrence and rarely reach the user. Zope automatically retries the request and delivers the successful page. They do not always correspond to entries in the undo log. But the ones that do are always (?) committed successfully. The problem is, as I can see from other threads here, that they are related to other systems that cannot be rolled back (e.g., sending email / faxes), so we get duplicate emails / faxes. I'll look at how to wrap these systems in transaction-aware wrappers, as was mentioned in a related thread. In any case, is there no way to find out more detailed information about why a conflict happened? There must be some oids somewhere? Regards, -- Bjorn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Chris Withers writes: However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case... This has recently been answered in the list. Here a short summary: a persistent object with all its attributes. If the attribute is itself a persistent object, then the "separate" object contains only a reference to it. Otherwise, the complete attribute value (until persistent objects are hit) is part of the object's value. Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
On Tue, 02 Jan 2001 16:50:49 +, Chris Withers [EMAIL PROTECTED] wrote: However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case... That is exactly correct. Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Chris McDonough wrote: The ZODB tries hard to avoid error conditions for you by retrying conflicting writes, but if you really pound the snot out of a particular object, it'll let you know. That's gotta be one for the Zope quotes board, if I could onyl remember the URL :-) and step through a real-world ConflictError situation. Neither of which I have time to do right now... sorry, Chris. this one by saying "I don't know" because it's one of those things where learning exactly what is going on can chew up time like nobody's business, because I'd need to step through the code line-by-line. I shouldn't even be in here. :-( Hey, no problem, the help was much appreciated :-) However, if anyone does know what constitutes a 'seperate object' in ZODB terms, it be really useful to know. I'm guessing a class which inherits from Persistent? Dictionaries don't, 'cos they're python builtins, which I'm pretty certain was the problem in my case... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Y'know since replying to this message originally, I've learned a lot about
pickled representations of Zope objects and I think I can answer this more
cogently:
- Instances of classes which inherit from Persistence.Persistent are
represented as an individual pickle. Their attributes are contained in this
pickle, unless those attributes are themselves instances of something that
inherits from Persistence.Persistent.
- References to persistent objects in other objects are kept as an object
identifier. This means, for example, that a dictionary of integers to
persistent object instances looks something like this in a pickle::
{1:oid 1, 2:oid 2, ...}
If this dictionary is an attribute of a persistent object, adding,
changing, or removing items from it will modify its containing persistent
object.
It was this "reference by oid" concept that turned the light on for me here,
hopefully it can do the same for you.
One of the potential big wins with the current Zope implementation of a
BTree is that it exposes a mapping interface, so you can treat it just like
a dictionary. However, internally, it's composed of a number of separate
persistent objects that can be changed independently of each other. This
means that changing one item in a BTree will change one persistent object
that makes up a BTree, but will not necessarily effect the other persistent
objects that compose it. Unortunately, currently, the BTree implementation
has some problems that cause that *not* to be the case in many situations,
and independent writes to it often conflict. We are working on another
BTree implementation to resolve the issue.
- Original Message -
From: "Chris Withers" [EMAIL PROTECTED]
To: "Chris McDonough" [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Sent: Tuesday, January 02, 2001 11:50 AM
Subject: Re: [Zope-dev] Conflict Errors
Chris McDonough wrote:
The ZODB tries
hard to avoid error conditions for you by retrying conflicting writes,
but
if you really pound the snot out of a particular object, it'll let you
know.
That's gotta be one for the Zope quotes board, if I could onyl remember
the URL :-)
and step through a real-world ConflictError situation. Neither of which
I
have time to do right now... sorry, Chris.
this one by saying "I don't know" because it's one of those things where
learning exactly what is going on can chew up time like nobody's
business,
because I'd need to step through the code line-by-line. I shouldn't
even be
in here. :-(
Hey, no problem, the help was much appreciated :-)
However, if anyone does know what constitutes a 'seperate object' in
ZODB terms, it be really useful to know.
I'm guessing a class which inherits from Persistent? Dictionaries don't,
'cos they're python builtins, which I'm pretty certain was the problem
in my case...
cheers,
Chris
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Chris Withers wrote: i'm no expert but deeply nested data structures in a single object are rife with possibilities for conflict errors. if data might be changing concurrently in the structures or frequently changing you would be better off to switch the structure to nested persistent objects. No, for once it has nothing to do with the catalog. ;-) This is a python product of ours that stores lots of data in somewhat nested, persistent dictionaries. How can this be improved so this doesn't happen? I know your next questions are going to be "why?" and "how do I fix it?" and "why don't you fix it?". While I'd love it if you'd fix our code for free, it's probably beyond the OSS remit ;-) Any clues about how to make our code 'less bad' would be cool... cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
On Tue, 21 Nov 2000 17:03:17 +, Chris Withers [EMAIL PROTECTED] wrote: It happens when more than one thread tries to modify the same object in storage at the same time. It's Zope's equivalent of record-locking... It's normal. There's no corruption or anything, it's telling you that it avoided a write that might have caused problems. Zope tries three times to retry the write. If the object is still locked by another database connection after three writes, it propogates the exception up to the app level. That's the error you see. Hurm, well, it appears to happen when we're doing a lot of sequential write to a product (automated bulk data upload), but the thing that triggers it off is actually trying to read a page from the same area of the site... confusing. Although that could also be a one-off coincidence... ZODB marks each persistent object that is modified during a transaction. Before committing, it checks whether any of those objects have been modified-and-commited in another transaction. If it has, it raises a ConflictError. If your 'page read' really doesnt modify any objects (check your undo log, or use tranalyzer) then it cant cause a ConflictError. You say no corruption or anything, but if a submit results in that error, does the submitted form data get processed or not? As Chris explianed, the publisher will retry up to three ConflictErrors. If you see this message only in a log then it probably suceeded on one of those retries. If you see it in a traceback returned over http then the transaction has been aborted. No corruption, however your request has not been processed. Careful application coding can reduce the chance of conflict errors. Can you describe what you mean by careful application coding? "Minimise the chance of a single persistent object being modified by two concurrent transactions." Some examples: 1. The much-maligned ZODB-page-hit-counter is bad in this respect. Each 'read' of a page modifies the counter object, causing the second concurrent read (which is actually a write) to raise ConflictError. 2. Folder objects are pretty good, although you cant concurrently add two objects to a folder since both transactions modify the folder object. 3. If you need something folderlike with a higher hit rate, it is possible to avoid some conflicts by splitting the folder. For example, having one sub-folder for each initial letter of the id. You can add objects 'chrisw' and 'tdickenson' concurrently (into subfolders with the id 'c' and 't'), however concurrently adding 'chrisw' and 'chrism' would cause one ConflictErrorm because they both modify the sub-folder 'c'. (BTreeFolders have the same advantage, for the same reason, only neater) Toby Dickenson [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
Hurm, well, it appears to happen when we're doing a lot of sequential write to a product (automated bulk data upload), but the thing that triggers it off is actually trying to read a page from the same area of the site... confusing. Although that could also be a one-off coincidence... Conflict errors can be raised by three things: the storage, the transaction, and the connection. The storage will only raise the conflict error on a write, this is for sure. The transaction and the connection objects I looked at a little today, and it's unclear .. although as I understand it, the idea of a ConflictError is to prevent the modification of an object by two threads at the same time, and should have nothing to do with reads. You say no corruption or anything, but if a submit results in that error, does the submitted form data get processed or not? No... the transaction is aborted... Careful application coding can reduce the chance of conflict errors. Can you describe what you mean by careful application coding? Ah I knew I wasn't gonna get away with that one. ;-) I probably shoulda kept my mouth shut. If you're writing a ZODB-based app that uses multiple database connections (like Zope ;-), you should take care to stagger high-load writes against an object in the threads that use those database connections. The ZODB tries hard to avoid error conditions for you by retrying conflicting writes, but if you really pound the snot out of a particular object, it'll let you know. That said, I doubt I can be more specific, because to be honest I don't know the *real* answer myself. Sigh. I think I need to learn more about pickling (to figure out how the objects get split up into component pieces) and step through a real-world ConflictError situation. Neither of which I have time to do right now... sorry, Chris. I bet it happens on a catalog write for you. No, for once it has nothing to do with the catalog. ;-) This is a python product of ours that stores lots of data in somewhat nested, persistent dictionaries. How can this be improved so this doesn't happen? Can you replicate the error on demand? Or is it intermittent? This should be something covered in developer docs (along with a zillion other things). I suppose a gross hack would be to raise the number of write retries on conflicts in the ZODB code if you didn't want to analyze your app code. But y'know, as I think more about it, I'm afraid I'm gonna need to bail on this one by saying "I don't know" because it's one of those things where learning exactly what is going on can chew up time like nobody's business, because I'd need to step through the code line-by-line. I shouldn't even be in here. :-( Sorry, C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Conflict Errors
Hi, Whenever there's a lot of writes and the occasional read going on, we get the following error: 2000-11-19T16:03:44 INFO(0) Z2 CONFLICT Competing writes at, /aFolder/anObject Traceback (innermost last): File D:\Program Files\Zope\lib\python\ZPublisher\Publish.py, line 175, in publ ish File D:\Program Files\Zope\lib\python\Zope\__init__.py, line 235, in commit File D:\Program Files\Zope\lib\python\ZODB\Transaction.py, line 251, in commit File D:\Program Files\Zope\lib\python\ZODB\Connection.py, line 267, in commit ConflictError:y Can anyone shed any light on: -what this means -how bad it is -how we can make it stop happening cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Conflict Errors
It happens when more than one thread tries to modify the same object in storage at the same time. It's Zope's equivalent of record-locking... It's normal. There's no corruption or anything, it's telling you that it avoided a write that might have caused problems. Zope tries three times to retry the write. If the object is still locked by another database connection after three writes, it propogates the exception up to the app level. That's the error you see. Careful application coding can reduce the chance of conflict errors. I've seen conflict errors pop up a lot when the catalog is used for whatever reason, especially with catalogaware. I bet it happens on a catalog write for you. I think this may have something to do with the current BTree implementation's propensity to write child-parent relationships all the way up and down the tree every time a node changes. I know your next questions are going to be "why?" and "how do I fix it?" and "why don't you fix it?". Chris Petrilli did some work on the catalog for 2.2.3 that might have a positive impact. The catalog is (I think) being actively worked on if you can nail it down some, it'd probably be helpful to figure out when the conflicterror occurs and pass it along to Chris in the collector. Besides that, I don't have any specific hints for you, besides waiting for the new BTree implementation. The answer to "when?": I have no idea. HTH, C - Original Message - From: "Chris Withers" [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Monday, November 20, 2000 10:46 AM Subject: [Zope-dev] Conflict Errors Hi, Whenever there's a lot of writes and the occasional read going on, we get the following error: 2000-11-19T16:03:44 INFO(0) Z2 CONFLICT Competing writes at, /aFolder/anObject Traceback (innermost last): File D:\Program Files\Zope\lib\python\ZPublisher\Publish.py, line 175, in publ ish File D:\Program Files\Zope\lib\python\Zope\__init__.py, line 235, in commit File D:\Program Files\Zope\lib\python\ZODB\Transaction.py, line 251, in commit File D:\Program Files\Zope\lib\python\ZODB\Connection.py, line 267, in commit ConflictError:y Can anyone shed any light on: -what this means -how bad it is -how we can make it stop happening cheers, Chris ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
