Re: [Zope-dev] Conflict Errors

[Jim Fulton]

On Jun 6, 2008, at 2:41 PM, Dieter Maurer wrote:


Jim Fulton wrote at 2008-6-5 10:46 -0400:


On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote:



On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote:


I would like to know:
- Can conflict errors appear when reading objects from the ZODB?


Yes



Gaaa. I meant no.


Nevertheless, I think you have been right in the first place.

Usually, MVCC prevents read conflict errors but "TempStorage"
only maintains a very limited history -- occasionally to small
to fully implement MVCC. Then read conflict errors will occur.



Sure. The right answer is "generally, no". :)

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Dieter Maurer]

Jim Fulton wrote at 2008-6-5 10:46 -0400:
>
>On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote:
>
>>
>> On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote:
>>>
>>> I would like to know:
>>> - Can conflict errors appear when reading objects from the ZODB?
>>
>> Yes
>
>
>Gaaa. I meant no.

Nevertheless, I think you have been right in the first place.

Usually, MVCC prevents read conflict errors but "TempStorage"
only maintains a very limited history -- occasionally to small
to fully implement MVCC. Then read conflict errors will occur.



-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Jim Fulton]

On Jun 5, 2008, at 10:16 AM, Jim Fulton wrote:



On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote:


I would like to know:
- Can conflict errors appear when reading objects from the ZODB?


Yes



Gaaa. I meant no.

Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Tino Wildenhain]

Jim Fulton wrote:


On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote:


I would like to know:
- Can conflict errors appear when reading objects from the ZODB?


Yes


shouln't MVCC resolve this particular problem?

Cheers
Tino


smime.p7s
Description: S/MIME Cryptographic Signature
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Jim Fulton]

On Jun 5, 2008, at 9:52 AM, David Otero Figueroa wrote:


I would like to know:
- Can conflict errors appear when reading objects from the ZODB?


Yes



- Could conflict errors be the cause of a "spinning ZOPE" scenario?


Yes.



- Is there any situation where ZOPE could try to write objects when  
rendering an "only read" page?


Yes.  Sessions can often cause writes, depending on how you use them  
and on the session implementation you're using.



- How can they be solved?


You can set a break point in ZODB.Connection.register and then look at  
the call stack to see who's writing.



- Can I realize partial commits? How to?


I don't know what a "partial commit" is.  You can commit more often.  
This usually is a bad idea and wouldn't help if you don't intent to  
write in the first place.


Jim

--
Jim Fulton
Zope Corporation


___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce

http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Wichert Akkerman]

Previously David Otero Figueroa wrote:
>  Hello everyone,
> 
> 
> I have a couple questions related to "conflict errors".
> 
> During our home page stress testing (20 concurrent users), we detected
> several conflict errors (see trace below), We thought these could ONLY
> appear when writing objects in the ZODB.

That is correct. You need to audit your check to check for undesired
writes.

Wichert.

-- 
Wichert Akkerman <[EMAIL PROTECTED]>It is simple to make things.
http://www.wiggy.net/   It is hard to make things simple.
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 02:29:50PM -0500, Paul Winkler wrote:
> This is a topic that comes up occasionally, but since the symptom
> seems to be rare and hard to provoke, it never gets resolved.
> Well it's biting me again and I'm determined to fix it,
...

And since my ConflictError turned out to be merely caused by
a borked ZEO client cache, I've given up on the hunt ... again :-\

For all I know, ZEO cache may have been the cause of my previous 
run-ins with ConflictError on startup too.


-PW

-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Tim Peters]

>> ZODB.POSException.ConflictError: database conflict error (
>> serial this txn started with 0x035b25f36751f988 2005-02-10 18:59:24.215675,
>> serial currently committed 0x035b743c6d186822 2005-02-24 17:00:25.569220)
 
[Dieter Maurer]
> Where is the "oid" in this "ConflictError" report?
> It is vital information to analyse which object causes the problem...

I don't know. I reformatted Paul's traceback to line up the timestamps
for easier reading, but there was no oid in the original either. 
There wouldn't be, either, if no oid was passed to the ConflictError
constructor.  ConflictError.__str__ only includes info for the stuff
it was told about:

def __str__(self):
extras = []
if self.oid:
extras.append("oid %s" % oid_repr(self.oid))
if self.class_name:
extras.append("class %s" % self.class_name)
if self.serials:
current, old = self.serials
extras.append("serial this txn started with %s" %
  readable_tid_repr(old))
extras.append("serial currently committed %s" %
  readable_tid_repr(current))
if extras:
return "%s (%s)" % (self.message, ", ".join(extras))
else:
return self.message

In Paul's case, the ConflictError occurred on the ZEO server, but the
traceback came from the ZEO client (just raising the Exception object
the ZEO server  passed to it).  There's no way then to guess what the
traceback might have looked like on the ZEO server side (b!). 
Perhaps looking at the ZEO server log would have helped.
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Dieter Maurer]

Tim Peters wrote at 2005-2-24 15:29 -0500:
>[Paul Winkler]
>> ...
>> ZODB.POSException.ConflictError: database conflict error (
>> serial this txn started with 0x035b25f36751f988 2005-02-10 18:59:24.215675,
>> serial currently committed 0x035b743c6d186822 2005-02-24 17:00:25.569220)

Where is the "oid" in this "ConflictError" report?
It is vital information to analyse which object causes the problem...

-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Dieter Maurer]

Paul Winkler wrote at 2005-2-24 14:29 -0500:
>This is a topic that comes up occasionally, but since the symptom
>seems to be rare and hard to provoke, it never gets resolved.
>Well it's biting me again and I'm determined to fix it, but
>would like some input from the list.

I know that "ZopeProfiler" can cause such "ConflictError"s
and (what is more) why it does this.

"ZopeProfiler" (like some other products) wants to install
itself in "Control_Panel".

Because, this requires a database connection
and because "App.ProductContext.ProductContest" tries
hard to hide its application object,
"ZopeProfiler" opens a new one, modifies "Control_Panel"
and commits the transaction and then closes its connection.

Such modifications via a second ZODB connection
can cause unexpected (and puzzling) "ConflictError" on
startup.

The next "ZopeProfiler" version will access
"ProductContext.__app" instead (using the mangled name).

-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Dieter Maurer]

Paul Winkler wrote at 2005-2-24 15:16 -0500:
> ...
>Twiddling around with some old suggestions from Dieter
>Maurer (see
>http://aspn.activestate.com/ASPN/Mail/Message/Zope-CMF/1978842 ),
>I added a _p_jar.sync() where he suggested - right
>before the final "except:" in initialize_product().
>No joy.  

Note that the database is initialized in a separate connection
-- in case it is empty.
This may cause "ConflictError"s in the primary connection.


-- 
Dieter
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Chris Withers]

Tim Peters wrote:
I'm not really happy with that, because I have no theory for how ZEO
client cache files _get_ out of synch. 
The most common one for me is connecting a ZEO client to one storage 
server (say a dev one) and then changing the config to connect to 
another storage server (say a staging one) without making sure the cache 
files are gone. That seems ot do it most time, and I'm sure I've got 
POSKeyErrors and all manner of other weirdness too.

My workaround is never to use persistent client caches on clients that 
I'm likely to connect to different servers. This seems a shame, I wish 
there was some kind of safety check that ZEO could do to go "wait, these 
caches are insane, they don't even apply to the server you're connecting to"

cheers,
Chris
--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 04:11:35PM -0500, Tim Peters wrote:
> [Tim Peters]
> > ...
> > Do you use a persistent ZEO client cache?  If so, one thing to try is
> > to physically remove all the cache files -- maybe they've gotten into
> > an insane state.
> 
> Off list, Paul said "bingo!" to that one -- deleting his ZEO client
> cache files apparently made the problem go away.

Yep, meant to mail the list, hit the wrong button.
Thanks again Tim!
 
> I'm not really happy with that, because I have no theory for how ZEO
> client cache files _get_ out of synch.  I suspect they do, though,
> more often than stray gamma rays and pilot error could account for.
> 
> Alas, I've got no time to give to that these days.  If someone else
> wants to solve that, I'll pass on the cigar Paul offered to me .

Oh-sure-pretend-you-didn't-want-the-bag-of-ham-ly y'rs,

-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Tim Peters]

[Tim Peters]
> ...
> Do you use a persistent ZEO client cache?  If so, one thing to try is
> to physically remove all the cache files -- maybe they've gotten into
> an insane state.

Off list, Paul said "bingo!" to that one -- deleting his ZEO client
cache files apparently made the problem go away.

I'm not really happy with that, because I have no theory for how ZEO
client cache files _get_ out of synch.  I suspect they do, though,
more often than stray gamma rays and pilot error could account for.

Alas, I've got no time to give to that these days.  If someone else
wants to solve that, I'll pass on the cigar Paul offered to me .
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 03:16:15PM -0500, Paul Winkler wrote:
> But - duh - my traceback clearly shows the ConflictError
> happens before that, in the call to get_transaction().commit().
> SO I moved the call to app._p_jar.sync() to right before hte call
> to get_transaction().commit().
> Aaaand... no more ConflictError.

Never mind, this breaks existing unit tests.
Duh, it prevents products from getting initialized at all :-\

now checking out tim's suggestion...
 
-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Tim Peters]

[Paul Winkler]
> ...
> But - duh - my traceback clearly shows the ConflictError
> happens before that, in the call to get_transaction().commit().
> SO I moved the call to app._p_jar.sync() to right before hte call
> to get_transaction().commit().
> Aaaand... no more ConflictError.

Well, that's certain:  the first thing sync() does is abort the
current transaction.  Then of course commit() won't raise an
exception:  sync() threw away all the changes.

> ...
> If nobody objects, I wouldnt' mind checking the fix in anyway :-)

It would be much clearer instead to change the

get_transaction().commit().

to

get_transaction().abort()

That would get rid of the ConflictError too, and for the same reason. 
It has the advantage of being obviously insane, though .
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Tim Peters]

[Paul Winkler]
> ...

> ZODB.POSException.ConflictError: database conflict error (
> serial this txn started with 0x035b25f36751f988 2005-02-10 18:59:24.215675,
> serial currently committed 0x035b743c6d186822 2005-02-24 17:00:25.569220)

No time here for thought, but those timestamps are two weeks apart. 
Yikes!  Just thought I'd mention that .

A write conflict (which this is) happens when (and in this time order):

1. A transaction reads an object O.
2. A different transaction modifies O and commits its new state.
3. The original transaction also modifies O, and tries to commit its new state.

#3 fails, because the original transaction has, in general, no way to
know whether the changes it tried to make to O's state are consistent
with the changes the other transaction made to its state in step #2.

For your specific exception above, step #1 started by reading the
state of the object as it existed on 2005-02-10.  In step #2, someone
else modified the same object, and committed its state around
2005-02-24 17:00:25.  The original transaction then failed in step #3.

Do you use a persistent ZEO client cache?  If so, one thing to try is
to physically remove all the cache files -- maybe they've gotten into
an insane state.

It's not impossible that you "should" have a 2-week difference in
timestamps here, but it does look unusual.  It's certainly possible
(even expected) if two transactions try to modify the same object, and
the last time that object was changed was in fact two weeks ago.
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 02:58:01PM -0500, Chris McDonough wrote:
> On Thu, 2005-02-24 at 14:57, Paul Winkler wrote:
> > On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote:
> > > Two words: *hurt system*.  Please, let me kill it.
> > 
> > Interesting. How on earth did you determine that?
> 
> Womanly intuition.
> 
> > 
> > You think it has to do with registering help for
> > PluginIndexes? I'm gonna go test that theory -
> > i'll hack registerHelp() to no-op.
> 
> That's a good verifier.

Well, no joy.  I hacked ProductContext.register* to
return from the first line, so even registerClass()
now does nothing.  Barf, I still get the ConflictError.

Twiddling around with some old suggestions from Dieter
Maurer (see
http://aspn.activestate.com/ASPN/Mail/Message/Zope-CMF/1978842 ),
I added a _p_jar.sync() where he suggested - right
before the final "except:" in initialize_product().
No joy.  

But - duh - my traceback clearly shows the ConflictError
happens before that, in the call to get_transaction().commit().
SO I moved the call to app._p_jar.sync() to right before hte call
to get_transaction().commit().
Aaaand... no more ConflictError.

Weird weird weird.

I'd be happy to check this in to cvs / svc, except that
A) I have no clue how to write a test that exercises the bug,
and B) so far nobody seems to understand what's really happening.

If nobody objects, I wouldnt' mind checking the fix in anyway :-)


-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Chris McDonough]

On Thu, 2005-02-24 at 14:57, Paul Winkler wrote:
> On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote:
> > Two words: *hurt system*.  Please, let me kill it.
> 
> Interesting. How on earth did you determine that?

Womanly intuition.

> 
> You think it has to do with registering help for
> PluginIndexes? I'm gonna go test that theory -
> i'll hack registerHelp() to no-op.

That's a good verifier.

- C


>  

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 02:52:45PM -0500, Chris McDonough wrote:
> Two words: *hurt system*.  Please, let me kill it.

Interesting. How on earth did you determine that?

You think it has to do with registering help for
PluginIndexes? I'm gonna go test that theory -
i'll hack registerHelp() to no-op.
 
-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Paul Winkler]

On Thu, Feb 24, 2005 at 02:29:50PM -0500, Paul Winkler wrote:
> * Is the PIL stuff in the traceback a red herring or what?

Note that PIL is not referred to by anything in my Products
directory. (Heck, *nothing* is referred to by anything in Products.)
That's what's so baffling about it appearing in the traceback.
 
-- 

Paul Winkler
http://www.slinkp.com
___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on startup, revisited

[Chris McDonough]

Two words: *hurt system*.  Please, let me kill it.

On Thu, 2005-02-24 at 14:29, Paul Winkler wrote:
> This is a topic that comes up occasionally, but since the symptom
> seems to be rare and hard to provoke, it never gets resolved.
> Well it's biting me again and I'm determined to fix it, but
> would like some input from the list.
> 
> Related collector issues:
> http://www.zope.org/Collectors/Zope/300
> http://www.zope.org/Collectors/Zope/1201
> 
> The issue:
> On rare occasions I have seen ConflictErrors raised during
> product installation.
> 
> Today, I can finally provoke them at will with a pretty minimal setup.
> 
> The setup:
> Zope 2.7.3, using ZEO, with DirectoryStorage on the ZEO server.
> Hardware:  dual-processor PIII 800, some kinda scsi disk, ext3, 
> some old redhat flavor of linux.
> 
> To provoke it I need merely do
> ./bin/zopectl run foo.py
> 
> ... where foo.py is empty.
> Products/ now contains only one dummy product,
> which I've named Conflictor, which contains only an empty __init__.py.
> I get a ConflictError during registration of Conflictor.
> 
> Interestingly I can only provoke this ConflictError in this particular
> INSTANCE_HOME.  I have several other instances and am not seeing the
> symptom there.
> 
> I started trivially hacking OFS/Application.py to add some debugging output
> (a diff against 2.7.3 is attached), and here's what I get:
> 
> 
> [EMAIL PROTECTED] PaulMiniInstanceHome]$ ls Products/
> [EMAIL PROTECTED] PaulMiniInstanceHome]$ mkdir Products/Conflicter
> [EMAIL PROTECTED] PaulMiniInstanceHome]$ touch Products/Conflicter/__in
> it__.py
> [EMAIL PROTECTED] PaulMiniInstanceHome]$ touch foo.py
> [EMAIL PROTECTED] PaulMiniInstanceHome]$ ./bin/zopectl run foo.py
> installing product PluginIndexes...OK!
> installing product Conflicter... BZZT
> Traceback (most recent call last):
>   File "", line 1, in ?
>   File "/usr/local/lib/python2.3/site-packages/PIL/__init__.py", line 51, in 
> app
> 
> 
>   File "/usr/local/lib/python2.3/site-packages/PIL/__init__.py", line 47, in 
> startup
> 
>   File "/zope/Zope-2.7.3-SoftwareHome/lib/python/Zope/App/startup.py", line 
> 96,
> in startup
>   File "/zope/ZopeSoftwareHome/lib/python/OFS/Application.py", line 279, in 
> init
> ialize
> initializer.initialize()
>   File "/zope/ZopeSoftwareHome/lib/python/OFS/Application.py", line 306, in 
> init
> ialize
> self.install_products()
>   File "/zope/ZopeSoftwareHome/lib/python/OFS/Application.py", line 561, in 
> inst
> all_products
> return install_products(app)
>   File "/zope/ZopeSoftwareHome/lib/python/OFS/Application.py", line 597, in 
> inst
> all_products
> folder_permissions, raise_exc=debug_mode)
>   File "/zope/ZopeSoftwareHome/lib/python/OFS/Application.py", line 803, in 
> inst
> all_product
> get_transaction().commit()
>   File "/zope/Zope-2.7.3-SoftwareHome/lib/python/ZODB/Transaction.py", line 
> 252,
>  in commit
>   File "/zope/Zope-2.7.3-SoftwareHome/lib/python/ZODB/Connection.py", line 
> 738,
> in tpc_vote
>   File "/zope/Zope-2.7.3-SoftwareHome/lib/python/ZEO/ClientStorage.py", line 
> 841
> , in tpc_vote
>   File "/zope/Zope-2.7.3-SoftwareHome/lib/python/ZEO/ClientStorage.py", line 
> 825
> , in _check_serials
> ZODB.POSException.ConflictError: database conflict error (serial this txn 
> starte
> d with 0x035b25f36751f988 2005-02-10 18:59:24.215675, serial currently 
> committed
>  0x035b743c6d186822 2005-02-24 17:00:25.569220)
> 
> 
> I guess what I'm asking, before I hack more on OFS/Application.py,
> is this:
> 
> * any ideas re. what on earth could be causing this ConflictError?
>   the product doesn't DO anything fer chrissakes.
>   (Maybe registration is finishing too fast??)
> 
> * Is the PIL stuff in the traceback a red herring or what?
> 

___
Zope-Dev maillist  -  Zope-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Dieter Maurer]

Gerhard Schmidt wrote at 2004-5-17 21:01 +0200:
> ...
>> However, due to a ZEO weakness, its conflict resolution
>> currently does not work (ZEO no longer supports "INSTANCE_HOME").
>
>Is that a bug or intetional. 

I expect it to be a bug.

I plan to fix it soon -- but currently, I have lots of things to do.
May take some months...

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Chris McDonough]

Hi Dr. Wagner,

> thanks for the hint with minimal storage. however the same problem 
> occurs with filestorage and is even worse.
> 
> we know very well, that we cannot avoid the appearance of conflict 
> errors in particular with zeo, and we understand what causes them.
> 
> what wonders us is, that we find some contributions about this errors in 
> the mailing lists, but no one seems to have a real problem with it. but 
> we have! we are not worried about some errors appearing in a log, but we 
> are using plone2 witch currently 30 concurrent authenticated users, so 
> that this error occurs permanently, thus decreasing system performance 
> due to several retries and moreover leading to very frequent nasty 
> 'POS.Exception Errors' in the browser, whenever four successive tries 
> fail (almost every 100. request).
> 
> this is not acceptable, and i  wonder wheter there are any plone sites 
> with more than a handful of authenticated users that experience the same 
> problem.  we found that one (there are more) product responsible for it 
> is the placeless translation service, which heavily writes to the session.

Are you using ZEO for your session storage or are you using a local
storage?  (You hint at it above but you don't actually say one way or
the other.)

> so we desperately need a solution for this problem, maybe uncouple the 
> session of the locking mechanism or whatever, and this should be a 
> problem that should be addressed by the core team. because this is a 
> strong reason (and so far the only one) to avoid using zope/plone in a 
> large user environment.

I also need reliable sessioning for my own customers, so I am working
towards fixing the issues I know about.  But the issues I know about may
not be the same issues as those that you want fixed.  You can ensure
your issues are addressed by devoting resources towards the problem, be
that time+expertise or money or both.

- C



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Michael Dunstan]

On 18/05/2004, at 6:57 AM, Gerhard Schmidt wrote:
On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote:
AFAIK Zope is the only appserver that even makes an attempt at 
allowing
session data to be shared between servers like this.  Most other
appservers require you to set up "session affinity" in your load
balancer and have a session database on each appserver.
and nealy every system failed to do so. How do you implement this
session affinity. Based on source IP. At the moment every large ISP
uses multiple Proxyserver so every request comes from a different
IP so the affinity doesn´t work for them. The Proxyserver provided by
our Computing Center is also split on at least 8 Hosts, so everyone
using this proxy would lose session infomation, getting very strange
effekts.
Perhaps you can get some relief by using pound as your load balancer. 
It is quite capable at sniffing out user sessions including watching 
the Cookie header. See http://www.apsis.ch/pound/

Michael
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Dr. Thomas Wagner]

Hi Chris, hi folks.
thanks for the hint with minimal storage. however the same problem 
occurs with filestorage and is even worse.

we know very well, that we cannot avoid the appearance of conflict 
errors in particular with zeo, and we understand what causes them.

what wonders us is, that we find some contributions about this errors in 
the mailing lists, but no one seems to have a real problem with it. but 
we have! we are not worried about some errors appearing in a log, but we 
are using plone2 witch currently 30 concurrent authenticated users, so 
that this error occurs permanently, thus decreasing system performance 
due to several retries and moreover leading to very frequent nasty 
'POS.Exception Errors' in the browser, whenever four successive tries 
fail (almost every 100. request).

this is not acceptable, and i  wonder wheter there are any plone sites 
with more than a handful of authenticated users that experience the same 
problem.  we found that one (there are more) product responsible for it 
is the placeless translation service, which heavily writes to the session.

so we desperately need a solution for this problem, maybe uncouple the 
session of the locking mechanism or whatever, and this should be a 
problem that should be addressed by the core team. because this is a 
strong reason (and so far the only one) to avoid using zope/plone in a 
large user environment.

i hope some of the community sees the importance of this.
Regards
Thomas

On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
 

Hi,
whe are running portal unter zope2.7 with one zeo server and six frontend 
application servers. 

I have set up zeo to server two storages

 path $INSTANCE_HOME/var/Data.fs


 envdir $INSTANCE_HOME/var/session

temp should provide an shared session. But when is set up the frontends 
to use temp as /temp_folder, we see many  ZODB conflict errors. The 
performance decreases and PosErrors appear quite often. 
   

BDBMininalStorage is no longer maintained and doesn't do any "conflict
resolution".  Even though you'll need to pack frequently, you might want
to try a FileStorage instead.
Regardless, using ZEO will presumably always generate more conflict
errors than using a "local" ZODB because the commit time is longer, and
thus there's a better chance that two threads will be committing at the
same time.
- C
 


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Chris McDonough]

On Mon, 2004-05-17 at 14:57, Gerhard Schmidt wrote:
> > > Why isnÂt there an implementation of temporary storage for ZEO
> > 
> > There is, just use the "tempstorage" package from Zope.
> 
> I didnÂt find a way to use Tempstorage in zeo. 
> 
> 
>name temporary storage for sessioning
> 
> 
> didnÂt work. How do i Set up a temporarystorage in zeo. 

Good question. ;-)  It involves changing the ZEO configuration machinery
to recognize the tempstorage package.  The steps to do this I couldn't
tell you without actually doing it, however.

> > > I dont care about conflit resolution. Its only session data.
> > 
> > For better or worse, because it is based on ZODB, the implementation
> > makes you need to care.
> 
> Than maybe the sessioncode should be redesigned not to use ZODB as 
> storage but store ther information in some different system maybe 
> a RDMB or some other system able to handle the kind of usage. 

Zope doesn't ship with a useful and maintained relational database.  You
can implement your own SQL-based TransientObjectContainer using the
interfaces defined in Transience/Interfaces.py if you wish, or use
Anthony Baxter's SQLSession product.  It would be nice if there was an
alternate implementation that used a relational database like this.

> > > I simply 
> > > need a shared session storage. But as Far as i can see the is no way 
> > > of doing this.

Of course there is, it's just not built in. ;-)

> > 
> > I don't understand.  You seem to have it working.  Is performance the
> > issue?
> 
> The Performance decrase is there but the Problem is far deeper. If 
> an read conflict isnÂt resolved after three retries an Exception is 
> raised an the request is aborted. And thats the real problem. 

I see.  I'm not sure switching to TemporaryStorage will help much.  I
would first try using a FileStorage and make sure to put the Zope
"lib/python" directory on the ZEO server's Python's PYTHONPATH (to allow
for conflict resolution under ZEO, ZEO must be able to import the
Products package and subpackages therein).  I don't know if this will
help very much, but it's worth a shot.

> > >  Are we the only site out there running Plone2 (Placeless
> > > transaltion Service) at that scale. How do others handle there Session 
> > 
> > AFAIK Zope is the only appserver that even makes an attempt at allowing
> > session data to be shared between servers like this.  Most other
> > appservers require you to set up "session affinity" in your load
> > balancer and have a session database on each appserver.
>  
> and nealy every system failed to do so. How do you implement this 
> session affinity. Based on source IP. At the moment every large ISP 
> uses multiple Proxyserver so every request comes from a different 
> IP so the affinity doesnÂt work for them. The Proxyserver provided by 
> our Computing Center is also split on at least 8 Hosts, so everyone 
> using this proxy would lose session infomation, getting very strange 
> effekts. 

Yes.  It's a hard problem.

- C



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Gerhard Schmidt]

On Mon, May 17, 2004 at 07:48:22PM +0200, Dieter Maurer wrote:
> Gerhard Schmidt wrote at 2004-5-13 11:23 +0200:
> > ...
> >Is there any why to get a shared session working. 
> 
> You know that the ZODB is not designed for high frequency
> concurrent writes?
> 
> When you want to reduce conflicts you will need to try hard
> to minimize concurrent writes.

Thats not that easy. Because we use many Thrid Party Products many of 
them use session quite often (namely Placeless Translation Service). 

> I have a (non-open) "Transience" implementation that
> writes much less then Zope's standard implementation.
> However, due to a ZEO weakness, its conflict resolution
> currently does not work (ZEO no longer supports "INSTANCE_HOME").

Is that a bug or intetional. 

Bye
Estartu 

--
Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | [EMAIL PROTECTED] |   auf Anfrage/
Tel: 08232 77 36 4 ||on request 
Fax: 08232 77 36 3 || 


pgpMh1q5XACbt.pgp
Description: PGP signature
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Gerhard Schmidt]

On Mon, May 17, 2004 at 01:38:15PM -0400, Chris McDonough wrote:
> On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote:
> > > BDBMininalStorage is no longer maintained and doesn't do any "conflict
> > > resolution".  
> > 
> > Why this. I was told some time ago we should use BDBMinimalStorage to 
> > fix the conflict problems. 
> 
> When was that and who told you so?
> 
> > > Even though you'll need to pack frequently, you might want
> > > to try a FileStorage instead.
> > 
> > There is no difference between BDB Storage and Filestorage regarding 
> > the conflicts. 
> 
> OK.
> 
> > > Regardless, using ZEO will presumably always generate more conflict
> > > errors than using a "local" ZODB because the commit time is longer, and
> > > thus there's a better chance that two threads will be committing at the
> > > same time.
> > 
> > The bottom line is that there is no way of getting an shared session 
> > running.
> 
> It works doesn't it?  You just get conflict errors, right?  Conflict
> errors are normal at a certain level.

No It doesn`t. At the Moment we run with single sessions on each frontend
we have rewitten most ob the code using sessions to store ther information 
in cockies. But that´s not realy a good solution. 

> > Why isn´t there an implementation of temporary storage for ZEO
> 
> There is, just use the "tempstorage" package from Zope.

I didn´t find a way to use Tempstorage in zeo. 


   name temporary storage for sessioning


didn´t work. How do i Set up a temporarystorage in zeo. 

> > I dont care about conflit resolution. Its only session data.
> 
> For better or worse, because it is based on ZODB, the implementation
> makes you need to care.

Than maybe the sessioncode should be redesigned not to use ZODB as 
storage but store ther information in some different system maybe 
a RDMB or some other system able to handle the kind of usage. 

> > I simply 
> > need a shared session storage. But as Far as i can see the is no way 
> > of doing this.
> 
> I don't understand.  You seem to have it working.  Is performance the
> issue?

The Performance decrase is there but the Problem is far deeper. If 
an read conflict isn´t resolved after three retries an Exception is 
raised an the request is aborted. And thats the real problem. 

> >  Are we the only site out there running Plone2 (Placeless
> > transaltion Service) at that scale. How do others handle there Session 
> 
> AFAIK Zope is the only appserver that even makes an attempt at allowing
> session data to be shared between servers like this.  Most other
> appservers require you to set up "session affinity" in your load
> balancer and have a session database on each appserver.
 
and nealy every system failed to do so. How do you implement this 
session affinity. Based on source IP. At the moment every large ISP 
uses multiple Proxyserver so every request comes from a different 
IP so the affinity doesn´t work for them. The Proxyserver provided by 
our Computing Center is also split on at least 8 Hosts, so everyone 
using this proxy would lose session infomation, getting very strange 
effekts. 

Bye
Estartu

--
Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | [EMAIL PROTECTED] |   auf Anfrage/
Tel: 08232 77 36 4 ||on request 
Fax: 08232 77 36 3 || 


pgpsBoo3UooYb.pgp
Description: PGP signature
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Dieter Maurer]

Gerhard Schmidt wrote at 2004-5-13 11:23 +0200:
> ...
>Is there any why to get a shared session working. 

You know that the ZODB is not designed for high frequency
concurrent writes?

When you want to reduce conflicts you will need to try hard
to minimize concurrent writes.

I have a (non-open) "Transience" implementation that
writes much less then Zope's standard implementation.
However, due to a ZEO weakness, its conflict resolution
currently does not work (ZEO no longer supports "INSTANCE_HOME").

-- 
Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Chris McDonough]

On Mon, 2004-05-17 at 13:06, Gerhard Schmidt wrote:
> > BDBMininalStorage is no longer maintained and doesn't do any "conflict
> > resolution".  
> 
> Why this. I was told some time ago we should use BDBMinimalStorage to 
> fix the conflict problems. 

When was that and who told you so?

> > Even though you'll need to pack frequently, you might want
> > to try a FileStorage instead.
> 
> There is no difference between BDB Storage and Filestorage regarding 
> the conflicts. 

OK.

> > Regardless, using ZEO will presumably always generate more conflict
> > errors than using a "local" ZODB because the commit time is longer, and
> > thus there's a better chance that two threads will be committing at the
> > same time.
> 
> The bottom line is that there is no way of getting an shared session 
> running.

It works doesn't it?  You just get conflict errors, right?  Conflict
errors are normal at a certain level.

> Why isnÂt there an implementation of temporary storage for ZEO

There is, just use the "tempstorage" package from Zope.
 
> I dont care about conflit resolution. Its only session data.

For better or worse, because it is based on ZODB, the implementation
makes you need to care.

> I simply 
> need a shared session storage. But as Far as i can see the is no way 
> of doing this.

I don't understand.  You seem to have it working.  Is performance the
issue?

>  Are we the only site out there running Plone2 (Placeless
> transaltion Service) at that scale. How do others handle there Session 

AFAIK Zope is the only appserver that even makes an attempt at allowing
session data to be shared between servers like this.  Most other
appservers require you to set up "session affinity" in your load
balancer and have a session database on each appserver.

- C



___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Gerhard Schmidt]

On Mon, May 17, 2004 at 12:45:16PM -0400, Chris McDonough wrote:
> On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
> > Hi,
> > 
> > whe are running portal unter zope2.7 with one zeo server and six frontend 
> > application servers. 
> > 
> > I have set up zeo to server two storages
> > 
> > 
> >   path $INSTANCE_HOME/var/Data.fs
> > 
> > 
> > 
> >   envdir $INSTANCE_HOME/var/session
> > 
> > 
> > temp should provide an shared session. But when is set up the frontends 
> > to use temp as /temp_folder, we see many  ZODB conflict errors. The 
> > performance decreases and PosErrors appear quite often. 
> 
> BDBMininalStorage is no longer maintained and doesn't do any "conflict
> resolution".  

Why this. I was told some time ago we should use BDBMinimalStorage to 
fix the conflict problems. 

> Even though you'll need to pack frequently, you might want
> to try a FileStorage instead.

There is no difference between BDB Storage and Filestorage regarding 
the conflicts. 

> Regardless, using ZEO will presumably always generate more conflict
> errors than using a "local" ZODB because the commit time is longer, and
> thus there's a better chance that two threads will be committing at the
> same time.

The bottom line is that there is no way of getting an shared session 
running. Why isn´t there an implementation of temporary storage for ZEO 
I dont care about conflit resolution. Its only session data. I simply 
need a shared session storage. But as Far as i can see the is no way 
of doing this. Are we the only site out there running Plone2 (Placeless
transaltion Service) at that scale. How do others handle there Session 

Bye
Estartu 

--
Gerhard Schmidt| Nick : estartu  IRC : Estartu  |
Fischbachweg 3 ||  PGP Public Key
86856 Hiltenfingen | [EMAIL PROTECTED] |   auf Anfrage/
Tel: 08232 77 36 4 ||on request 
Fax: 08232 77 36 3 || 


pgpqXUCWUVwl0.pgp
Description: PGP signature
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict errors on BDBMinimal storage

[Chris McDonough]

On Thu, 2004-05-13 at 05:23, Gerhard Schmidt wrote:
> Hi,
> 
> whe are running portal unter zope2.7 with one zeo server and six frontend 
> application servers. 
> 
> I have set up zeo to server two storages
> 
> 
>   path $INSTANCE_HOME/var/Data.fs
> 
> 
> 
>   envdir $INSTANCE_HOME/var/session
> 
> 
> temp should provide an shared session. But when is set up the frontends 
> to use temp as /temp_folder, we see many  ZODB conflict errors. The 
> performance decreases and PosErrors appear quite often. 

BDBMininalStorage is no longer maintained and doesn't do any "conflict
resolution".  Even though you'll need to pack frequently, you might want
to try a FileStorage instead.

Regardless, using ZEO will presumably always generate more conflict
errors than using a "local" ZODB because the commit time is longer, and
thus there's a better chance that two threads will be committing at the
same time.

- C




___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors; how to track them down?

[Jamie Heilman]

Chris Withers wrote:
> Dieter Maurer wrote:
> >
> >The attached patch to "Zope/App/startup.py" provides this
> >additional information.
> 
> Where's the patch?

http://marc.theaimsgroup.com/?l=zope-dev&m=105466926610469&q=p3

-- 
Jamie Heilman   http://audible.transient.net/~jamie/
"I was in love once -- a Sinclair ZX-81.  People said, "No, Holly, she's 
 not for you." She was cheap, she was stupid and she wouldn't load 
 -- well, not for me, anyway."  -Holly

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors; how to track them down?

[Chris Withers]

Dieter Maurer wrote:
The attached patch to "Zope/App/startup.py" provides this
additional information.
Where's the patch?

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors; how to track them down?

[Dieter Maurer]

Bjorn Stabell wrote at 2003-6-3 11:08 +0800:
 > Except for SARS, another problem that's been plagueing us for months are
 > seemingly random Conflict Errors.  We see about 10 every day on our Zope
 > 2.6.1, and they can happen on any page.  In most cases they are not
 > related to pages that actually updated anything (AFAIK), although we do
 > use SESSIONS (but no frames).  Any clue how to track down which objects
 > are having problems, what's causing this?
 > 
 > --
 > 2003-06-03T09:08:10 INFO(0) ZODB conflict error at
 > /VirtualHostBase/http/www.beijingsammies.com:80/sammies/VirtualHostRoot/
 > (7 conflicts since startup at 2003-06-02T13:15:05) 

The log message for conflict errors is far less instructive than
it could be (almost without any effort).

  It should indicate whether it is a ReadConflictError or
  a (Write)ConflictError.

  It should tell which object caused the conflict.


The attached patch to "Zope/App/startup.py" provides this
additional information.

Please file a feature request to "".


Dieter

--- startup.py~	2003-02-28 16:59:25.0 +0100
+++ startup.py	2003-03-17 11:13:28.0 +0100
@@ -140,9 +140,9 @@
 global conflict_errors
 conflict_errors = conflict_errors + 1
 method_name = REQUEST.get('PATH_INFO', '')
-err = ('ZODB conflict error at %s '
+err = ('ZODB conflict error at %s: %s'
'(%s conflicts since startup at %s)')
-LOG(err % (method_name, conflict_errors, startup_time),
+LOG(err % (method_name, str(v), conflict_errors, startup_time),
 INFO, '')
 LOG('Conflict traceback', BLATHER, '', error=sys.exc_info())
 raise ZPublisher.Retry(t, v, traceback)

Re: [Zope-dev] Conflict Errors; how to track them down?

[Shane Hathaway]

Bjorn Stabell wrote:
In any case, is there no way to find out more detailed information about
why a conflict happened?  There must be some oids somewhere?
I don't know of a way.  In a conflict, the transaction gets aborted, so 
the only data left is the log entry.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

RE: [Zope-dev] Conflict Errors; how to track them down?

[Bjorn Stabell]

On 06/02/2003 11:08 PM, Bjorn Stabell wrote:
> > Except for SARS, another problem that's been plagueing us for months

> > are seemingly random Conflict Errors.  We see about 10 every day on 
> > our Zope 2.6.1, and they can happen on any page.  In most  cases
they 
> > are not related to pages that actually updated anything (AFAIK), 
> > although we do use SESSIONS (but no frames).  Any clue how to track 
> > down which objects are having problems, what's causing this?

Shane wrote:
> Look at your undo log.  If the conflicts don't correspond 
> with entries in the undo log, and you're not mounting other databases,
the 
> conflicts  are almost certainly due to sessions.  However, conflicts
are 
> a normal  occurrence and rarely reach the user.  Zope automatically
retries the 
> request and delivers the successful page.

They do not always correspond to entries in the undo log.  But the ones
that do are always (?) committed successfully.  The problem is, as I can
see from other threads here, that they are related to other systems that
cannot be rolled back (e.g., sending email / faxes), so we get duplicate
emails / faxes.  I'll look at how to wrap these systems in
transaction-aware wrappers, as was mentioned in a related thread.

In any case, is there no way to find out more detailed information about
why a conflict happened?  There must be some oids somewhere?

Regards,
-- 
Bjorn

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists -
 http://mail.zope.org/mailman/listinfo/zope-announce
 http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors; how to track them down?

[Shane Hathaway]

On 06/02/2003 11:08 PM, Bjorn Stabell wrote:
Hi all,

Except for SARS, another problem that's been plagueing us for months are
seemingly random Conflict Errors.  We see about 10 every day on our Zope
2.6.1, and they can happen on any page.  In most cases they are not
related to pages that actually updated anything (AFAIK), although we do
use SESSIONS (but no frames).  Any clue how to track down which objects
are having problems, what's causing this?
Look at your undo log.  If the conflicts don't correspond with entries 
in the undo log, and you're not mounting other databases, the conflicts 
are almost certainly due to sessions.  However, conflicts are a normal 
occurrence and rarely reach the user.  Zope automatically retries the 
request and delivers the successful page.

Shane

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Dieter Maurer]

Chris Withers writes:
 > However, if anyone does know what constitutes a 'seperate object' in
 > ZODB terms, it be really useful to know.
 > I'm guessing a class which inherits from Persistent? Dictionaries don't,
 > 'cos they're python builtins, which I'm pretty certain was the problem
 > in my case...
This has recently been answered in the list.
Here a short summary:

  a persistent object with all its attributes.
  If the attribute is itself a persistent object, then
  the "separate" object contains only a reference to it.
  Otherwise, the complete attribute value (until persistent objects
  are hit) is part of the object's value.


Dieter

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Toby Dickenson]

On Tue, 02 Jan 2001 16:50:49 +, Chris Withers <[EMAIL PROTECTED]>
wrote:


>However, if anyone does know what constitutes a 'seperate object' in
>ZODB terms, it be really useful to know.
>I'm guessing a class which inherits from Persistent? Dictionaries don't,
>'cos they're python builtins, which I'm pretty certain was the problem
>in my case...

That is exactly correct.

Toby Dickenson
[EMAIL PROTECTED]

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Chris McDonough]

Y'know since replying to this message originally, I've learned a lot about
pickled representations of Zope objects and I think I can answer this more
cogently:

- Instances of classes which inherit from Persistence.Persistent are
represented as an individual pickle.  Their attributes are contained in this
pickle, unless those attributes are themselves instances of something that
inherits from Persistence.Persistent.

- References to persistent objects in other objects are kept as an object
identifier.  This means, for example, that a dictionary of integers to
persistent object instances looks something like this in a pickle::

  {1:<>, 2:<>, ...}

  If this dictionary is an attribute of a persistent object, adding,
changing, or removing items from it will modify its containing persistent
object.

It was this "reference by oid" concept that turned the light on for me here,
hopefully it can do the same for you.

One of the potential big wins with the current Zope implementation of a
BTree is that it exposes a mapping interface, so you can treat it just like
a dictionary.  However, internally, it's composed of a number of separate
persistent objects that can be changed independently of each other.  This
means that changing one item in a BTree will change one persistent object
that makes up a BTree, but will not necessarily effect the other persistent
objects that compose it.  Unortunately, currently, the BTree implementation
has some problems that cause that *not* to be the case in many situations,
and independent writes to it often conflict.  We are working on another
BTree implementation to resolve the issue.

- Original Message -
From: "Chris Withers" <[EMAIL PROTECTED]>
To: "Chris McDonough" <[EMAIL PROTECTED]>
Cc: <[EMAIL PROTECTED]>
Sent: Tuesday, January 02, 2001 11:50 AM
Subject: Re: [Zope-dev] Conflict Errors


> Chris McDonough wrote:
> >
> > The ZODB tries
> > hard to avoid error conditions for you by retrying conflicting writes,
but
> > if you really pound the snot out of a particular object, it'll let you
know.
>
> That's gotta be one for the Zope quotes board, if I could onyl remember
> the URL :-)
>
> > and step through a real-world ConflictError situation.  Neither of which
I
> > have time to do right now... sorry, Chris.
> >
> > this one by saying "I don't know" because it's one of those things where
> > learning exactly what is going on can chew up time like nobody's
business,
> > because I'd need to step through the code line-by-line.  I shouldn't
even be
> > in here.  :-(
>
> Hey, no problem, the help was much appreciated :-)
>
> However, if anyone does know what constitutes a 'seperate object' in
> ZODB terms, it be really useful to know.
> I'm guessing a class which inherits from Persistent? Dictionaries don't,
> 'cos they're python builtins, which I'm pretty certain was the problem
> in my case...
>
> cheers,
>
> Chris
>


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Chris Withers]

Chris McDonough wrote:
> 
> The ZODB tries
> hard to avoid error conditions for you by retrying conflicting writes, but
> if you really pound the snot out of a particular object, it'll let you know.

That's gotta be one for the Zope quotes board, if I could onyl remember
the URL :-)

> and step through a real-world ConflictError situation.  Neither of which I
> have time to do right now... sorry, Chris.
> 
> this one by saying "I don't know" because it's one of those things where
> learning exactly what is going on can chew up time like nobody's business,
> because I'd need to step through the code line-by-line.  I shouldn't even be
> in here.  :-(

Hey, no problem, the help was much appreciated :-)

However, if anyone does know what constitutes a 'seperate object' in
ZODB terms, it be really useful to know.
I'm guessing a class which inherits from Persistent? Dictionaries don't,
'cos they're python builtins, which I'm pretty certain was the problem
in my case...

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Chris McDonough]

> Hurm, well, it appears to happen when we're doing a lot of sequential
> write to a product (automated bulk data upload), but the thing that
> triggers it off is actually trying to read a page from the same area of
> the site... confusing. Although that could also be a one-off
> coincidence...

Conflict errors can be raised by three things:  the storage, the
transaction, and the connection.  The storage will only raise the conflict
error on a write, this is for sure.  The transaction and the connection
objects I looked at a little today, and it's unclear .. although as I
understand it, the idea of a ConflictError is to prevent the modification of
an object by two threads at the same time, and should have nothing to do
with reads.

> You say no corruption or anything, but if a submit results in that
> error, does the submitted form data get processed or not?

No... the transaction is aborted...

> > Careful application coding can reduce the chance of conflict errors.
>
> Can you describe what you mean by careful application coding?

Ah I knew I wasn't gonna get away with that one.  ;-)  I probably shoulda
kept my mouth shut.

If you're writing a ZODB-based app that uses multiple database connections
(like Zope ;-), you should take care to stagger high-load writes against an
object in the threads that use those database connections.  The ZODB tries
hard to avoid error conditions for you by retrying conflicting writes, but
if you really pound the snot out of a particular object, it'll let you know.

That said, I doubt I can be more specific, because to be honest I don't know
the *real* answer myself.  Sigh.  I think I need to learn more about
pickling (to figure out how the objects get split up into component pieces)
and step through a real-world ConflictError situation.  Neither of which I
have time to do right now... sorry, Chris.

> > I bet it happens on a catalog write
> > for you.
>
> No, for once it has nothing to do with the catalog. ;-)
> This is a python product of ours that stores lots of data in somewhat
> nested, persistent dictionaries. How can this be improved so this
> doesn't happen?

Can you replicate the error on demand?  Or is it intermittent?  This should
be something covered in developer docs (along with a zillion other things).

I suppose a gross hack would be to raise the number of write retries on
conflicts in the ZODB code if you didn't want to analyze your app code.

But y'know, as I think more about it, I'm afraid I'm gonna need to bail on
this one by saying "I don't know" because it's one of those things where
learning exactly what is going on can chew up time like nobody's business,
because I'd need to step through the code line-by-line.  I shouldn't even be
in here.  :-(

Sorry,

C


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Toby Dickenson]

On Tue, 21 Nov 2000 17:03:17 +, Chris Withers <[EMAIL PROTECTED]>
wrote:

>> It happens when more than one thread tries to modify the same object in
>> storage at the same time.  It's Zope's equivalent of record-locking... It's
>> normal.  There's no corruption or anything, it's telling you that it avoided
>> a write that might have caused problems.  Zope tries three times to retry
>> the write.  If the object is still locked by another database connection
>> after three writes, it propogates the exception up to the app level.  That's
>> the error you see.
>
>Hurm, well, it appears to happen when we're doing a lot of sequential
>write to a product (automated bulk data upload), but the thing that
>triggers it off is actually trying to read a page from the same area of
>the site... confusing. Although that could also be a one-off
>coincidence...

ZODB marks each persistent object that is modified during a
transaction. Before committing, it checks whether any of those objects
have been modified-and-commited in another transaction. If it has, it
raises a ConflictError.

If your 'page read' really doesnt modify any objects (check your undo
log, or use tranalyzer) then it cant cause a ConflictError. 

>You say no corruption or anything, but if a submit results in that
>error, does the submitted form data get processed or not?

As Chris explianed, the publisher will retry up to three
ConflictErrors. If you see this message only in a log then it probably
suceeded on one of those retries.

If you see it in a traceback returned over http then the transaction
has been aborted. No corruption, however your request has not been
processed.

>> Careful application coding can reduce the chance of conflict errors.  
>
>Can you describe what you mean by careful application coding?

"Minimise the chance of a single persistent object being modified by
two concurrent transactions."

Some examples:

1. The much-maligned ZODB-page-hit-counter is bad in this respect.
Each 'read' of a page modifies the counter object, causing the second
concurrent read (which is actually a write) to raise ConflictError.

2. Folder objects are pretty good, although you cant concurrently add
two objects to a folder since both transactions modify the folder
object.

3. If you need something folderlike with a higher hit rate, it is
possible to avoid some conflicts by splitting the folder. For example,
having one sub-folder for each initial letter of the id. You can add
objects 'chrisw' and 'tdickenson' concurrently (into subfolders with
the id 'c' and 't'), however concurrently adding 'chrisw' and 'chrism'
would cause one ConflictErrorm because they both modify the sub-folder
'c'.

(BTreeFolders have the same advantage, for the same reason, only
neater)



Toby Dickenson
[EMAIL PROTECTED]

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Ender]

Chris Withers wrote:
> 

i'm no expert but deeply nested data structures in a single object are
rife with possibilities for conflict errors. if data might be changing
concurrently in the structures or frequently changing you would be
better off to switch the structure to nested persistent objects.

> No, for once it has nothing to do with the catalog. ;-)
> This is a python product of ours that stores lots of data in somewhat
> nested, persistent dictionaries. How can this be improved so this
> doesn't happen?
> 
> > I know your next questions are going to be "why?" and "how do I fix it?" and
> > "why don't you fix it?".
> 
> While I'd love it if you'd fix our code for free, it's probably beyond
> the OSS remit ;-) Any clues about how to make our code 'less bad' would
> be cool...
> 
> cheers,
> 
> Chris
> 
> ___
> Zope-Dev maillist  -  [EMAIL PROTECTED]
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Chris Withers]

Thanks for the reply :-)

Chris McDonough wrote:
> 
> It happens when more than one thread tries to modify the same object in
> storage at the same time.  It's Zope's equivalent of record-locking... It's
> normal.  There's no corruption or anything, it's telling you that it avoided
> a write that might have caused problems.  Zope tries three times to retry
> the write.  If the object is still locked by another database connection
> after three writes, it propogates the exception up to the app level.  That's
> the error you see.

Hurm, well, it appears to happen when we're doing a lot of sequential
write to a product (automated bulk data upload), but the thing that
triggers it off is actually trying to read a page from the same area of
the site... confusing. Although that could also be a one-off
coincidence...

You say no corruption or anything, but if a submit results in that
error, does the submitted form data get processed or not?

> Careful application coding can reduce the chance of conflict errors.  

Can you describe what you mean by careful application coding?

> I bet it happens on a catalog write
> for you.  

No, for once it has nothing to do with the catalog. ;-)
This is a python product of ours that stores lots of data in somewhat
nested, persistent dictionaries. How can this be improved so this
doesn't happen?

> I know your next questions are going to be "why?" and "how do I fix it?" and
> "why don't you fix it?".  

While I'd love it if you'd fix our code for free, it's probably beyond
the OSS remit ;-) Any clues about how to make our code 'less bad' would
be cool...

cheers,

Chris

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )

Re: [Zope-dev] Conflict Errors

[Chris McDonough]

It happens when more than one thread tries to modify the same object in
storage at the same time.  It's Zope's equivalent of record-locking... It's
normal.  There's no corruption or anything, it's telling you that it avoided
a write that might have caused problems.  Zope tries three times to retry
the write.  If the object is still locked by another database connection
after three writes, it propogates the exception up to the app level.  That's
the error you see.

Careful application coding can reduce the chance of conflict errors.  I've
seen conflict errors pop up a lot when the catalog is used for whatever
reason, especially with catalogaware.  I bet it happens on a catalog write
for you.  I think this may have something to do with the current BTree
implementation's propensity to write child-parent relationships all the way
up and down the tree every time a node changes.

I know your next questions are going to be "why?" and "how do I fix it?" and
"why don't you fix it?".  Chris Petrilli did some work on the catalog for
2.2.3 that might have a positive impact.  The catalog is (I think) being
actively worked on if you can nail it down some, it'd probably be
helpful to figure out when the conflicterror occurs and pass it along to
Chris in the collector.  Besides that, I don't have any specific hints for
you, besides waiting for the new BTree implementation.  The answer to
"when?":  I have no idea.

HTH,

C



- Original Message -
From: "Chris Withers" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Monday, November 20, 2000 10:46 AM
Subject: [Zope-dev] Conflict Errors


> Hi,
>
> Whenever there's a lot of writes and the occasional read going on, we
> get the following error:
>
> 2000-11-19T16:03:44 INFO(0) Z2 CONFLICT Competing writes at,
> /aFolder/anObject
> Traceback (innermost last):
>   File D:\Program Files\Zope\lib\python\ZPublisher\Publish.py, line 175,
> in publ
> ish
>   File D:\Program Files\Zope\lib\python\Zope\__init__.py, line 235, in
> commit
>   File D:\Program Files\Zope\lib\python\ZODB\Transaction.py, line 251,
> in commit
>
>   File D:\Program Files\Zope\lib\python\ZODB\Connection.py, line 267, in
> commit
> ConflictError:y
>
> Can anyone shed any light on:
> -what this means
> -how bad it is
> -how we can make it stop happening
>
> cheers,
>
> Chris
>
> ___
> Zope-Dev maillist  -  [EMAIL PROTECTED]
> http://lists.zope.org/mailman/listinfo/zope-dev
> **  No cross posts or HTML encoding!  **
> (Related lists -
>  http://lists.zope.org/mailman/listinfo/zope-announce
>  http://lists.zope.org/mailman/listinfo/zope )
>
>


___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://lists.zope.org/mailman/listinfo/zope-announce
 http://lists.zope.org/mailman/listinfo/zope )