[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Joachim Schmitz

Tres Seaver schrieb:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Not if the sessions being used are from 'faster' -- it doesn't use
IOBTree.  The major application use of that module is in the catalog.


you correct see below:


Try dumping out the contents of the bucket:

  for k, v in bucket.items():
  print k, type(v)


resolve built-in method _p_resolveConflict of BTrees._IOBTree.IOBucket 
object at 0xb1ab82b4


print root._p_jar[p64(0xb1ab82b4)]
*** POSKeyError: ZODB.POSException.POSKeyError instance at 0xa8f9e6cc

with the recepies here http://www.zopelabs.com/cookbook/1114086617
I was able to get the information about the oid, which is passed to
tryToResolveConflict, here is the result

BTrees._IOBTree.IOBTree object at 0xb562fadc
DateIndex at created
Products.ZCatalog.Catalog.Catalog object at 0xb12d622c
CatalogTool at portal_catalog
CPSDefaultSite at uniben
Application at 
{'Application': Application at , 'ZGlobals': BTrees._OOBTree.OOBTree 
object at 0xb2739224}


What does this tell us ?




--
Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Joachim Schmitz wrote:
 Tres Seaver schrieb:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Not if the sessions being used are from 'faster' -- it doesn't use
 IOBTree.  The major application use of that module is in the catalog.

 you correct see below:
 
 Try dumping out the contents of the bucket:

   for k, v in bucket.items():
   print k, type(v)
 
 resolve built-in method _p_resolveConflict of BTrees._IOBTree.IOBucket 
 object at 0xb1ab82b4
 
 print root._p_jar[p64(0xb1ab82b4)]
 *** POSKeyError: ZODB.POSException.POSKeyError instance at 0xa8f9e6cc
 
 with the recepies here http://www.zopelabs.com/cookbook/1114086617
 I was able to get the information about the oid, which is passed to
 tryToResolveConflict, here is the result
 
 BTrees._IOBTree.IOBTree object at 0xb562fadc
 DateIndex at created
 Products.ZCatalog.Catalog.Catalog object at 0xb12d622c
 CatalogTool at portal_catalog
 CPSDefaultSite at uniben
 Application at 
 {'Application': Application at , 'ZGlobals': BTrees._OOBTree.OOBTree 
 object at 0xb2739224}
 
 What does this tell us ?

That is a real conflict:  both transactions have inserted values into
the 'created' date index's '_index'  under the same key, which can't be
resolved.  Retrying the transaction is the only choice here.

Because DateIndexes convert the indexed value to an integer with
precision of one minute, a date index on 'created' is fairly likely to
generate such conflicts when two parties both create content at the same
time.

Ideally, one would examine the two values being inserted, note that they
were both IITreeSet instances containing one int apiece, and exploit our
knowledge of the application semantics to merge them, removing the
conflict;   however, *because* they are IITreeSets, and therefore
separate persistent objects, their state is not available to the
bucket's '_p_resolveConflict', which must therefore lose.


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGYBDQ+gerLs4ltQ4RAoRNAKCb86Bjhp5fuk7bp9OV2IMUXDKm7ACeO/aH
hVfzx/U0rXsM3iNT2fOl2As=
=egtx
-END PGP SIGNATURE-

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Joachim Schmitz

Tres Seaver schrieb:


BTrees._IOBTree.IOBTree object at 0xb562fadc
DateIndex at created
Products.ZCatalog.Catalog.Catalog object at 0xb12d622c
CatalogTool at portal_catalog
CPSDefaultSite at uniben
Application at 
{'Application': Application at , 'ZGlobals': BTrees._OOBTree.OOBTree 
object at 0xb2739224}


What does this tell us ?


That is a real conflict:  both transactions have inserted values into
the 'created' date index's '_index'  under the same key, which can't be
resolved.  Retrying the transaction is the only choice here.

Because DateIndexes convert the indexed value to an integer with
precision of one minute,

really one minute or do you mean second.

 a date index on 'created' is fairly likely to

generate such conflicts when two parties both create content at the same
time.
But then I wonder how a CMF site could ever work, this error shows up 
already on my lokal system when to users add something. On our live 
system it is killing our portal ?

I would consider this a bug.



Ideally, one would examine the two values being inserted, note that they
were both IITreeSet instances containing one int apiece, and exploit our
knowledge of the application semantics to merge them, removing the
conflict;   however, *because* they are IITreeSets, and therefore
separate persistent objects, their state is not available to the
bucket's '_p_resolveConflict', which must therefore lose.


--
Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Joachim Schmitz wrote:
 Tres Seaver schrieb:
 BTrees._IOBTree.IOBTree object at 0xb562fadc
 DateIndex at created
 Products.ZCatalog.Catalog.Catalog object at 0xb12d622c
 CatalogTool at portal_catalog
 CPSDefaultSite at uniben
 Application at 
 {'Application': Application at , 'ZGlobals': BTrees._OOBTree.OOBTree 
 object at 0xb2739224}

 What does this tell us ?
 That is a real conflict:  both transactions have inserted values into
 the 'created' date index's '_index'  under the same key, which can't be
 resolved.  Retrying the transaction is the only choice here.

 Because DateIndexes convert the indexed value to an integer with
 precision of one minute,
 really one minute or do you mean second.

Yes, really one minute.

   a date index on 'created' is fairly likely to
 generate such conflicts when two parties both create content at the same
 time.
 But then I wonder how a CMF site could ever work,

You only see conflicts on this index when two people try to insert
content at the same time (i.e., they both start from the same base
transaction ID).

 this error shows up 
 already on my lokal system when to users add something. On our live 
 system it is killing our portal ?
 I would consider this a bug.

Perhaps the QueueCatalog product would help you:   that would allow you
to defer updates to the conflict-prone indexes (created, modified,
plus any {ZC,}TextIndexes) for handling by a separate batch process:

 http://svn.zope.org/Products.QueueCatalog

It looks as though you may need the trunk version for compatibility with
Zope = 2.9.



Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGYCDR+gerLs4ltQ4RAuI8AJ9eRg+ZIdh3Cdqog3adXQuSIdByHgCeJN0M
8Qb5I8saP9W0eIy+5/OlAmg=
=e5qN
-END PGP SIGNATURE-

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Jürgen Kartnaller

We also run into this kind of problems.

The only save way to solve it was to serialize our index updates.

We do this by using one zope instance which runs only a 
lovely.remotetask task service. This service contains an indexing task. 
When indexing is needed we just create a new indexing job which is 
stored in the job list of the remote task service.


This solves two problems :

- conflicts
- low speed because of complex index value calculations

Btw the setup for this application contains 18 zope's each zope running 
one thread. 17 zope's are used to handle browser requests and one is 
used to handle the remote tasks (not only indexing tasks).


Jürgen

Tres Seaver wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Joachim Schmitz wrote:
Any suggestion for a temporary fix would be very welcome, since we get 
about 6000 conflict errors per day now about 15 % unresolved, and they 
are really killing our portal.

A workaround might be to replace the DateIndexes for 'created' and
'modified' with FieldIndexes:  you will see a big jump in the number of
keys in the index, but (perhaps) a reduction in conflicts (altheough
there will be more bucket splits, which can also cause conflicts).


Unfortunately that does not work the FieldIndex also gives an conflict-error
FieldIndex at Date
Products.ZCatalog.Catalog.Catalog object at 0xb250a6ec
CatalogTool at portal_catalog
CPSDefaultSite at uniben
Application at 
{'Application': Application at , 'ZGlobals': BTrees._OOBTree.OOBTree 
object at 0xb242989c}
2007-06-01 20:23:41 INFO ZPublisher.Conflict ConflictError at 
/uniben/campus/students/A923157/study_course/create_level: database 
conflict error (oid 0x36c429, class BTrees._OOBTree.OOBTree, serial this 
txn started with 0x036e0590d58661ee 2007-06-01 17:52:50.044906, serial 
currently committed 0x036e059346ef95dd 2007-06-01 17:55:16.625597) (1 
conflicts (0 unresolved) since startup at Fri Jun  1 19:54:43 2007)




A longer term solution might be to come up with a derived index type
which used non-persistent objects (rather than the IITreeSet used by
UnIndex) for the set of RIDs stored under each 'minute':  you could then
inspect the state and do more aggressive conflict resolution than the
stock implementation allows (see my other post).

any other shortterm suggestions.


QueueCatalog?


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGYGZU+gerLs4ltQ4RAk4BAJ46DKm1vLlygIqee1OxUjSPYF40pwCfSLvy
mS+9UyTtv0OuNWuotzqk5Tg=
=Fevy
-END PGP SIGNATURE-

___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce

 http://mail.zope.org/mailman/listinfo/zope )



___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jürgen Kartnaller wrote:

 We also run into this kind of problems.
 
 The only save way to solve it was to serialize our index updates.
 
 We do this by using one zope instance which runs only a 
 lovely.remotetask task service. This service contains an indexing task. 
 When indexing is needed we just create a new indexing job which is 
 stored in the job list of the remote task service.

That is exactly how QueueCatalog functions:  it batches expensive /
conflict-prone indexing operations into jobs which are handled later, in
a serialized way.

 This solves two problems :
 
 - conflicts
 - low speed because of complex index value calculations
 
 Btw the setup for this application contains 18 zope's each zope running 
 one thread. 17 zope's are used to handle browser requests and one is 
 used to handle the remote tasks (not only indexing tasks).

How many of the front-end servers do writes?  Or are all your visitors
potential writers?


Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGYOWr+gerLs4ltQ4RAmpyAJwPxdx+ob46RgvasStZwoPWQQTa8gCdG/+t
05J+ocnTgZVOnf7MvLw3oGw=
=kyUn
-END PGP SIGNATURE-

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-06-01 Thread Jürgen Kartnaller



Tres Seaver wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Jürgen Kartnaller wrote:


We also run into this kind of problems.

The only save way to solve it was to serialize our index updates.

We do this by using one zope instance which runs only a 
lovely.remotetask task service. This service contains an indexing task. 
When indexing is needed we just create a new indexing job which is 
stored in the job list of the remote task service.


That is exactly how QueueCatalog functions:  it batches expensive /
conflict-prone indexing operations into jobs which are handled later, in
a serialized way.


This solves two problems :

- conflicts
- low speed because of complex index value calculations

Btw the setup for this application contains 18 zope's each zope running 
one thread. 17 zope's are used to handle browser requests and one is 
used to handle the remote tasks (not only indexing tasks).


How many of the front-end servers do writes?  Or are all your visitors
potential writers?


All servers do writes.




Tres.
- --
===
Tres Seaver  +1 540-429-0999  [EMAIL PROTECTED]
Palladion Software   Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGYOWr+gerLs4ltQ4RAmpyAJwPxdx+ob46RgvasStZwoPWQQTa8gCdG/+t
05J+ocnTgZVOnf7MvLw3oGw=
=kyUn
-END PGP SIGNATURE-
___
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
 http://mail.zope.org/mailman/listinfo/zope-announce

 http://mail.zope.org/mailman/listinfo/zope )



___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-31 Thread Joachim Schmitz

Dieter Maurer schrieb:

Perry wrote at 2007-5-25 13:16 +0200:

database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket,
serial this txn started with 0x036ddc2a44454dee 2007-05-25
09:14:16.000950, serial currently committed 0x036ddc2c21950377
2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup
at Fri May 25 05:19:08 2007)
...
ConflictError: database conflict error (oid 0x7905e6, class
BTrees._IOBTree.IOBucket, serial this txn started with
0x036ddc2b3e989fdd 2007-05-25 09:15:14.670982, serial currently
committed 0x036ddc2dd48f4e33 2007-05-25 09:17:49.818700)


These log entries indicate a bug in ZODB's invalidation processing.

  The first entry tells us that the object was read at 9:14:16
  and the modification conflicts with a write from 9:16:07.

  The second entry tells us that the object was read at 9:15:14
  *BUT* at the time this transaction has started,
  the ZODB should already have known the modification from 9:16:07
  and the object read at 9:15:14 should have been invalidated.
  The new transaction should not have seen any state older than 9:16:07
  (as it begins after this time).


I could reproduce the conflict error on my local machine not using ZEO.

I invoked the longrunning process create_level for two users after a 
zope-restart.

here is the log:

2007-05-31 09:44:24 INFO Zope Ready to handle requests

2007-05-31 09:44:39 INFO Skins.create_level A923157 started to create 
level 200
2007-05-31 09:44:41 INFO Skins.create_level Y617041 started to create 
level 400

the next two entries are printed before the redirect/commit
2007-05-31 09:45:01 INFO Skins.create_level Y617041 finished to create 
level 400
2007-05-31 09:45:06 INFO Skins.create_level A923157 finished to create 
level 200
Now the conflict error, look at the transaction start-time, this is 
before the restart of zope !!


2007-05-31 09:45:25 INFO ZPublisher.Conflict ConflictError at 
/uniben/campus/students/A923157/study_course/create_level: database 
conflict error (oid 0x3360e3, class BTrees._OIBTree.OIBucket, serial 
this txn started with 0x036dfd7c73dfc1dd 2007-05-31 07:24:27.157981, 
serial currently committed 0x036dfd9131112455 2007-05-31 
07:45:11.500069) (1 conflicts (0 unresolved) since startup at Thu May 31 
09:44:24 2007)


now this is retried for A923157

2007-05-31 09:45:26 INFO Skins.create_level A923157 started to create 
level 200
2007-05-31 09:45:45 INFO Skins.create_level A923157 finished to create 
level 200





--
Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-31 Thread Dieter Maurer
Joachim Schmitz wrote at 2007-5-31 12:07 +0200:
 ...
2007-05-31 09:45:06 INFO Skins.create_level A923157 finished to create 
level 200
Now the conflict error, look at the transaction start-time, this is 
before the restart of zope !!

You are probably tricked out here: the serials are in fact UTC timestamps.
I am not sure but it may well be that the times shown for the serials
are UTC (GMT +0) and not local times.



-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-31 Thread Perry

Dieter Maurer schrieb:

Joachim Schmitz wrote at 2007-5-31 12:07 +0200:

...
2007-05-31 09:45:06 INFO Skins.create_level A923157 finished to create 
level 200
Now the conflict error, look at the transaction start-time, this is 
before the restart of zope !!





You are probably tricked out here: the serials are in fact UTC timestamps.
I am not sure but it may well be that the times shown for the serials
are UTC (GMT +0) and not local times.


I added 2 hours to the txn times and only looked at minutes.

so the observation is still true.

--

Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-29 Thread Dieter Maurer
Joachim Schmitz wrote at 2007-5-28 17:45 +0200:
In ZODB.Connection.Connection.open I see:

 if self._reset_counter != global_reset_counter:
 # New code is in place.  Start a new cache.
 self._resetCache()
 else:
 self._flush_invalidations()

So self._flush_invalidations() is only called in the else-condition.
In your patch it is always called. I try your version and report back.

As almost always self._reset_counter == global_reset_counter
and _resetCache effectively flushes the cache, the change is *VERY* unlikely
to affect what you see with respect to conflict errors.


-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-28 Thread Joachim Schmitz

Hi Dieter,

thanks for this hint.

Dieter Maurer schrieb:

Perry wrote at 2007-5-25 13:16 +0200:

database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket,
serial this txn started with 0x036ddc2a44454dee 2007-05-25
09:14:16.000950, serial currently committed 0x036ddc2c21950377
2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup
at Fri May 25 05:19:08 2007)


In our private Zope version, I have still a note like this:

# DM 2005-08-22: always call '_flush_invalidations' as it does
#  more than cache handling only
self._flush_invalidations()
if self._reset_counter != global_reset_counter:
# New code is in place.  Start a new cache.
self._resetCache()
# DM 2005-08-22: always call '_flush_invalidations'
##else:
##self._flush_invalidations()

The note indicates that the bug was fixed at least at 2005-08-22
(though the handling was not completely right in case the cache
was reset).

In ZODB.Connection.Connection.open I see:

if self._reset_counter != global_reset_counter:
# New code is in place.  Start a new cache.
self._resetCache()
else:
self._flush_invalidations()

So self._flush_invalidations() is only called in the else-condition.
In your patch it is always called. I try your version and report back.



--
Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-28 Thread Joachim Schmitz

Joachim Schmitz schrieb:

Hi Dieter,

thanks for this hint.

Dieter Maurer schrieb:

Perry wrote at 2007-5-25 13:16 +0200:

database conflict error (oid 0x7905e6, class BTrees._IOBTree.IOBucket,
serial this txn started with 0x036ddc2a44454dee 2007-05-25
09:14:16.000950, serial currently committed 0x036ddc2c21950377
2007-05-25 09:16:07.870801) (80 conflicts (10 unresolved) since startup
at Fri May 25 05:19:08 2007)


In our private Zope version, I have still a note like this:

# DM 2005-08-22: always call '_flush_invalidations' as it does
#  more than cache handling only
self._flush_invalidations()
if self._reset_counter != global_reset_counter:
# New code is in place.  Start a new cache.
self._resetCache()
# DM 2005-08-22: always call '_flush_invalidations'
##else:
##self._flush_invalidations()

The note indicates that the bug was fixed at least at 2005-08-22
(though the handling was not completely right in case the cache
was reset).

In ZODB.Connection.Connection.open I see:

if self._reset_counter != global_reset_counter:
# New code is in place.  Start a new cache.
self._resetCache()
else:
self._flush_invalidations()

So self._flush_invalidations() is only called in the else-condition.
In your patch it is always called. I try your version and report back.

I patched the ZODB to always do the self._flush_invalidations(). But 
that didn't change anything, and after looking at the code it couldn't 
cause it was already always called.


here again is the history of a conflict error for one user which finally 
fails:
2007-05-28T18:32:12 INFO ZPublisher.Conflict ConflictError at 
/VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/V659242/study_course/create_level: 
database conflict error (oid 0x7fd771, class BTrees._IOBTree.IOBucket, 
serial this txn started with 0x036deefb352fd600 2007-05-28 
17:31:12.465670, serial currently committed 0x036deefb9aa4d733 
2007-05-28 17:31:36.244666) (34 conflicts (4 unresolved) since startup 
at Mon May 28 17:41:55 2007)
2007-05-28T18:32:44 INFO ZPublisher.Conflict ConflictError at 
/VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/V659242/study_course/create_level: 
database conflict error (oid 0x7fd771, class BTrees._IOBTree.IOBucket, 
serial this txn started with 0x036deefb9aa4d733 2007-05-28 
17:31:36.244666, serial currently committed 0x036deefc2e4ff122 
2007-05-28 17:32:10.854439) (35 conflicts (4 unresolved) since startup 
at Mon May 28 17:41:55 2007)
2007-05-28T18:33:15 INFO Skins.create_level V659242 finished to create 
level 1002007-05-28T18:34:29 INFO ZPublisher.Conflict ConflictError at 
/VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/V659242/study_course/create_level: 
database conflict error (oid 0x7fd771, class BTrees._IOBTree.IOBucket, 
serial this txn started with 0x036deefc2e4ff122 2007-05-28 
17:32:10.854439, serial currently committed 0x036deefe18489244 
2007-05-28 17:34:05.691441) (36 conflicts (4 unresolved) since startup 
at Mon May 28 17:41:55 2007)
2007-05-28T18:35:21 INFO Skins.create_level V659242 finished to create 
level 1002007-05-28T18:38:36 INFO ZPublisher.Conflict ConflictError at 
/VirtualHostBase/http/uniben.waeup.org:80/uniben/VirtualHostRoot/campus/students/V659242/study_course/create_level: 
database conflict error (oid 0x7fd771, class BTrees._IOBTree.IOBucket, 
serial this txn started with 0x036deefe18489244 2007-05-28 
17:34:05.691441, serial currently committed 0x036def009207e011 
2007-05-28 17:36:34.225960) (42 conflicts (4 unresolved) since startup 
at Mon May 28 17:41:55 2007)


--
2007-05-28T18:38:36 ERROR Zope.SiteErrorLog 
http://uniben.waeup.org/campus/students/V659242/study_course/create_level

Traceback (innermost last):
  Module Zope2.App.startup, line 173, in zpublisher_exception_hook
  Module ZPublisher.Publish, line 121, in publish
  Module Zope2.App.startup, line 240, in commit
  Module transaction._manager, line 96, in commit
  Module Products.CPSCompat.PatchZODBTransaction, line 175, in commit
  Module transaction._transaction, line 436, in _commitResources
  Module ZODB.Connection, line 665, in tpc_vote
  Module ZEO.ClientStorage, line 893, in tpc_vote
  Module ZEO.ClientStorage, line 877, in _check_serials
ConflictError: database conflict error (oid 0x7fd771, class 
BTrees._IOBTree.IOBucket, serial this txn started with 
0x036deefe18489244 2007-05-28 17:34:05.691441, serial currently 
committed 0x036def009207e011 2007-05-28 17:36:34.225960)

--

How can I find out, which objects are really involved in the conflict.


--
Gruß Joachim

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-28 Thread Andreas Jung



--On 28. Mai 2007 20:03:47 +0200 Joachim Schmitz [EMAIL PROTECTED] wrote:


Joachim Schmitz schrieb:
ConflictError: database conflict error (oid 0x7fd771, class
BTrees._IOBTree.IOBucket, serial this txn started with 0x036deefe18489244
2007-05-28 17:34:05.691441, serial currently committed 0x036def009207e011
2007-05-28 17:36:34.225960)
--

How can I find out, which objects are really involved in the conflict.


from ZODB.utils import p64
print app._p_jar[p64(some_oid)]

The oid is available from the traceback. If you are running ZEO it might be 
necessary to replace 'app' with the top-level (or some other object) of the 
related storage (might be tricky to figure out if you have multiple 
storages).


-aj

pgpjMqIfNMzGP.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] Re: [Bug] ZODB invalidation processing

2007-05-28 Thread Andreas Jung



--On 28. Mai 2007 21:12:13 +0200 Andreas Jung [EMAIL PROTECTED] wrote:




--On 28. Mai 2007 20:03:47 +0200 Joachim Schmitz [EMAIL PROTECTED] wrote:


Joachim Schmitz schrieb:
ConflictError: database conflict error (oid 0x7fd771, class
BTrees._IOBTree.IOBucket, serial this txn started with 0x036deefe18489244
2007-05-28 17:34:05.691441, serial currently committed 0x036def009207e011
2007-05-28 17:36:34.225960)
--

How can I find out, which objects are really involved in the conflict.


from ZODB.utils import p64
print app._p_jar[p64(some_oid)]

The oid is available from the traceback. If you are running ZEO it might
be necessary to replace 'app' with the top-level (or some other object)
of the related storage (might be tricky to figure out if you have
multiple storages).


Another trick that helped me lately to identify the reason for conflict 
errors was to implement the _p_resolveConflict() hook as method of the 
related class having the conflict.



-aj

pgpWMpg9SDVcI.pgp
Description: PGP signature
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev