Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2014-02-03 Thread Marius Gedminas
On Mon, Feb 03, 2014 at 03:33:13PM +0100, Godefroid Chapelle wrote:
> Le 03/02/14 15:12, Marius Gedminas a écrit :
> >Could you tell us why that test depends on Acquisition?  Is it possible
> >to replicate the bug using pure Python code?
> 
> I have not been able to replicate the bug with pure Python code.
> Reason why it took me so much time.
> 
> IIRC, the issue is that on newly created objects that inherit from
> Acquisition.Implicit, the _p_jar attribute has a value (I guess it
> is acquired). Without acquisition, because a newly created
> BTree._p_jar does not have any value before transaction is
> committed, any call to readCurrent is then discarded.

Ah, I thought it might be caused by the acquisition of _p_oid, but then
decided that made no sense.  (Actually I thought Acquisition would skip
_p_* or _v_* attributes, and I'm surprised to hear that it doesn't.)

> I understand this gives the feeling that the fix should be in
> Acquisition rather than in ZODB. I have tried to make a fix for the
> Acquisition code but was unable to understand enough.
> 
> Nevertheless, my understanding of the readCurrent API makes me think
> that it is actually useless to keep reference to newly created
> objects (iow in same transaction). And that the fix I propose makes
> sense anyway.

I'm not disputing that.  (This is way beyond my familiarity with ZODB
internals.)

Marius Gedminas
-- 
"question = (to) ? be : !be;" -- Shakespeare


signature.asc
Description: Digital signature
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2014-02-03 Thread Godefroid Chapelle

Le 03/02/14 15:12, Marius Gedminas a écrit :

Could you tell us why that test depends on Acquisition?  Is it possible
to replicate the bug using pure Python code?

Marius Gedminas


I have not been able to replicate the bug with pure Python code. Reason 
why it took me so much time.


IIRC, the issue is that on newly created objects that inherit from 
Acquisition.Implicit, the _p_jar attribute has a value (I guess it is 
acquired). Without acquisition, because a newly created BTree._p_jar 
does not have any value before transaction is committed, any call to 
readCurrent is then discarded.


I understand this gives the feeling that the fix should be in 
Acquisition rather than in ZODB. I have tried to make a fix for the 
Acquisition code but was unable to understand enough.


Nevertheless, my understanding of the readCurrent API makes me think 
that it is actually useless to keep reference to newly created objects 
(iow in same transaction). And that the fix I propose makes sense anyway.


--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2014-02-03 Thread Marius Gedminas
On Mon, Feb 03, 2014 at 02:50:42PM +0100, Godefroid Chapelle wrote:
> It needed time and obstinacy to find how to write a test that
> triggers the same POSKeyError during commit as triggered by some
> Plone 3 to 4 migrations.
> 
> For more details about the error, see
> http://rpatterson.net/blog/poskeyerror-during-commit
> 
> The test is now pushed to the 3.10 branch on github :
> 
> https://github.com/zopefoundation/ZODB/commit/49919d31cfa73410cede0af969d80527a9fd852c
> 
> The fix is pushed as well.
> 
> https://github.com/zopefoundation/ZODB/commit/9ce470f70acc9655aaf915e5a74678b43f6312da
...
> - As the test above depends on Acquisition (which is not ported to
> Python 3), what should be the strategy to include my work in master
> ? Pull the fix only ? Pull the fix and do some magic with the test
> so that it runs only in Python 2 ? Other suggestion ?

Could you tell us why that test depends on Acquisition?  Is it possible
to replicate the bug using pure Python code?

Marius Gedminas
-- 
Q: A good ninja is hard to find.
A: Well, yes, actually, being hard to find is the whole point with them.
-- http://tinyurl.com/5tnrwr


signature.asc
Description: Digital signature
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2014-02-03 Thread Godefroid Chapelle

Hi all,

It needed time and obstinacy to find how to write a test that triggers 
the same POSKeyError during commit as triggered by some Plone 3 to 4 
migrations.


For more details about the error, see 
http://rpatterson.net/blog/poskeyerror-during-commit


The test is now pushed to the 3.10 branch on github :

https://github.com/zopefoundation/ZODB/commit/49919d31cfa73410cede0af969d80527a9fd852c

The fix is pushed as well.

https://github.com/zopefoundation/ZODB/commit/9ce470f70acc9655aaf915e5a74678b43f6312da

Both commits are referenced in an issue
https://github.com/zopefoundation/ZODB/issues/16

I have a request and a question :

- Can we get a new 3.10.X release (iow 3.10.6) with the fix ?

I am happy to do it myself if needed.

- As the test above depends on Acquisition (which is not ported to 
Python 3), what should be the strategy to include my work in master ? 
Pull the fix only ? Pull the fix and do some magic with the test so that 
it runs only in Python 2 ? Other suggestion ?


Regards,
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2013-09-23 Thread Godefroid Chapelle

Le 17/09/13 10:44, Godefroid Chapelle a écrit :

Hi,

Plone 3 to 4 migration occasionally triggers a POSKeyError during
transaction commit. See
http://rpatterson.net/blog/poskeyerror-during-commit

I encountered the issue yesterday.

I committed a test and a fix in branch 3.10.

https://github.com/zopefoundation/ZODB/commit/6457bcfd07b3b77f240cbc6d60f222f0d40e8d5d


Before the fix, the test triggered a traceback very similar to the one
seen when migrating Plone.

As I am not an expert of ZODB internals, I'd like someone to review my
fix even if the test suite does not break.


I confirm that I am not a ZODB internals expert :-(

The fix does avoid the issue triggered by my test.

But does NOT fix the Plone migration issue. IOW, the test is not 
reproducing the issue triggered by the Plone migration.


This is what I have found :

At the end of the Connection.commit() method, oids found in 
Connection._readCurrent mapping are checked for mismatch between the 
committed tid and the one stored in _readCurrent mapping.


If there is a mismatch, the object must be invalidated in cache.

In the Plone migration issue, when getting to the step where the 
committed tid is fetched, a POSKeyError is triggered by getTid() method.


I found out that the tid stored in _readCurrrent for this falty oid is 
equal to z64 (eight nul chars).


This is what I deduce :

the oid, tid pair (where tid is null) corresponds to an object that was 
added to the transaction but not completely cleaned up by the 
transaction rollback


(I am looking at rollback because my migration does call 
savepoint.rollback() five times and there are five oids with null tid 
leftover in Connection._readCurrent when the POSKeyError is triggered. 
But this might just be a coincidence.)


Any hint where to add the call to pop those falty oids out of 
_readCurrent during transaction rollback, or to enlighten me about 
things I misunderstand is welcome.


Regards
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent

2013-09-17 Thread Godefroid Chapelle

Hi,

Plone 3 to 4 migration occasionally triggers a POSKeyError during 
transaction commit. See http://rpatterson.net/blog/poskeyerror-during-commit


I encountered the issue yesterday.

I committed a test and a fix in branch 3.10.

https://github.com/zopefoundation/ZODB/commit/6457bcfd07b3b77f240cbc6d60f222f0d40e8d5d

Before the fix, the test triggered a traceback very similar to the one 
seen when migrating Plone.


As I am not an expert of ZODB internals, I'd like someone to review my 
fix even if the test suite does not break.


Thanks
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] PosKeyError in DateRangeIndex with RelStorage

2012-08-03 Thread Martijn Pieters
On Fri, Aug 3, 2012 at 11:56 PM, Maurits van Rees
 wrote:
> Earlier this week I encountered a PosKeyError.  This is in a Plone site with
> RelStorage (postgres).  The traceback pointed to a DateRangeIndex.  In the
> end I solved it by removing this index (the effectiveRange index) and
> recreating and indexing it.
>
> So: problem solved, but I wonder if it is likely to happen again.  Has
> anyone seen this problem specifically in a DateRangeIndex?

Nope, the RelStorage site on Oracle I maintain has not shown any such
problems. It does run on Plone 4.1 these days though (upgraded from
3.x last year).

-- 
Martijn Pieters
___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] PosKeyError in DateRangeIndex with RelStorage

2012-08-03 Thread Maurits van Rees

Hi,

Earlier this week I encountered a PosKeyError.  This is in a Plone site 
with RelStorage (postgres).  The traceback pointed to a DateRangeIndex. 
 In the end I solved it by removing this index (the effectiveRange 
index) and recreating and indexing it.


So: problem solved, but I wonder if it is likely to happen again.  Has 
anyone seen this problem specifically in a DateRangeIndex?


Here is the last part of the traceback, happening on every page, at 
least for anonymous users, because it happens when building the navigation:


Module Products.PageTemplates.Expressions, line 125, in render
Module Products.CMFPlone.browser.ploneview, line 67, in globalize
Module Products.CMFPlone.browser.ploneview, line 124, in _initializeData
Module Products.CMFPlone.browser.navigation, line 179, in topLevelTabs
Module Products.CMFPlone.CatalogTool, line 421, in searchResults
Module Products.ZCatalog.ZCatalog, line 663, in searchResults
Module Products.ZCatalog.Catalog, line 738, in searchResults
Module Products.ZCatalog.Catalog, line 477, in search
Module Products.PluginIndexes.DateRangeIndex.DateRangeIndex, line 288, 
in _apply_index

Module ZODB.Connection, line 797, in setstate
Module ZODB.Connection, line 837, in _setstate
Module relstorage.storage, line 470, in load
POSKeyError:  
(Also, the following error occurred while attempting to render the 
standard error message, please see the event log for full details: 0x68b741)


This is a Plone 3.1.7 site that has been moved to RelStorage (1.5.0) 
earlier this year.  Zope 2.10.12, ZODB3 3.7.3-polling, python 2.4.


Thanks,


--
Maurits van Rees: http://maurits.vanrees.org/
Zest Software: http://zestsoftware.nl

___
For more information about ZODB, see http://zodb.org/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError

2011-06-22 Thread Erik Dahl
Shane,

I think the pre-pack will take that long (we have never waited long enough for 
a gc pack to complete the first stages).  We have db replication and I'll peel 
of one of the slaves to do the pack... so it's not the end of the world (unless 
the db is hosed :) )

-EAD



On Jun 22, 2011, at 6:39 PM, Shane Hathaway wrote:

> On 06/22/2011 04:37 PM, Shane Hathaway wrote:
>> On 06/22/2011 04:27 PM, Erik Dahl wrote:
>>> Ugh.  Ok I'll see what we get.  So I'm clear we are looking for references 
>>> to the object/tid pairs that don't exist.  Pack will take at lest 2 days to 
>>> run. :(
>> 
>> Well, just pre-pack shouldn't take that long (I hope).
> 
> BTW don't forget you can (in theory) do this on a copy of the database rather 
> than use production hardware.
> 
> Shane

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError

2011-06-22 Thread Shane Hathaway
On 06/22/2011 04:37 PM, Shane Hathaway wrote:
> On 06/22/2011 04:27 PM, Erik Dahl wrote:
>> Ugh.  Ok I'll see what we get.  So I'm clear we are looking for references 
>> to the object/tid pairs that don't exist.  Pack will take at lest 2 days to 
>> run. :(
>
> Well, just pre-pack shouldn't take that long (I hope).

BTW don't forget you can (in theory) do this on a copy of the database 
rather than use production hardware.

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError

2011-06-22 Thread Shane Hathaway
On 06/22/2011 04:27 PM, Erik Dahl wrote:
> Ugh.  Ok I'll see what we get.  So I'm clear we are looking for references to 
> the object/tid pairs that don't exist.  Pack will take at lest 2 days to run. 
> :(

Well, just pre-pack shouldn't take that long (I hope).

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError

2011-06-22 Thread Erik Dahl
Ugh.  Ok I'll see what we get.  So I'm clear we are looking for references to 
the object/tid pairs that don't exist.  Pack will take at lest 2 days to run. :(

-EAD



On Jun 22, 2011, at 5:08 PM, Shane Hathaway wrote:

> On 06/21/2011 07:18 AM, Erik Dahl wrote:
>> I'm using relstorage 1.4.2 during a batch job I was running last night I got 
>> the following errors.
>> 
>> 2011-06-21 07:55:02,664 WARNING relstorage: POSKeyError on oid 23916102: no 
>> tid found; Current transaction is 256466219826629358; Recent object tids: []
>> 2011-06-21 07:55:02,665 WARNING relstorage: POSKeyError on oid 23916103: no 
>> tid found; Current transaction is 256466219826629358; Recent object tids: []
>> 2011-06-21 08:00:19,850 WARNING relstorage: POSKeyError on oid 23917425: no 
>> tid found; Current transaction is 256466242546653525; Recent object tids: []
>> 2011-06-21 08:00:19,854 WARNING relstorage: POSKeyError on oid 23917423: no 
>> tid found; Current transaction is 256466242546653525; Recent object tids: []
>> 2011-06-21 08:00:19,865 WARNING relstorage: POSKeyError on oid 23917424: no 
>> tid found; Current transaction is 256466242546653525; Recent object tids: []
>> 2011-06-21 08:01:56,607 WARNING relstorage: POSKeyError on oid 23917839: no 
>> tid found; Current transaction is 256466249456206779; Recent object tids: []
>> 2011-06-21 08:01:56,616 WARNING relstorage: POSKeyError on oid 23917838: no 
>> tid found; Current transaction is 256466249456206779; Recent object tids: []
>> 
>> Whenever I see POSKeyError I get really nervous but I see these are marked 
>> as warnings.  A quick look at the database seemed to show that it was ok.  
>> Should I be worried?
> 
> I would investigate more before saying whether it's OK.  Run a pre-pack with 
> GC to populate the object_ref table, then find out what refers to those 
> objects by exploring object_ref.
> 
> (It would be really nice if someone invented a ZODB browser that's aware of 
> the object_ref table, perhaps through some API that RelStorage has not yet 
> exposed.  object_ref makes it easy and fast to discover the lineage of any 
> ZODB object.)
> 
> Shane

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError

2011-06-22 Thread Shane Hathaway
On 06/21/2011 07:18 AM, Erik Dahl wrote:
> I'm using relstorage 1.4.2 during a batch job I was running last night I got 
> the following errors.
>
> 2011-06-21 07:55:02,664 WARNING relstorage: POSKeyError on oid 23916102: no 
> tid found; Current transaction is 256466219826629358; Recent object tids: []
> 2011-06-21 07:55:02,665 WARNING relstorage: POSKeyError on oid 23916103: no 
> tid found; Current transaction is 256466219826629358; Recent object tids: []
> 2011-06-21 08:00:19,850 WARNING relstorage: POSKeyError on oid 23917425: no 
> tid found; Current transaction is 256466242546653525; Recent object tids: []
> 2011-06-21 08:00:19,854 WARNING relstorage: POSKeyError on oid 23917423: no 
> tid found; Current transaction is 256466242546653525; Recent object tids: []
> 2011-06-21 08:00:19,865 WARNING relstorage: POSKeyError on oid 23917424: no 
> tid found; Current transaction is 256466242546653525; Recent object tids: []
> 2011-06-21 08:01:56,607 WARNING relstorage: POSKeyError on oid 23917839: no 
> tid found; Current transaction is 256466249456206779; Recent object tids: []
> 2011-06-21 08:01:56,616 WARNING relstorage: POSKeyError on oid 23917838: no 
> tid found; Current transaction is 256466249456206779; Recent object tids: []
>
> Whenever I see POSKeyError I get really nervous but I see these are marked as 
> warnings.  A quick look at the database seemed to show that it was ok.  
> Should I be worried?

I would investigate more before saying whether it's OK.  Run a pre-pack 
with GC to populate the object_ref table, then find out what refers to 
those objects by exploring object_ref.

(It would be really nice if someone invented a ZODB browser that's aware 
of the object_ref table, perhaps through some API that RelStorage has 
not yet exposed.  object_ref makes it easy and fast to discover the 
lineage of any ZODB object.)

Shane
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] POSKeyError

2011-06-21 Thread Erik Dahl
I'm using relstorage 1.4.2 during a batch job I was running last night I got 
the following errors.  

2011-06-21 07:55:02,664 WARNING relstorage: POSKeyError on oid 23916102: no tid 
found; Current transaction is 256466219826629358; Recent object tids: []
2011-06-21 07:55:02,665 WARNING relstorage: POSKeyError on oid 23916103: no tid 
found; Current transaction is 256466219826629358; Recent object tids: []
2011-06-21 08:00:19,850 WARNING relstorage: POSKeyError on oid 23917425: no tid 
found; Current transaction is 256466242546653525; Recent object tids: []
2011-06-21 08:00:19,854 WARNING relstorage: POSKeyError on oid 23917423: no tid 
found; Current transaction is 256466242546653525; Recent object tids: []
2011-06-21 08:00:19,865 WARNING relstorage: POSKeyError on oid 23917424: no tid 
found; Current transaction is 256466242546653525; Recent object tids: []
2011-06-21 08:01:56,607 WARNING relstorage: POSKeyError on oid 23917839: no tid 
found; Current transaction is 256466249456206779; Recent object tids: []
2011-06-21 08:01:56,616 WARNING relstorage: POSKeyError on oid 23917838: no tid 
found; Current transaction is 256466249456206779; Recent object tids: []

Whenever I see POSKeyError I get really nervous but I see these are marked as 
warnings.  A quick look at the database seemed to show that it was ok.  Should 
I be worried?

-EAD



___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] POSKeyError in zodb-3.6.0

2006-11-10 Thread Alan Runyan
Hi Chris,I am no ZODB expert.  But I see a few thins that are wrong with the code.  Others should be able to comment at length:  - You cant just catch ConflictError and pass  - I think you can catch a ReadConflictError and *retry* that is ok.
  - But a ConflictError needs to be *retried* manually in your client code.If you catch a ConflictError you need to abort the transaction.You should be explicit about *beginning* transactions after ending previous transaction.
If your using threads.  I believe you need to be a bit more careful.  Does this codecorrupt ZODB if your *not* using threads?Instead of using threads try:  - Write the code to be used in single process
  - Start 3 processes; see if that fails  - Remove your try/except around ConflictError.  ConflictErrors need to be handled.  NOTE: In Zope the framework handled ConflictError's automatically but your doing
  everything *outside* of Zope.  Which means your code needs to do the retries.make sense?-- Alan RunyanEnfold Systems, Inc.http://www.enfoldsystems.com/
phone: +1.713.942.2377x111fax: +1.832.201.8856
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] POSKeyError in zodb-3.6.0

2006-11-06 Thread Chris Bainbridge

Hi,

This looks like a bug to me, but maybe I'm just doing something stupid
(possibly it's not allowed to deepcopy a Persistent object and then
re-sync the connection?). Try running several instances of the
following code in parallel (3 is enough for me here). Seems to be some
race condition; I quickly get an error like:

2006-11-06T16:17:37 (4/127.0.0.1:60772) Transaction blocked
waiting for storage. Clients waiting: 1.
2006-11-06T16:17:37 (4/127.0.0.1:60753) Blocked transaction restarted.
2006-11-06T16:17:37 (4/127.0.0.1:60772) Transaction blocked
waiting for storage. Clients waiting: 1.
2006-11-06T16:17:37 (4/127.0.0.1:60753) Blocked transaction restarted.
2006-11-06T16:17:37 (4/127.0.0.1:60753) Transaction blocked
waiting for storage. Clients waiting: 1.
2006-11-06T16:17:37 (4/127.0.0.1:60772) Blocked transaction restarted.
2006-11-06T16:17:37 (127.0.0.1:60753) loadEx() raised exception: 0xc9
Traceback (most recent call last):
 File "/usr/lib/python2.4/site-packages/ZEO/zrpc/connection.py", line
421, in handle_request
   ret = meth(*args)
 File "/usr/lib/python2.4/site-packages/ZEO/StorageServer.py", line
248, in loadEx
   return self.storage.loadEx(oid, version)
 File "/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py",
line 523, in loadEx
   pos = self._lookup_pos(oid)
 File "/usr/lib/python2.4/site-packages/ZODB/FileStorage/FileStorage.py",
line 514, in _lookup_pos
   raise POSKeyError(oid)
POSKeyError: 0xc9

And one of the clients will die. From that point on, any attempt to
access the bad entry in the list generates a POSKeyError.

#!/usr/bin/python

import thread
import asyncore
import random
import copy
from ZEO.ClientStorage import ClientStorage
from ZODB import DB
from persistent import Persistent
from persistent.list import PersistentList
from ZODB.POSException import ConflictError
from ZEO.zrpc.error import DisconnectedError
import transaction

storage = ClientStorage(('localhost', 12345), cache_size=16*1024**2)
db = DB(storage)
conn = db.open()
root = conn.root()
conn.sync()
thread.start_new_thread(asyncore.loop,())

try:
   g = root['test']
except:
   root['test'] = PersistentList()
   g = root['test']
   for x in range(100):
   g.append(PersistentList([x]))
   try:
   transaction.commit()
   except ConflictError:
   pass


while 1:
   conn.sync()
   x = random.choice(g)
   y = copy.deepcopy(x)
   while 1:
   try:
   conn.sync()
   i = g.index(random.choice(g))
   g[i] = y
   transaction.commit()
   except ConflictError:
   pass
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError tracing

2005-11-24 Thread Paolo Linux

Dieter Maurer wrote:


I cannot reproduce easily the problem so I'd like to know
if there are suggestions in order to understand and
track down what's happening..



Forgetting "abort"s or passing persistent objects to different
threads may cause "POSKeyError"s (though there are probably more
possibilities).


That has been more that helpful!!! thanks dieter...
I think that I've tracked down the problem to a missing abort()...

Thanks again!
Paolo

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError tracing

2005-11-24 Thread Paolo Linux

Thanks Chris,

Chris Withers wrote:


I cannot reproduce easily the problem so I'd like to know
if there are suggestions in order to understand and
track down what's happening..



Your best bet is to post the actual traceback here and explain the data 
structures involved...


Chris


I'm trying to collect some info and to narrow down the problem...
I'll ask your help as soon as I've got more data...

Paolo

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError tracing

2005-11-18 Thread Dieter Maurer
Paolo Linux wrote at 2005-11-17 10:23 +0100:
> ...
>I suspect that in some cases we do not manage Conflict
>Error (we retry for 3 times but then we give up...).
>Could this be connected with POSKeyErrors?

It should not. Because "giving up" should mean you abort the
transaction and your transaction should have no effect at all.

>I cannot reproduce easily the problem so I'd like to know
>if there are suggestions in order to understand and
>track down what's happening..

Forgetting "abort"s or passing persistent objects to different
threads may cause "POSKeyError"s (though there are probably more
possibilities).


-- 
Dieter
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


Re: [ZODB-Dev] POSKeyError tracing

2005-11-18 Thread Chris Withers

Paolo Linux wrote:

I suspect that in some cases we do not manage Conflict
Error (we retry for 3 times but then we give up...).
Could this be connected with POSKeyErrors?


Maybe, but probably not...


I cannot reproduce easily the problem so I'd like to know
if there are suggestions in order to understand and
track down what's happening..


Your best bet is to post the actual traceback here and explain the data 
structures involved...


Chris

--
Simplistix - Content Management, Zope & Python Consulting
   - http://www.simplistix.co.uk

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev


[ZODB-Dev] POSKeyError tracing

2005-11-17 Thread Paolo Linux

Hi all,
we've developed an application developed using
ZODB 3.5.1. The load has recently grown up considerably
and we're starting getting POSKeyErrors...

We've never used undoing, so that cannot be the cause.

I suspect that in some cases we do not manage Conflict
Error (we retry for 3 times but then we give up...).
Could this be connected with POSKeyErrors?

I cannot reproduce easily the problem so I'd like to know
if there are suggestions in order to understand and
track down what's happening..

Thanks!
Paolo
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev