Re: [ZODB-Dev] example for the following?
Am 22.01.2009, 17:46 Uhr, schrieb Jamie McQuay ja...@scimatic.com: Hi, I'm using ZODB in a desktop app (i.e. not using zope). I have class A which contains a list of instances for class B. I can persist class A when the list of class B is empty. When i add an instance of class B to the list (and set _p_changed = 1) i get the error: TypeError: can't pickle PySwigObject objects Can someone please give me a link of some advice in how i go about persisting lists of class instances? It seems you used SWIG to create class B (or you use a library which provides B and the library uses SWIG). By default, instances of classes created by SWIG are not serializable. You'd get the same error if you tried import pickle pickle.dumps( B() ) So what you (or the library author) need to do is add one or more of the methods for serialization explained here: http://www.python.org/doc/2.5/lib/pickle-inst.html http://www.python.org/doc/2.5/lib/node321.html http://www.python.org/doc/2.5/lib/node322.html -Matthias ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage pack problems
Interesting. RelStorage is already set up to do something like that. It splits packing into two phases. In the first phase, it makes a complete list of every row for the second phase to delete. The second phase could be delayed as long as we want. Between the two phases, if any database connections use objects that have been marked for deletion, we could cancel the pack and flag a software bug. If you want we make any kind of test with our databases, It will be a pleasure It would be very helpful to me if you could provide a copy of your database for me to debug. I'm hoping for a compressed SQL dump. Shane Hi again I already know what happens: either object_ref and object_refs_added tables are completely empty in my database. I can't understand why, but it explains the data loose when packing (nothing references nothing). So, the mystery is not in Relstorage but in the data. The only right ways to solve it is to disable pack-gc or try to fill this tables correctly. Do you know any simple way to do it ? Perhaps an export / import should fill this tables ? Thanks a lot for your help -- Santi Camps (Earcon S.L.) http://www.earcon.com http://www.kmkey.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] example for the following?
Hello Jamie, I would say to persist that object either you rewrite it in python, (and use the persistent module) or fiddle around the __getstate__ and __setstate__ methods. Though no idea how you can get those working with your swig'ed objects. See: http://docs.python.org/library/pickle.html Thursday, January 22, 2009, 5:46:08 PM, you wrote: JM Hi, JM I'm using ZODB in a desktop app (i.e. not using zope). JM I have class A which contains a list of instances for class B. JM I can persist class A when the list of class B is empty. When i JM add an instance of class B to the list (and set _p_changed = 1) i get the error: JM TypeError: can't pickle PySwigObject objects JM Can someone please give me a link of some advice in how i go JM about persisting lists of class instances? JM Thanks, JM Jamie JM JM ~~~ JM Jamie McQuay JM Scimatic Software Inc. JM www.scimatic.com JM JM We build software for scientists. JM -- Best regards, Adam GROSZERmailto:agros...@gmail.com -- Quote of the day: You will feel hungry again in another hour. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage pack problems
Santi, I hope you don't mind me discussing your database in public. I'm not going to talk about anything that looks like it could be private. Other RelStorage users might benefit from the analysis. Looking at your database, I see that something bad happened just before transaction 250499913441768123. That number is an encoded time stamp: from ZODB.TimeStamp import TimeStamp from ZODB.utils import p64 str(TimeStamp(p64(250499913441768123))) '2008-11-17 19:36:04.913130' The transaction log entry says initial database creation, which means that the database had no root object (OID 0), so ZODB created one and started a brand new database. Strange! This happened about an hour after a transaction labeled: /asp_ekartek/kmkey_iso/portal_setup/manage_doUpgrades I'm guessing that an upgrade script did something horribly wrong that day. Furthermore, the entry for OID 0 in the current_object table points to an old transaction rather than the most recent transaction that modified OID 0. That's not supposed to happen, even when you undo. I hope RelStorage didn't do that! Did you or someone on your team change current_object by hand? I can understand why you would, since a simple modification to current_object would be a nice quick fix for the broken upgrade. The fix would not be complete, though, because now the object_state table and the current_object table disagree on the current state of OID 0. According to object_state, even now, the current state of OID 0 still points to the small object graph that was accidentally created on November 17. The pack code relies more on object_state than on current_object, so the pack code sees only a handful of objects that are reachable. Packing with garbage collection removes everything that is not reachable. The current_object table is really just a cache of object_state. If the schema were fully normalized, there would not be a current_object table. In theory, the current_object table makes it possible to load ZODB objects quickly. But if the current_object table results in problems like this, I need to consider alternatives. In any case, I believe you can get out of this mess pretty easily. You need to delete the extra object states for OID 0 created on November 17. I tried this in my copy of your database: delete from object_state where zoid = 0 and tid in ( 250499913441768123, 250499913748614178); After that, select count(1) from object_state where zoid = 0; should tell you there is only one state in the database for OID 0. Packing should work fine then. It seemed to do the right thing on my copy, but I don't have your application code to check it. Please let me know whether current_object was edited by hand. If it was, then we have a documentation bug. If RelStorage did that on its own, we may have a software bug. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Relstorage pack problems
Laurence Rowe wrote: Shane Hathaway wrote: I should note that this KeyError occurs while trying to report on a KeyError. I need to fix that. Fortunately, the same error pops out anyway. There's a fix for this in the Jarn branch. Note that to collect more interesting data it rolls back the load connection at this point, relying on the KeyError to cause the transaction to fail. I saw that, but I didn't feel good about the idea of rolling back the transaction. I wanted the log message to show the state of the database as seen by that connection at that moment in time. That's the kind of information you can't get with a SQL query executed at a later time. I figured that if you really want to see all object states, you can just use SQL queries later. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] How to turn off 'GC' when packing on ZODB3.6.2
eastxing wrote: For choice 3, 'RelStorage' provide a 'ZODBConverter', but I do not to know if it can used with ZODB3.6.2? I think RelStorage could support ZODB 3.6 without much effort, but I haven't tested it. I'd go with Jim's suggestion of using a new ZEO server. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
