Re: [ZODB-Dev] ZODB not saving sometimes
Gary, I have been using the ZODB for about a year and a half with a bookstore application. I am just now about ready to put it out on the internet for people to use. I have had the same problem with saving data. I have tried alot of things. But I have never gotten the database to save consistently. I can create an x number of records one right after the other that uses the exact same code to save them, and it is likely that all of them will save perfectly except one - or maybe two. I am posting another question because I have got to get this fixed. I do not understand the ZODB very well and I have been pulling my hair out trying to figure out why my data is not there when I restart my server. I am posting another question about this trying to figure out how the ZODB saves. Maybe we can both figure this thing out. Your post hits home! Thanks, Tom mottai wrote: Hi, I have inherited a rather large project using ZODB, so would be hard to paste code in here for example, so I will try to explain my problem well. This app allows creating new users, the users always save good. This app allows taking surveys, and these survey results are stored the same way as the new users. The Objects are persistent. The surveys sometimes disappear when the app is closed, and reopened. I've verified they really are not in the database. What is confusing is I never see the behavior on my xp Pro machine. A xp Home machine here very often does not save the data on the commit, but even more odd, after a while starts to work and save. I've tried forcing the objects to be dirty with ._p_changed = 1 and this did not help. I've verified the commit does not throw any errors, and that the db is closed properly on exit. I'm using python 2.5 and zodb ZODB3-3.8.0-py2.5-win32.egg I appreciate any thoughts at all. Sorry for the lack of code example. If need be I can try to piece some examples together from different files. Thanks, Gary -- View this message in context: http://www.nabble.com/ZODB-not-saving-sometimes-tp17927283p18055848.html Sent from the Zope - ZODB-Dev mailing list archive at Nabble.com. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB design. Size and saving records
ZODB guys,
I have a bookstore application in which I use the ZODB in a simple way. The
database saves stores and records in the stores.
I open the database as follows where the databasePath argument points to my
bookserver.fs file.
from ZODB import FileStorage, DB
import transaction
import BTrees.OOBTree, BTrees.IOBTree
def open(self, databasePath):
self.bookDatabase_storage =
FileStorage.FileStorage(databasePath)
self.bookDatabase_db =
DB(self.bookDatabase_storage)
self.bookDatabase_connection = self.bookDatabase_db.open()
self.dbRoot = self.bookDatabase_connection.root()
# our btrees
ibTree = BTrees.IOBTree
I define the key 'books' to be an IOBTree
db.dbRoot['books'] = ibTree.IOBTree()
And when the user creates a store I define the book number to be an IOBTree
db.dbRoot['books'][storeNumber] = ibTree.IOBTree()
Then I save the book using storeNumber and bookNumber as keys.
record = {'title':The Grapes of Wrath, 'author':John
Steinbeck,'publisher':Randomhouse}
db.dbRoot['books'][storeNumber][bookNumber] = record
So now I can qualify an entire store by using
currentStore = db.dbRoot['books'][storeNumber]
And a single book by
currentBook=
db.dbRoot['books'][storeNumber][bookNumber]
And look at the books in the store by
for k, v in currentStore.values():
print k, v
I have two questions.
1) When I already have a storeNumber and I save a record to
db.dbRoot['books'][storeNumber][bookNumber] =
record
I have to set the _p_changed flag on the 'books' IOBTREE
structure to get
the book to save.
db.dbRoot['books']._p_changed = True
Which means that it saves the ENTIRE 'books' IOBTREE' structure
every time
a save a single book. (at least it looks like it is doing this). When I
edit a book and save it the database grows by more than 64k. And it looks
like it will get worse and worse as more books are added.
Am I looking at this correctly. Or am I doing something really
ignorant?
2) When I assign a value to the db such as
db.dbRoot['books'][storeNumber][bookNumber] = record
I initially assumed that setting the _p_changed flag on
the storeNumber
key would only save the single record that I want to save. As mentioned
above, I am setting the flag on the 'books' IOBTREE also. Should I have to
set it on both. I have come to the conclusion that the _p_changed flag must
be set at the highest key level. ie.. 'books'. Again, Am I doing something
really ignorant?
Thank you for your responses. I really need to know and get this fixed
before my wife divorces me! She spends time entering books and we still
seem to not really know when the changes are going to permanently be saved.
Tom
--
View this message in context:
http://www.nabble.com/ZODB-design.-Size-and-saving-records-tp18055895p18055895.html
Sent from the Zope - ZODB-Dev mailing list archive at Nabble.com.
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] ZODB not saving sometimes
--On 22. Juni 2008 08:49:32 -0700 tsmiller [EMAIL PROTECTED] wrote: Gary, I have been using the ZODB for about a year and a half with a bookstore application. I am just now about ready to put it out on the internet for people to use. I have had the same problem with saving data. I have tried alot of things. But I have never gotten the database to save consistently. I can create an x number of records one right after the other that uses the exact same code to save them, and it is likely that all of them will save perfectly except one - or maybe two. We have never seen that - except with badly written application code. Commiting a transaction should always commit the data. That's the sense of a transaction system. The only reason I can imagine causing such a failure: bare try..except with in your code suppressing ZODB conflict errors. Andreas pgpf0qC3ICt6U.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: ZODB not saving sometimes
Andreas Jung wrote: --On 22. Juni 2008 08:49:32 -0700 tsmiller [EMAIL PROTECTED] wrote: Gary, I have been using the ZODB for about a year and a half with a bookstore application. I am just now about ready to put it out on the internet for people to use. I have had the same problem with saving data. I have tried alot of things. But I have never gotten the database to save consistently. I can create an x number of records one right after the other that uses the exact same code to save them, and it is likely that all of them will save perfectly except one - or maybe two. We have never seen that - except with badly written application code. Commiting a transaction should always commit the data. That's the sense of a transaction system. The only reason I can imagine causing such a failure: bare try..except with in your code suppressing ZODB conflict errors. The other likely cause of this is modifying non-persistent sub objects and not setting _p_changed = True on the parent persistent object. e.g: dbroot['a_list'] = [1, 2, 3] transaction.commit() a_list = dbroot['a_list'] a_list.append(4) transaction.commit() The second commit actually has no effect as the persistence machinary has not been notified that the object has changed. This is not immediately apparent though as the 'live' shows what you expect: a_list [1, 2, 3, 4] And if a later transaction also modifies the persistent object, then all of the data is saved. To avoid this, avoid using mutable, non-persistent types for storage in the ZODB, replace lists and dicts with PersistentList and PersistentMapping. Laurence ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
