RE: [Zope3-Users] zodb objects backup
It would be very nice if there were a faq or other doc addressing all of these scalability-related questions systematically for someone like me who understands what a relational database is doing for them (in return for squashing all their objects). It seems I'm not the only one with these concerns. :) Welcome to Open Source, we look forward to seeing your newly-written faq as it becomes available online ;-) Well, I've got enough info to write one by now... only, not having reviewed or tested the code, I couldn't actually vouch for it. But seriously, coming to Zope for an app server, I had no idea that ZODB was as robust as it seems to be. I think, from a marketing standpoint for the Zope corp (and other in the community deriving sustenance from providing zope-related products and services), it would make a lot of sense if they at least made the claim in a prominent place that ZODB was a serious candidate to replace a RDBMS if you didn't absolutely need SQL. Collecting the info together could be put on the todo list, but motivating people to seek it out would be a good first step. In the meantime, I had an idea about my current implementation: maybe instead of __getstate__ and __setstate__ I should put the external data in _v_data (marking as volatile). Then I could trap for its existence, and load if necessary; and also have an explicit refresh wired to a button in the GUI. Yup, this is exactly what _v_ was designed for... Great. - Shaun ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] zodb objects backup
Shaun Cutts wrote: So far so good, modulo the replication issue. (We don't have much funding yet... but if initial launch goes well, we're hopeful :)). As people have mentioned, you can use repozo to get almost the same effect. Either that or do app-level replication ;-) It would be very nice if there were a faq or other doc addressing all of these scalability-related questions systematically for someone like me who understands what a relational database is doing for them (in return for squashing all their objects). It seems I'm not the only one with these concerns. :) Welcome to Open Source, we look forward to seeing your newly-written faq as it becomes available online ;-) In the meantime, I had an idea about my current implementation: maybe instead of __getstate__ and __setstate__ I should put the external data in _v_data (marking as volatile). Then I could trap for its existence, and load if necessary; and also have an explicit refresh wired to a button in the GUI. Yup, this is exactly what _v_ was designed for... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
RE: [Zope3-Users] zodb objects backup
From: Tom Dossis [mailto:[EMAIL PROTECTED] Shaun Cutts wrote: But with replication, this issue is taken care of. (Too bad replication isn't part of the core functionality) Maybe soon.. http://hathawaymix.org/Software/PGStorage Great! Another potential issue on the scalability front I had forgotten about earlier: How do BTrees perform under lots of concurrent updates? (I know this is a tricky one, as implementations can get pretty complex to deal with this.) I note that http://www.zope.org/Wikis/ZODB/FrontPage/guide/node6.html#SECTION0006300 00 Says that As with a Python dictionary or list, you should not mutate a BTree-based data structure while iterating over it. Does this apply only to thread-local modifications or to any modification by anyone else? Ie, are BTrees versioned as the ZODB is... if I'm iterating over a BTree in my process (in ZEO, say), and another process modifies the BTree, does that sometimes show up in my copy, or only after commit? Also, wrt ConflictError -- is the BTree considered one object, or are the python objects (buckets, tree structure, ...) treated separately? In general, are the BTrees just written naively on top of ZODB, or do they interact in some special way with the storage? If they are just sitting on top of ZODB, it would seem that this dims the possibility that one could use the ZODB as a Postgres replacement. - Shaun ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
[Zope3-Users] zodb objects backup
Hello, what would be the best way of taking a backup of persistent objects inside Data.fs with possibility to rebuild it on a fresh Zope 3 install in case of a disaster recovery lets say. Thanks Alen ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] zodb objects backup
--On 26. Februar 2006 00:04:39 +1100 Alen Stanisic [EMAIL PROTECTED] wrote: Hello, what would be the best way of taking a backup of persistent objects inside Data.fs with possibility to rebuild it on a fresh Zope 3 install in case of a disaster recovery lets say. Just backup the Data.fs file. -aj pgppCQEp73WCn.pgp Description: PGP signature ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] zodb objects backup
On Sat, 2006-02-25 at 07:52 -0600, Andreas Jung wrote: --On 26. Februar 2006 00:04:39 +1100 Alen Stanisic [EMAIL PROTECTED] wrote: Hello, what would be the best way of taking a backup of persistent objects inside Data.fs with possibility to rebuild it on a fresh Zope 3 install in case of a disaster recovery lets say. Just backup the Data.fs file. For some reason it doesn't feel completely safe just relying on Data.fs. Maybe I am thinking too much in rdb land and transaction logging where you could rebuild your db from the logs. Alen ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
RE: [Zope3-Users] zodb objects backup
On Sat, 2006-02-25 at 08:59 -0600, Andreas Jung wrote: --On 26. Februar 2006 01:52:29 +1100 Alen Stanisic [EMAIL PROTECTED] wrote: For some reason it doesn't feel completely safe just relying on Data.fs. That means what? Why shouln't it be safe...please come up with some reasonable arguments.. -aj I did mention that it could be because most rdb systems have a database and also keep transaction logs. In case of a failure you put the latest backup of the db and transaction logs together and you could rebuild your db to the point just before the failure. If you only had a daily back up of your db you could potentially lose a full day of transactions. One could also mention failover: if one computer with Data.fs goes down, you're down, period; whereas many RDBMSes support keeping slave copies of the database, which are then available. Also, there are ACID transactions (see http://en.wikipedia.org/wiki/ACID) in a good RDBMS. I don't know how zodb ensures consistency if there are multiple concurrent users, and can we rollback gracefully? ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] zodb objects backup
Alen, please see http://www.zope.org/Wikis/ZODB/FileStorageBackup Shaun, many of the other questions in this thread--and others recently--are answered in this guide: http://www.zope.org/Wikis/ZODB/FrontPage/guide/index.html It is highly recommended reading if you are doing serious Zope 3 apps. Both of these are found in the ZODB wiki, which has some other helpful docs: http://www.zope.org/Wikis/ZODB/FrontPage Gary ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
Re: [Zope3-Users] zodb objects backup
Thanks Gary. On Sat, 2006-02-25 at 15:39 -0500, Gary Poster wrote: Alen, please see http://www.zope.org/Wikis/ZODB/FileStorageBackup Shaun, many of the other questions in this thread--and others recently--are answered in this guide: http://www.zope.org/Wikis/ZODB/FrontPage/guide/index.html It is highly recommended reading if you are doing serious Zope 3 apps. Both of these are found in the ZODB wiki, which has some other helpful docs: http://www.zope.org/Wikis/ZODB/FrontPage Gary ___ Zope3-users mailing list Zope3-users@zope.org http://mail.zope.org/mailman/listinfo/zope3-users
