Re: [Zope] Re: Questions about demostorage
Duncan Booth wrote: Urm, that would involve taking a complete copy of the db at open time... Not knowing the details of the implementation I don't know that it would. Knowing the details as I do, I'm telling you it would... Got that, but the problem is that the test database isn't exactly identical to the live one (e.g. the ldap config points to the test ldap server) but Then fix the flakey test environment rather than applying even more fragile machinery in an attempt to ignore the real problem ;-) I'm sure we could script the tweaks needed. More significantly is the pain involved in copying 10Gb from the live system onto dev on more than an occasional basis. It's still worth it... The problem I'm trying to address is that sometimes we make mistakes when updating the live server because of minor differences between the test and live environments. Ah, a flakey upgrade process too ;-) Take a look at Stepper. One of it's main uses is to write upgrade scripts than can be consistently run when doing upgrades and test upgrades on a development environment. took some time and caused site errors until it was complete. I have a script which I believe should do the migration quickly but I want to be absolutely positive it is going to work without any hitches. In your position, I would have little to no confidence that using DemoStorage in the way you suggest would meet your requirements... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: Questions about demostorage
Chris Withers [EMAIL PROTECTED] wrote: (So in other words, the connection from the demostorage front-end to the zeo-client isn't frozen at the point when the storage is opened which might have been a plausible alternative). Urm, that would involve taking a complete copy of the db at open time... Not knowing the details of the implementation I don't know that it would. I'd have thought it would involve copying the index rather than the data, and since I don't know the implementation of the index it might only need to do a copy-on-write as and when parts of the index change. But really, what you're doing is not what DemoStorage was designed for and will likely end in tears or a lot of hair pulling... I know, that's why I'm trying to establish the boundaries of what I can expect from it. What I'm hoping to achieve is a way of testing upgrades on our live system in an environment as close as possible to the real live environment, but without actually affecting the live system in any way. Demostorage seems to offer that as an option. It really doesn't ;-) Do this kinda testing the same way that everyone does: build yourself a uat environment that mirrors your production environment, maybe with a few less zeo clients, run it off a copy of your production zodb and then test there... Got that, but the problem is that the test database isn't exactly identical to the live one (e.g. the ldap config points to the test ldap server) but I'm sure we could script the tweaks needed. More significantly is the pain involved in copying 10Gb from the live system onto dev on more than an occasional basis. The problem I'm trying to address is that sometimes we make mistakes when updating the live server because of minor differences between the test and live environments. So on the live servers we have a separate zope instance for final testing. It has its own copy of the Products folder (checked out from the relevant svn tag) and when we are happy that the checkout works as expected I just rsync that Products folder onto the live Products. I'm not suggesting running the system on demostorage for long periods of time. Simply long enough to check that whatever weird buildout changes we make are doing are going to work. That means that the only guarantee I really need is that the live system won't be affected by any changes to the test system; if the test system becomes internally inconsistent and falls over I'll just restart it and try again. For example, we are currently running with an ancient svn checkout of CacheFu. I know, from playing on the dev system, that migrating from that to [a slightly patched copy of] the current version is possible, but it took some time and caused site errors until it was complete. I have a script which I believe should do the migration quickly but I want to be absolutely positive it is going to work without any hitches. If I can do a dry run in the live environment then I should be able to get that confidence. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Re: Questions about demostorage
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Duncan Booth wrote: Can someone explain to me what is and is not safe to do with demostorage? Say I have a zope setup involving a zeo server and clients, and I create another zope instance with demostorage wrapped around both zeoclient mounts (the catalogs are mounted separately from the rest of the content). The main zope setup remains active while I'm using the demostorage setup, so content will be added/edited or deleted in the zeo zodb. A bit of experimenting seems to indicate if I edit an object in the main system, the demostorage version picks up the changes until such time as I edit the same object in the demostorage instance. (So in other words, the connection from the demostorage front-end to the zeo-client isn't frozen at the point when the storage is opened which might have been a plausible alternative). THat is correct. DemoStorage itself won't write to the backing layer, but other clients (your main appservers, for instance) might. If you want to freeze the backing layer, then you need to use the 'stop' parameter when creating the FileStorage. Unfortunately, it doesn't seem to be exposed at the ZConfig level. More unfortunately for your case, it is not available at all for ZEO.ClientStorage. What I'm a little worried about is the catalog. I'd guess that there is no guarantee that the demostorage instance gets a catalog which is consistent with the rest of the data (so I could easily get site errors in the demostorage instance), but there should be no danger that any changes I make can affect the live site and restarting the demo instance would get it back into a consistent state, at least for a while. Is that a reasonable interpretation? If both storages are wrapped as DemoStorages, yes. What I'm hoping to achieve is a way of testing upgrades on our live system in an environment as close as possible to the real live environment, but without actually affecting the live system in any way. Demostorage seems to offer that as an option. I would just bite the bullet and use repozo to replicate your database to the dev system. Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFFgroV+gerLs4ltQ4RAj+TAJ0SrqNuehRvQoNH7tEPDdqY54LDowCffVgx ormTTUilfWeGDY42tlktFKo= =F87X -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )