Re: [ZODB-Dev] Migrating from Plone 4.1.4 ZEO data.fs and blobs to Relstorage
On Sun, Apr 15, 2012 at 6:36 PM, Sam Wilson s...@dotsec.com wrote: Hanno gave me a hand off list with a few bits of config that I couldn't find listed in the official documentation on http://pypi.python.org/pypi/RelStorage#zodbconvert Once you figure out the configuration options, zodbconvert is indeed a gem -- should you ever feel inclined, it also makes a decent way to backup RelStorage to a FileStorage should that be useful for your backup processes. I use this to rsync gzipped daily FileStorage snapshots of my database from production RelStorage to development on my workstation (which I load into RelStorage via another zodbconvert step), and it also supplements my pgdump backups from PostgreSQL. Sean ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Migrating from Plone 4.1.4 ZEO data.fs and blobs to Relstorage
On Fri, Apr 13, 2012 at 02:25, Sam Wilson s...@dotsec.com wrote: I have configured zodbconvert to import the data.fs however I cannot for the life of me find doco on how to migrate the blob (bushy layout) into RelStorage. Note that I cannot recommend putting ZODB blobs into any relstorage database apart from Oracle. PostgreSQL doesn't handle large blobs very well (storing them internally as a series of chunks in a dedicated table), while MySQL doesn't really support blobs at all. We have one customer on RelStorage on a Oracle cluster (RelStorage was commissioned for them, in fact) where all ZODB blobs also live in the same database. Performance is quite decent but the customer doesn't try to store more than a few MBs per file. Better keep them in a NFS setup for most cases. -- Martijn Pieters ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Upgraded from ZODB 3.8 to 3.10, do I need to upgrade my Data.fs?
Paul Warner paul.warner at gmail.com writes: We have been running an in house application on ZODB 3.8.2 and recently upgraded to 3.10.5. We run a ZEO server with FileStorage. It seems like everything works great, no actions on our part, but I wondered if it is suggested to some how migrate the on disk FileStorage to take advantage of new features or optimizations in 3.10? The format and data in the file storage has stayed exactly the same for quite some time, so no migration is necessary. For 3.10 the index file format (data.fs.index) has changed, but this has happened transparently. ZODB 3.8 introduced experimental support for blobs (binary large objects), which have gotten stable and production ready with 3.9. There's no good documentation for them, but z3c.blobfile might serve as an example (http://svn.zope.org/z3c.blobfile/trunk/src/z3c/blobfile/). If you have any data that's bytes of 1mb or more each, you should investigate blobs. Not having to load all that data into the ZODB caches alone is really helpful, not to mention the various nice effects of having a smaller main file storage or the various supported backends for blobs. Apart from those, there's been a multitude of new config options listed in the changelogs (http://pypi.python.org/pypi/ZODB3/3.9.7#id14 and http://pypi.python.org/pypi/ZODB3/3.10.5#id10). It's hard to tell, if any of those would be helpful in your particular case. Hanno ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Upgraded from ZODB 3.8 to 3.10, do I need to upgrade my Data.fs?
Many thanks, this helps to clear things up. Those new features look great. We do store a few large objects so I will look at moving them over to blobs. -Paul On Mon, Apr 16, 2012 at 9:17 AM, Hanno Schlichting ha...@hannosch.eu wrote: Paul Warner paul.warner at gmail.com writes: We have been running an in house application on ZODB 3.8.2 and recently upgraded to 3.10.5. We run a ZEO server with FileStorage. It seems like everything works great, no actions on our part, but I wondered if it is suggested to some how migrate the on disk FileStorage to take advantage of new features or optimizations in 3.10? The format and data in the file storage has stayed exactly the same for quite some time, so no migration is necessary. For 3.10 the index file format (data.fs.index) has changed, but this has happened transparently. ZODB 3.8 introduced experimental support for blobs (binary large objects), which have gotten stable and production ready with 3.9. There's no good documentation for them, but z3c.blobfile might serve as an example (http://svn.zope.org/z3c.blobfile/trunk/src/z3c/blobfile/). If you have any data that's bytes of 1mb or more each, you should investigate blobs. Not having to load all that data into the ZODB caches alone is really helpful, not to mention the various nice effects of having a smaller main file storage or the various supported backends for blobs. Apart from those, there's been a multitude of new config options listed in the changelogs (http://pypi.python.org/pypi/ZODB3/3.9.7#id14 and http://pypi.python.org/pypi/ZODB3/3.10.5#id10). It's hard to tell, if any of those would be helpful in your particular case. Hanno ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
