On 02/06/2013 04:23 AM, Jürgen Herrmann wrote:
I think this is not entirely correct. I ran in to problems serveral
times when new_oid was emptied! Maybe Shane can confirm this?
(results in read conlfict errors)
Ah, that's true. You do need to replicate new_oid.
Then I'd like to talk a little about my current relstorage setup here:
It's backed by mysql, history-preserving setup. Recently one of our
DBs started to grow very quickly and it's object_state.ibd (InnoDB)
file is just over 86GB as of today. Packing now fails due to mysql
not being able to complete sorts in the object_ref table. object_ref
is also very big (36GB MYD file, 25GB MYI file). I took a backup of the
DB and let zodbconvert convert it back to a FileStorage, the resulting
file is 6GB (!). I will pack it and see how big it is then. I will
also investigate how big on disk this DB would be when stored in
postgresql. This situation poses another problem for us: using
zodbconvert to convert this mess to a Filestorage tages just over an
hour when writing to a ramdisk. I suspect converting to postgres
will take more than 10 hours, which is unacceptable for us as this
is a live database an cannot be offline for more than 2-3 hours in
the nicht. So we will have to investigate into a special zodbconvert
that uses a two step process:
1. import transactions to new storage from a mysql db backup
2. import "rest" of transactions that occurred after the backup was
made from the "live" database (which is offline for that time
of course)
looking at zodbconvert using copyTransactionsFrom() i thnik this
should be possible but up to now i did non investigate furhter.
maybe shane could confirm this? maybe this could also be transformed
into a neat way of getting incremental backups out of zodbs in
general?
Yes, that could work.
As for MySQL growing tables without bounds... well, that wouldn't
surprise me very much.
Shane
_______________________________________________
For more information about ZODB, see http://zodb.org/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
