On Tue, Jun 14, 2011 at 22:22, Martijn Pieters <m...@zopatista.com> wrote:
> I am considering altering the ON COMMIT behaviour of the
> temp_blob_chunk table to TRUNCATE to extend this behaviour to all
> remaining blob rows on commit. However, such events should be rare
> enough, and there is a `vacuumlo` command that can find orphaned blob
> objects in the pg_largeobjects table and clean these out.

I've now made this change. This means that the temp_blob_chunk table
is dropped when a storage is opened, not on commit, and instead it's
contents are truncated on commit and any oids left in there that
didn't make it to blob_chunk are unlinked.

> I'm also writing a unittest that'll exercise the blob storage a little
> more, by creating a file larger than the maximum storable in one
> chunk, then upload, commit and re-download to see if the data survived
> intact. This'll be set to test level 2 to not clog up the normal test
> cycle as generating a 4GB+ random file can take a little while! :-)


The tests run at test level 2, meaning that they do not run by
default. You need to either set the test level to 2 (-a 2) or use the
--all switch to run all levels regardless. The PostgreSQL version
(2GB+ blob size) takes a healthy quarter of an hour on my laptop to
complete one such test.

> Last but not least I'll need to write a migration script for those
> users of RelStorage 1.5.0b2 already in production.

I have figured out how to do this (see
http://archives.postgresql.org/pgsql-general/2009-01/msg00771.php). I
won't try to consolidate the blob_chunks, so you'll end up with 1MB
oid blobs but that's fine as the download code can handle those

The query just needs to be written and tested.

Martijn Pieters
For more information about ZODB, see the ZODB Wiki:

ZODB-Dev mailing list  -  ZODB-Dev@zope.org

Reply via email to