On Tue, Jun 14, 2011 at 22:22, Martijn Pieters <m...@zopatista.com> wrote: > I am considering altering the ON COMMIT behaviour of the > temp_blob_chunk table to TRUNCATE to extend this behaviour to all > remaining blob rows on commit. However, such events should be rare > enough, and there is a `vacuumlo` command that can find orphaned blob > objects in the pg_largeobjects table and clean these out.
I've now made this change. This means that the temp_blob_chunk table is dropped when a storage is opened, not on commit, and instead it's contents are truncated on commit and any oids left in there that didn't make it to blob_chunk are unlinked. > I'm also writing a unittest that'll exercise the blob storage a little > more, by creating a file larger than the maximum storable in one > chunk, then upload, commit and re-download to see if the data survived > intact. This'll be set to test level 2 to not clog up the normal test > cycle as generating a 4GB+ random file can take a little while! :-) See http://zope3.pov.lt/trac/changeset/121949/relstorage/branches/postgres_blob_oid The tests run at test level 2, meaning that they do not run by default. You need to either set the test level to 2 (-a 2) or use the --all switch to run all levels regardless. The PostgreSQL version (2GB+ blob size) takes a healthy quarter of an hour on my laptop to complete one such test. > Last but not least I'll need to write a migration script for those > users of RelStorage 1.5.0b2 already in production. I have figured out how to do this (see http://archives.postgresql.org/pgsql-general/2009-01/msg00771.php). I won't try to consolidate the blob_chunks, so you'll end up with 1MB oid blobs but that's fine as the download code can handle those perfectly. The query just needs to be written and tested. -- Martijn Pieters _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
