[ZODB-Dev] storage record_iternext() question

2014-12-09 Thread Sean Upton
Does record_iternext() for a storage supporting it only iterate over *current* object records (FileStorage)? I have implemented a monkey-patch for RelStorage to support this (to make zodbupdate work with it) [1], but my patch only works (without conflict errors) when the database is fully packed

[ZODB-Dev] RelStorage: ConflictError causes deadlock without RELSTORAGE_ABORT_EARLY

2014-07-19 Thread Sean Upton
Folks, I have been dealing with locking issues and RelStorage for the past few days, and want to verify what I believe is a bug: without RELSTORAGE_ABORT_EARLY set in environment, tpc_vote() could potentially leave an ILocker adapter setting an RDBMS table lock (originally set in either

Re: [ZODB-Dev] RelStorage PostgreSQLLocker deadlock/wait problem

2014-07-18 Thread Sean Upton
time to commit? Or is this lock timeout just used at the very beginning of a transaction? Forgive my ignorance. Any input on this change? Sean On Thu, Jul 17, 2014 at 1:20 PM, Sean Upton sdup...@gmail.com wrote: Any insights from RelStorage users appreciated on this; I have a case where

Re: [ZODB-Dev] RelStorage PostgreSQLLocker deadlock/wait problem

2014-07-18 Thread Sean Upton
On Thu, Jul 17, 2014 at 2:37 PM, Leonardo Rochael Almeida leoroch...@gmail.com wrote: I seem to recall, though I couldn't find references to it now, that this could happen if there is a firewall between the RelStorage using processes/threads and your database. If a long time passes without

[ZODB-Dev] RelStorage PostgreSQLLocker deadlock/wait problem

2014-07-17 Thread Sean Upton
Any insights from RelStorage users appreciated on this; I have a case where multiple Zope2 instances (each with four threads) get stuck (most/all threads) waiting on execution of LOCK TABLE statements (presumably stuck commits) in relstorage.adapters.locker.PostgreSQLLocker.hold_commit_lock [1].

Re: [ZODB-Dev] grab zodb dict keys by search term

2013-12-12 Thread Sean Upton
On Wed, Dec 11, 2013 at 3:27 AM, Tamer Higazi tamerito...@arcor.de wrote: Perhaps it's my fault, the way I stored the data. The point is, that I am looking the fastest and performant way to grab the data from a big pool. I do not think folks are suggesting you iterate over the big pool

Re: [ZODB-Dev] modifying tree under root and save ?!

2013-12-09 Thread Sean Upton
On Mon, Dec 9, 2013 at 10:19 AM, Tamer Higazi tamerito...@arcor.de wrote: Is there a way, to change the ZODB keys subcontent without to copy the entire entry in the memory ?! You should -- as suggested previously -- for mutable collections, use PersistentDict and PersistentList, otherwise

[ZODB-Dev] Attempt to release connection when storage is None

2013-08-28 Thread Sean Upton
I am seeing something curious on a new Zope2 deployment (2.13.21, Plone 4.3) where ZODB attempts to release a storage on Control+C shutdown of a foregrounded zope2 instance, but a connection has a _storage attribute equal to None, thus this traceback: http://pastie.org/8275219 This is Zope

Re: [ZODB-Dev] Changing namespace - best strategy

2013-08-22 Thread Sean Upton
Caveat to this below. On Mon, Jul 15, 2013 at 7:54 AM, Marius Gedminas mar...@gedmin.as wrote: On Mon, Jul 15, 2013 at 03:36:21PM +0200, Pedro Ferreira wrote: We need to move a considerable number of persistent objects (~40K) from one module to another, and we were wondering what is the

[ZODB-Dev] RelStorage, PostgreSQL backup method comparison

2012-12-26 Thread Sean Upton
For cron job RelStorage backups (databse not including blobs backed-up seperately, using a PostgreSQL 9.0.x backend), I use both zodbconvert to save FileStorage copies of my database, and pgdump for low-level binary dumps (pg_restore custom format, preserving postgres OIDs). bzip2-compressed, the

Re: [ZODB-Dev] RelStorage - zodbconvert script and the --clear flag

2012-09-09 Thread Sean Upton
On Thu, Sep 1, 2011 at 12:45 AM, Shane Hathaway sh...@hathawaymix.org wrote: On 08/31/2011 05:11 PM, Darryl Dixon - Winterhouse Consulting wrote: Just had a quick query from my friendly local DBA; he wanted to know why --clear was using DELETE rather than TRUNCATE; his comments were along the

Re: [ZODB-Dev] Storing blob data out of zodb into amazon s3?

2012-07-05 Thread Sean Upton
On Thu, Jul 5, 2012 at 1:05 PM, Noe Nieto nni...@noenieto.com wrote: I was wondering if it's possible to write some sort of cloudstorage plugin for ZODB that saves blobs in amazon S3 or something similar. The idea is cool because all things cloud are cool. But there might be some shortcomings

Re: [ZODB-Dev] Migrating from Plone 4.1.4 ZEO data.fs and blobs to Relstorage

2012-04-16 Thread Sean Upton
On Sun, Apr 15, 2012 at 6:36 PM, Sam Wilson s...@dotsec.com wrote: Hanno gave me a hand off list with a few bits of config that I couldn't find listed in the official documentation on http://pypi.python.org/pypi/RelStorage#zodbconvert Once you figure out the configuration options, zodbconvert

Re: [ZODB-Dev] RelStorage zodbconvert, ConflictError on Zope2 startup

2011-07-18 Thread Sean Upton
On Fri, Jul 15, 2011 at 5:35 PM, Shane Hathaway sh...@hathawaymix.org wro= te: I am thinking of changing the memcache code to use a random per-database = key prefix. =A0If I had done that already, you would not have run into this problem, since clearing the database would cause RelStorage to

Re: [ZODB-Dev] RelStorage zodbconvert, ConflictError on Zope2 startup

2011-07-14 Thread Sean Upton
On Thu, Jul 14, 2011 at 1:43 PM, Hanno Schlichting ha...@hannosch.eu wrote: On Thu, Jul 14, 2011 at 9:32 PM, Sean Upton sdup...@gmail.com wrote: Full traceback: http://pastie.org/2214036 That's weird, you are getting a conflict error on inserting the root application object (oid 0x00). I am