Re: [ZODB-Dev] [Zope-dev] when did transaction lose the ability to be usable as a context manager?

2013-02-06 Thread Chris Withers
On 06/02/2013 11:47, Wichert Akkerman wrote: You can also use the transaction manager directly instead of the bound methods transaction lifts out of it: import transaction with transaction.manager: pass yeah, this is what I'll do. Chris -- Simplistix - Content Management, Batch Proces

[ZODB-Dev] when did transaction lose the ability to be usable as a context manager?

2013-02-06 Thread Chris Withers
Hi All, I used to do this: >>> import transaction >>> with transaction: ... print 'hello' ... Traceback (most recent call last): File "", line 1, in AttributeError: __exit__ When did that stop working and what should I now do instead? cheers, Chris -- Simplistix - Content Management, Ba

Re: [ZODB-Dev] API question

2013-01-25 Thread Chris Withers
On 16/01/2013 18:01, Tres Seaver wrote: from ZDOB.DB import DB # This one can even be ambiguous now FTR, I don't like this style. Somewhat a matter of taste. Sure. I don't like using APIs via long, multi-dotted paths. What Tres said ;-) - - Nose gives us easy access to Ned Batchelder's

Re: [ZODB-Dev] RelStorage breaks history tab - solution?

2012-05-23 Thread Chris Withers
Hi Shane, On 23/05/2012 16:27, Chris Withers wrote: Okay, the issue appears to be that, in some circumstances, RelStorage is leaving the read connection with an open transaction that isn't rolled back. Upon investigation, this turns out to be the history method itself! :-) Switching t

Re: [ZODB-Dev] RelStorage breaks history tab

2012-05-23 Thread Chris Withers
Hi Shane, On 22/05/2012 06:37, Chris Withers wrote: So, the problem is that, with a Zope 2 app server, any revisions made to an object (say, a Page Template) don't show in the history tab of that object *until* the app server is restarted. Okay, the issue appears to be that, in

[ZODB-Dev] RelStorage breaks history tab

2012-05-21 Thread Chris Withers
Hi All, I've been chatting with Shane about this off-list but wanted to see if anyone else has experience of this and has found a solution. So, the problem is that, with a Zope 2 app server, any revisions made to an object (say, a Page Template) don't show in the history tab of that object *

[ZODB-Dev] git clone of relstorage

2012-05-15 Thread Chris Withers
Hi All, In case anyone else is interested, I've git svn clone'd svn://svn.zope.org/repos/main/relstorage and pushed the result up here: https://github.com/cjw296/relstorage It's only a master branch, sine git svn clone does svn branches and separate remotes by default and I didn't want to do

[ZODB-Dev] "Unable to acquire commit lock" from RelStorage

2012-03-04 Thread Chris Withers
Hi Shane, What does this exception mean: Traceback (innermost last): Module ZPublisher.Publish, line 135, in publish Module Zope2.App.startup, line 291, in commit Module transaction._manager, line 89, in commit Module transaction._transaction, line 329, in commit Module transaction._tr

Re: [ZODB-Dev] where does 32it vs 64bit matter?

2012-01-03 Thread Chris Withers
Hi Alan, On 21/12/2011 20:12, Alan Runyan wrote: Are there any migration steps or minimum ZODB/RelStorage versions needed to make this step? We run 32bit windows clients against 64bit linux zeo server; works fine. Good to know, but my concern is taking a system which is currently ru

[ZODB-Dev] where does 32it vs 64bit matter?

2011-12-21 Thread Chris Withers
Hi All, I have a set of ZODBs, they're currently served from a 64-bit mysql cluster running relstorage. At the moment, all the clients are 32-bit. At some point, those may be changing to be 64-bit. What are the implications of this? Do all relstorage clients need to be the same bit width? W

Re: [ZODB-Dev] RelStorage breaks History tab

2011-09-02 Thread Chris Withers
On 02/09/2011 08:26, Martijn Pieters wrote: > On Thu, Sep 1, 2011 at 22:56, Chris Withers wrote: >> I see the resulting transactions in both the root Undo tab and the Undo >> tab of the page template, but not in the History tab of the page template. > > Without looking, I&

[ZODB-Dev] RelStorage breaks History tab

2011-09-01 Thread Chris Withers
Hi Shane, It looks like RelStorage (1.4.2 in my case, connected to MySQL, ZODB 3.9.6 on Zope 2.12.7) breaks the history tab. To reproduce, I set up a stock ZODB with a history-preserving storage, went in and edited the index_html page a few times. I see the resulting transactions in both the r

Re: [ZODB-Dev] RelStorage and MySQL 5.5

2011-06-22 Thread Chris Withers
On 22/06/2011 22:25, Shane Hathaway wrote: > I think RelStorage is ready to become more of a community project, and > part of that is changing the meaning of "supported". Until now, it has > meant "Shane is confident that this configuration will not lose your > data, because he has tested it repea

Re: [ZODB-Dev] weird "MySQL gone away error" when viewing history tab on Script(Python) with RelStorage 1.4.2

2011-05-03 Thread Chris Withers
On 03/05/2011 09:02, Shane Hathaway wrote: >>> Maybe the query is killing MySQL, for whatever reason. >> >> I don't think so, since I'd then expect all the other storages served by >> that cluster to complain similarly. > > Not necessarily. RelStorage reconnects automatically. ...but logs, and I h

Re: [ZODB-Dev] weird "MySQL gone away error" when viewing history tab on Script(Python) with RelStorage 1.4.2

2011-05-03 Thread Chris Withers
On 27/04/2011 18:11, Shane Hathaway wrote: >> OperationalError: (2006, 'MySQL server has gone away') >> >> This is happening across at least two separate instances with separate >> storages. >> >> Any ideas? > > Maybe the query is killing MySQL, for whatever reason. I don't think so, since I'd the

Re: [ZODB-Dev] weird "MySQL gone away error" when viewing history tab on Script(Python) with RelStorage 1.4.2

2011-04-27 Thread Chris Withers
On 27/04/2011 16:47, David Glick wrote: >> OperationalError: (2006, 'MySQL server has gone away') >> >> This is happening across at least two separate instances with separate >> storages. > Not sure if this is the same, but I've gotten this error with > python-mysql in a case where the mysql connec

[ZODB-Dev] weird "MySQL gone away error" when viewing history tab on Script(Python) with RelStorage 1.4.2

2011-04-27 Thread Chris Withers
Hi Shane, Attempting to view the /manage_change_history_page of a history-keeping relstorage is giving me: script statement failed: '\nSELECT 1 FROM current_object WHERE zoid = %(oid)s\n'; parameters: {'oid': 1163686} ...and then: Traceback (innermost last): Module ZPublish

Re: [ZODB-Dev] transaction as context manager, exception during commit

2011-02-24 Thread Chris Withers
On 24/02/2011 11:38, Laurence Rowe wrote: >> Hi Jim, >> >> The current __exit__ for transaction managers looks like this: >> >> def __exit__(self, t, v, tb): >> if v is None: >> self.commit() >> else: >> self.abort() >> >> ..which means that if you'r

[ZODB-Dev] transaction as context manager, exception during commit

2011-02-24 Thread Chris Withers
Hi Jim, The current __exit__ for transaction managers looks like this: def __exit__(self, t, v, tb): if v is None: self.commit() else: self.abort() ..which means that if you're using the transaction package as a context manager and, say, a relati

Re: [ZODB-Dev] RelStorage and PosKey errors - is this a risky hotfix?

2011-02-02 Thread Chris Withers
On 02/02/2011 09:14, Shane Hathaway wrote: > On 02/02/2011 10:57 AM, Chris Withers wrote: >> Er, since when? If that were the case, I'm sure Shane would place >> explicit instructions that it should not be used... > > Safe is relative. MySQL is a good choice for Faceboo

Re: [ZODB-Dev] RelStorage and PosKey errors - is this a risky hotfix?

2011-02-02 Thread Chris Withers
On 01/02/2011 23:10, Ruda Porto Filgueiras wrote: > I had similar issues on past and now I follow some rules: > > - pack only with zodbpack > - pack-gc = false ...which means your database will still slowly (or quickly in my case!) grow over time with unreferenced objects. > - pack when th

Re: [ZODB-Dev] RelStorage and MySQL wait-timeout

2011-02-02 Thread Chris Withers
On 01/02/2011 18:11, Shane Hathaway wrote: > On 02/01/2011 07:51 PM, Chris Withers wrote: >> I can understand the problem being fairly terminal if there was a >> disconnect *during* a timeout, and I'd expect an exception, but not a >> segfault ;-) > > I haven&

Re: [ZODB-Dev] RelStorage and MySQL wait-timeout

2011-02-01 Thread Chris Withers
Hi Shane, On 01/02/2011 17:45, Shane Hathaway wrote: > On 02/01/2011 10:01 AM, Chris Withers wrote: >> OperationalError: (2006, 'MySQL server has gone away') >> >> ...which feels a little on the serious side for (what is for MySQL) >> quite a normal situation

Re: [ZODB-Dev] RelStorage - what tables and indexes should be present?

2011-02-01 Thread Chris Withers
On 01/02/2011 17:33, Shane Hathaway wrote: >> What's special about this storage? > > It sounds like RelStorage didn't get a chance to finish creating the > schema. In MySQL, DDL statements are not transactional, so errors during > schema creation (such as a timeout) leave a partial schema. Sounds

Re: [ZODB-Dev] RelStorage - what tables and indexes should be present?

2011-02-01 Thread Chris Withers
On 01/02/2011 14:42, Stephan Richter wrote: > On Tuesday, February 01, 2011, Chris Withers wrote: >> I also note that while the ones with lots of tables have this schema for >> object_state: >> >> CREATE TABLE `object_state` ( >> `zoid` bigint(20) NOT NULL,

Re: [ZODB-Dev] Error from MySQL when attempting to run zodbconvert

2011-02-01 Thread Chris Withers
On 01/02/2011 04:03, Shane Hathaway wrote: > On 01/31/2011 06:30 PM, Chris Withers wrote: >> Hi Shane, >> >> I got the following error when trying to run zodbconvert against a >> clustered MySQL environment running RHCS: >> >> File >> "/var/bui

[ZODB-Dev] (anecdotal) Restarting MySQL under RelStorage gives segfault

2011-02-01 Thread Chris Withers
Hi All, I've had a couple of instances now where RelStorage-backed ZODB client processes have crashed/segfaulted. As best I can tell, both times have been the result of the MySQL server being restarted while the client is connected. However, other server restarts haven't caused this problem, s

[ZODB-Dev] RelStorage - what tables and indexes should be present?

2011-02-01 Thread Chris Withers
Hi Shane, I notice that one of my history-free storages only has 'new_oid' and 'object_state' tables while all the others have 'new_oid', 'object_ref', 'object_refs_added', 'object_state' and 'pack_object' tables. What's special about this storage? I also note that while the ones with lots of

[ZODB-Dev] RelStorage and MySQL wait-timeout

2011-02-01 Thread Chris Withers
On 01/02/2011 04:11, Shane Hathaway wrote: >> My guess is that the zap_all took so long that the server had gone away >> by the time the sql statement had be executed. > > My guess is MySQL is configured to drop connections when they are idle. Indeed, Rackspace had configured a wait-timeout of 60

[ZODB-Dev] bug with RelStorage's zodbconvert --clear when clearing a large DB with MySQL

2011-01-31 Thread Chris Withers
Hi Shane, This one's less serious if it is what I think it is: Traceback (most recent call last): File "bin/zodbconvert", line 24, in relstorage.zodbconvert.main() File "/var/buildout-eggs/RelStorage-1.4.0-py2.6.egg/relstorage/zodbconvert.py", line 89, in main destination.zap_a

Re: [ZODB-Dev] Error from MySQL when attempting to run zodbconvert

2011-01-31 Thread Chris Withers
On 31/01/2011 16:30, Chris Withers wrote: > I got the following error when trying to run zodbconvert against a > clustered MySQL environment running RHCS: > > File > "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py", > lin

[ZODB-Dev] Error from MySQL when attempting to run zodbconvert

2011-01-31 Thread Chris Withers
Hi Shane, I got the following error when trying to run zodbconvert against a clustered MySQL environment running RHCS: File "/var/buildout-eggs/MySQL_python-1.2.3-py2.6-linux-i686.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler raise errorclass, errorvalue _mysql_exceptio

Re: [ZODB-Dev] RelStorage, history-free, pack causes POSKeyError with BTreeFolder2

2011-01-31 Thread Chris Withers
On 28/01/2011 21:58, Jürgen Herrmann wrote: > Afaics you use zodbpack's default of "days=0". This is known to produce > POSKeyErrors if the database is written to while packing. Where is this documented? Does this apply to history-keeping or history-free storages? > Try with > something >

Re: [ZODB-Dev] RelStorage, history-free, pack causes POSKeyError with BTreeFolder2

2011-01-28 Thread Chris Withers
On 28 Jan 2011, at 21:58, Jürgen Herrmann wrote: > On Fri, 28 Jan 2011 12:34:35 +0000, Chris Withers wrote: >> Bingo: >> >> svn co https://secure.simplistix.co.uk/svn/relstorage_pack -r 4595 >> cd relstorage_pack >> python2.6 bootstrap.py >> b

Re: [ZODB-Dev] RelStorage, history-free, pack causes POSKeyError with BTreeFolder2

2011-01-28 Thread Chris Withers
Bingo: svn co https://secure.simplistix.co.uk/svn/relstorage_pack -r 4595 cd relstorage_pack python2.6 bootstrap.py bin/buildout Then in one terminal: bin/generate --zap --folder_depth 3 Leave it running for a minute or two, and then in another do: bin/zodbpack pack.conf After a bit I get: T

Re: [ZODB-Dev] RelStorage, history-free, MySQL - errors

2011-01-28 Thread Chris Withers
Hi Shane, Okay, so here's the test harness: svn co https://secure.simplistix.co.uk/svn/relstorage_pack -r 4594 It uses the normal RelStorage test users and databases. After running the buildout, I do each of the following in their own terminal: bin/generate --zap bin/zodbpack pack.conf On 2

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-28 Thread Chris Withers
On 27/01/2011 11:24, Shane Hathaway wrote: > On 01/27/2011 03:57 AM, Chris Withers wrote: >> It would also be *really* handy if zodbpack could run off a normal >> zope.conf for both logging and storage config (although, in my case, I'd >> then need to be able to specif

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-27 Thread Chris Withers
On 27/01/2011 10:47, Shane Hathaway wrote: - e2fsck -f >> >> Hmm, how do I e2fsck a mounted filesystem? > > You don't. Don't even try. :-) Yeah, I got that from the warning message it squealed when I tried ;-) I was more curious about how you'd do this if you needed to (how do you unmount /

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-27 Thread Chris Withers
On 27/01/2011 10:40, Jürgen Herrmann wrote: > > i also had to up max_allowed_packet to 32M to make the tests work. Indeed, I upped to 32M and now I get no failures: Total: 420 tests, 0 failures, 0 errors in 59.173 seconds. cheers, Chris -- Simplistix - Content Management, Batch Processing & P

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-27 Thread Chris Withers
On 27/01/2011 03:15, Shane Hathaway wrote: >> Okay, so I'll do: >> >> - e2fsck -f Hmm, how do I e2fsck a mounted filesystem? The MySQL filesystem could be unmounted (after I shut down MySQL!), so I ran e2fsck on it: e2fsck -f /dev/sdb1 e2fsck 1.41.3 (12-Oct-2008) Pass 1: Checking inodes, blocks

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-26 Thread Chris Withers
On 26/01/2011 22:49, Laurence Rowe wrote: > On 26 January 2011 21:57, Jürgen Herrmann wrote: >> is there a script or some example code to search for cross db >> references? >> i'm also eager to find out... for now i disabled my packing cronjobs. > > Packing with garbage collection disabled (

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-26 Thread Chris Withers
On 26/01/2011 21:54, Chris Withers wrote: > On 26/01/2011 21:05, Shane Hathaway wrote: >> On 01/26/2011 11:52 AM, Chris Withers wrote: >>> On 26/01/2011 14:08, Shane Hathaway wrote: >>>> I've checked in a fix in Subversion. Please try it out. I need to look >&

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-26 Thread Chris Withers
On 26/01/2011 21:57, Jürgen Herrmann wrote: > is there a script or some example code to search for cross db > references? > i'm also eager to find out... for now i disabled my packing cronjobs. Googling for "cross database references zodb" should give you all the necessary background. I do

Re: [ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-26 Thread Chris Withers
On 26/01/2011 21:05, Shane Hathaway wrote: > On 01/26/2011 11:52 AM, Chris Withers wrote: >> On 26/01/2011 14:08, Shane Hathaway wrote: >>> I've checked in a fix in Subversion. Please try it out. I need to look >>> at the possible pack issue recently reported befor

[ZODB-Dev] RelStorage pack with history-free storage results in POSKeyErrors

2011-01-26 Thread Chris Withers
On 26/01/2011 14:08, Shane Hathaway wrote: > I've checked in a fix in Subversion. Please try it out. I need to look > at the possible pack issue recently reported before we make a release. Where is this pack issue documented/discussed? Also, does RelStorage have a bug tracker anywhere? I have a

Re: [ZODB-Dev] RelStorage recommended maintenance

2011-01-21 Thread Chris Withers
On 21/01/2011 22:09, Laurence Rowe wrote: > On 21 January 2011 20:57, Shane Hathaway wrote: >> On 01/21/2011 10:46 AM, Chris Withers wrote: >>> I'm wondering what the recommended maintenance for these two types of >>> storage are that I use: >>> >&g

[ZODB-Dev] RelStorage recommended maintenance

2011-01-21 Thread Chris Withers
Hi Shane, I'm wondering what the recommended maintenance for these two types of storage are that I use: - keep-history=true, never want to lose any revisions My guess is zodbpack with pack-gc as true, but what do I specify for the number of days in order to keep all history? - keep-history=fa

Re: [ZODB-Dev] RelStorage: Write on master, read from slave(s)?

2010-11-29 Thread Chris Withers
On 26/10/2010 11:27, Shane Hathaway wrote: > On 10/26/2010 04:04 AM, Anton Stonor wrote: >> In order to scale an application using RelStorage I was thinking about >> seperating reads and writes accross databases. Writes would go to a >> Mysql master and the app would read from one or more Mysql sla

Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL

2010-11-18 Thread Chris Withers
On 18/11/2010 16:48, Hanno Schlichting wrote: > On Thu, Nov 18, 2010 at 5:19 PM, Leonardo Santagada > wrote: >> On Thu, Nov 18, 2010 at 1:47 PM, Chris Withers >> wrote: >>> On 18/11/2010 15:39, Marius Gedminas wrote: >>>> About the only noticeable diff

Re: [ZODB-Dev] 32-bit vs 64-bit - RelStorage on MySQL

2010-11-18 Thread Chris Withers
On 18/11/2010 15:39, Marius Gedminas wrote: > About the only noticeable difference -- other than the obvious memory > growth What obvious memory growth? I guess I'm more concerned with a mix of 32 and 64 bit in a RelStorage setup... Does that mean my question is essentially "do 32-bit MySQL cli

[ZODB-Dev] 32-bit vs 64-bit

2010-11-18 Thread Chris Withers
Hi All, Are there any intricacies with migrating a ZODB-backed Zope 2 app from 32-bit to 64-bit machines? The specific scenario I have in mind is moving a 32-bit set of ZODBs into MySQL-backed RelStorages where the MySQL server runs 64-bit but the clients are still 32-bit, although in one inst

Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Chris Withers
On 04/11/2010 15:09, Jim Fulton wrote: >> Can you raise that, or would you mind if I raised that to CRITICAL? >> (it does feel like a CRITICAL kind of thing, based on the information you've >> provided today) > > +1 Feel free to do it yourself, or file a bug. https://bugs.launchpad.net/zodb/+bug/

Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Chris Withers
On 04/11/2010 14:50, Jim Fulton wrote: > Actually, I think it should be used, at least for applications with > high availability requirements. For applications like our's (ZC's) > it amounts to a "don't be stupid" setting. > > For some applications, it's better to risk inconsistency than to have >

Re: [ZODB-Dev] RelStorage: getting keep-history wrong results in silent death

2010-11-04 Thread Chris Withers
On 04/11/2010 14:13, Shane Hathaway wrote: >> Turns out I had keep-history set to true whereas the database have been >> created with it set to false. >> >> Should this not result in an exception being logged rather than just a >> silent death that causes zdaemon to lamely attempt to restart the in

Re: [ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Chris Withers
On 04/11/2010 13:43, Jim Fulton wrote: >> I'm curious: >> >> - Why is the timeout not logged at ERROR or even CRITICAL? > > Because it was considered to be neither. It's also not a server error. It is an erroneous condition; a transaction has taken long enough that a timeout has occurred. Thinki

[ZODB-Dev] RelStorage versus temp_folder

2010-11-04 Thread Chris Withers
Hi Shane, I have the following config for the temp_folders of my ZODB instances: host dr2.private user zope .. keep-history false mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer However, eve

[ZODB-Dev] RelStorage: getting keep-history wrong results in silent death

2010-11-04 Thread Chris Withers
Hi Shane, I was having trouble starting a relstorage 1.4.0-backed Zope instance this morning. The symptoms were that it would get going fine, until the point it tried to open the storages, at which point it would simply restart with no logging as to why. Turns out I had keep-history set to tru

[ZODB-Dev] transaction timeouts raise no exceptions and cause strange errors on clients

2010-11-04 Thread Chris Withers
Hi All, I have a ZODB 3.9.6 ZEO storage server configured with a transaction time of 20 seconds. On one zeo client I saw the following: -- 2010-11-04T10:40:50 INFO ZEO.ClientStorage unpackedstorage Disconnected from storage: "('xx', 6000)" -- 2010-11-04T10:40:50 INFO ZEO.ClientStorage

Re: [ZODB-Dev] Default comparison considered harmful in BTrees.

2010-10-26 Thread Chris Withers
On 25/10/2010 23:34, Marius Gedminas wrote: > Or perhaps make it emit DeprecationWarnings, but continue working. Then > make it a fatal error in the next minor/major release. Well, not a DeprecationWarning... Is there a DataLossWarning? Still, +1 on the warning followed by exception in 2 release

Re: [ZODB-Dev] "MySQL server has gone away" with RelStorage

2010-10-25 Thread Chris Withers
On 25/10/2010 15:57, Paul Winkler wrote: > On Sun, Oct 24, 2010 at 05:55:05PM +0100, Chris Withers wrote: >> Restarting Zope appeared to fixed the problem, but I'm slightly confused >> as I can't find any entries in the logs of the Apache instance that sits >> in

Re: [ZODB-Dev] "MySQL server has gone away" with RelStorage

2010-10-24 Thread Chris Withers
On 23/10/2010 18:48, Shane Hathaway wrote: > On 10/23/2010 01:29 AM, Chris Withers wrote: >> Hi Shane, >> >> With the server I'm testing on, when coming back to test after a couple >> of days working on something else, I get the following logged at Warning: >

[ZODB-Dev] "MySQL server has gone away" with RelStorage

2010-10-23 Thread Chris Withers
Hi Shane, With the server I'm testing on, when coming back to test after a couple of days working on something else, I get the following logged at Warning: Reconnecting load_conn: (2006, 'MySQL server has gone away') Does this mean the reconnection is being handled by RelStorage or that I need

Re: [ZODB-Dev] ZODB replication and high-availability across data centers

2010-10-21 Thread Chris Withers
On 21/10/2010 16:44, Andreas Jung wrote: > The well-known options are: > > Relstorage on top of some RDBMS-related clustering/replication mechanism This is where I'd look, since the RDBMS guys likely have patterns that match the one you're seeking... > ZEORaid Not yet! (and, performance wise, I

Re: [ZODB-Dev] Status of gocept.zeoraid

2010-10-20 Thread Chris Withers
On 20/10/2010 17:03, Sylvain Viollon wrote: >One of my customers would like to use gocept.zeoraid in production. >So him, and I have some questions: > >- Is it stable enough to be used in production ? No. >- Does it bring speed improvements over regular ZEO setup (I don't > t

Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

2010-10-14 Thread Chris Withers
On 14/10/2010 17:13, Alan Runyan wrote: >>> I went for the following in the end: >>> >>> ... >>> cache-size 200Mb >>> client app >>> wait on >>> >>> mount-point / >>> cache-size 5 >>> cache-size-bytes 500Mb >>> >>> Any obvious issues with that?

Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

2010-10-14 Thread Chris Withers
On 14/10/2010 17:09, Jim Fulton wrote: >> What does this error message imply? > > That two processes were trying to access the cache. > >> The above did not, however, occur if I stopped and then started the app >> server... >> >> Race condition? > > Hopefully, you screwed up somehow and started the

[ZODB-Dev] One more RelStorage question

2010-10-14 Thread Chris Withers
Hi Shane, I forgot to ask... Is RelStorage suitable for a temporary storage? If so, what would be the recommended options for the following storage: # Temporary storage database (for sessions) name temporary storage for sessioning mount-point /temp_folder c

[ZODB-Dev] RelStorage Questions

2010-10-14 Thread Chris Withers
Hi Shane, For all of these, if there are docs, please point me in their direction. I've been working from: http://pypi.python.org/pypi/RelStorage In my case, I'm looking at ZODB 3.9.6, Zope 2.12.7 onto MySQL 5.1 provided as a two machine cluster maintained by a hosting company. This sound goo

Re: [ZODB-Dev] zodb monitor port / tailing a .fs

2010-10-14 Thread Chris Withers
On 14/10/2010 02:40, Darryl Dixon - Winterhouse Consulting wrote: >> ...and fsdump appears to have no way of saying "show me the last few >> transactions in this 35GB file" :-( > > Yeah it's pretty verbose. I usually just pump the output to a text file ...yeah, but a 35GB zodb is going to result i

Re: [ZODB-Dev] transaction commits don't have to write data?

2010-10-13 Thread Chris Withers
On 14/10/2010 01:21, Chris Withers wrote: > On 13/10/2010 15:23, Jim Fulton wrote: >> You can connect to the monitor port in 3.9 and earlier, >> if the monitor port is configured. In 3.10, the monitor server is >> replaced by a ZEO client method, server_status. This tells

Re: [ZODB-Dev] zodb monitor port / tailing a .fs

2010-10-13 Thread Chris Withers
On 14/10/2010 01:30, Laurence Rowe wrote: > There's also fstail: > > $ bin/zopepy -m ZODB.scripts.fstail var/filestorage/Data.fs That looks more like it.. Any ideas what options there are to it? From reading the source, I see a -n, but no idea what it should do. I'm looking for a "more detail"

Re: [ZODB-Dev] zodb monitor port / tailing a .fs

2010-10-13 Thread Chris Withers
On 14/10/2010 01:28, Darryl Dixon - Winterhouse Consulting wrote: >> Do we have any kind of "tail -f /some/filestorage.fs" yet? (or have we >> always had such a tool) to see what the last few transactions in the >> underlying file storage look like in a human-readable form? > > fsdump.py gets you p

Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

2010-10-13 Thread Chris Withers
On 12/10/2010 17:56, Jim Fulton wrote: > Currently the only configuration "docs" are in the .xml Alan provided a link > to. > > The best size will depend on the app. Is there a downside to "too big", provided there's plenty of disk space free on the app servers? > There are some docs in > http:

Re: [ZODB-Dev] zodb monitor port / tailing a .fs

2010-10-13 Thread Chris Withers
On 13/10/2010 15:23, Jim Fulton wrote: > You can connect to the monitor port in 3.9 and earlier, > if the monitor port is configured. In 3.10, the monitor server is > replaced by a ZEO client method, server_status. This tells you > much the same information that's in the log messages. Okay, monit

Re: [ZODB-Dev] blocked transactions?

2010-10-13 Thread Chris Withers
On 13/10/2010 15:59, Jim Fulton wrote: >> So there's no logging I can turn on to see how long commits are taking, >> unless I can get more of those 'transaction blocked' messages to show up? > > You can set the log level to debug. This will log the entire ZEO > protocol. Analysis will take require

Re: [ZODB-Dev] blocked transactions?

2010-10-13 Thread Chris Withers
On 13/10/2010 15:40, Jim Fulton wrote: >> Wouldn't I see storage errors in that case? > > No. Unless you set a transaction timeout, the storage will > wait for the finish indefinitely. ...I meant on the client side. There were no errors other than the protocol timeout ones below: >> File >> "/

Re: [ZODB-Dev] blocked transactions?

2010-10-13 Thread Chris Withers
On 13/10/2010 15:23, Jim Fulton wrote: >> Their occurences coincided with an app server cluster of 4 clients locking >> up completely... > > Did the numbers get much bigger than 3? Nope. >> I would dearly love to know why that happened. > > Typically, it's due to a client that votes and fails to

Re: [ZODB-Dev] blocked transactions?

2010-10-13 Thread Chris Withers
On 13/10/2010 14:06, Jim Fulton wrote: > On Wed, Oct 13, 2010 at 4:02 AM, Chris Withers wrote: >> Hi All, >> >> What do the following error message mean: >> >> -- >> 2010-10-12T17:05:58 INFO ZEO.StorageServer (24471/192.168.100.100:54407) >> Tra

[ZODB-Dev] RelStorage and ZODB 3.10?

2010-10-13 Thread Chris Withers
Hi Shane, Does RelStorage 1.4 work with ZODB 3.10? (In particular, the Undo changes and the tightened storage API..) I see the mention of 3.10.0b7, but just wanted to check that means you're okay with 3.10 final ;-) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python

[ZODB-Dev] blocked transactions?

2010-10-13 Thread Chris Withers
Hi All, What do the following error message mean: -- 2010-10-12T17:05:58 INFO ZEO.StorageServer (24471/192.168.100.100:54407) Transaction blocked waiting for storage. Clients waiting: 3. -- 2010-10-12T17:07:16 INFO ZEO.StorageServer (24471/192.168.100.105:50112) Blocked transaction res

Re: [ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

2010-10-12 Thread Chris Withers
On 12/10/2010 17:11, Jim Fulton wrote: >> Now, I have the impression that these were unreliable and buggy and had >> been so for some time. > > They became much better in later 3.8 releases and still much better lately. > > We've been using them in production for quite a while. Any docs on recomme

[ZODB-Dev] persistent client caches okay to use again in ZODB 3.9.6?

2010-10-12 Thread Chris Withers
Hi All, I'm wondering if some of my issues might be alleviated by persistent client caches. Now, I have the impression that these were unreliable and buggy and had been so for some time. Is this still the case? If not, where are the docs on how to configure them nowadays? cheers, Chris --

[ZODB-Dev] zodb 3.9.6 performance tuning?

2010-10-12 Thread Chris Withers
Hi, After recently moving a medium-sized customer project from Zope 2.9.8 to Zope 2.12.7 sitting on ZODB 3.9.6 by way of zeoraid b7, I'm now seeing some fairly weird and bad behaviour that we didn't have before. So, everything was fine after the migration. However, today I turned on a batch jo

Re: [ZODB-Dev] semantics of _v_ attributes?

2010-10-06 Thread Chris Withers
On 06/10/2010 15:19, Jim Fulton wrote: > _v_ attributes should be dropped whenever an object's state is > cleared. It would be great if there was some clear documentation covering all this in the ZODB package... > So, you want to update an object using old state. > > I would use: > >base._p_

[ZODB-Dev] semantics of _v_ attributes?

2010-10-06 Thread Chris Withers
Hi All, I'm trying to fix this bug: https://bugs.launchpad.net/zope2/+bug/649605 ...and I'm struggling to find documentation on the semantics of _v_ attributes. In particular, surely this code: base._p_activate() # make sure we're not a ghost base.__setstate__(s

Re: [ZODB-Dev] ZODB history problem with Zope 2.12.7

2010-09-28 Thread Chris Withers
On 28/09/2010 12:55, Tres Seaver wrote: > On 09/28/2010 03:36 AM, Chris Withers wrote: >> On 28/09/2010 00:12, Marius Gedminas wrote: >>> --- ./OFS/History.py.orig 2010-09-28 02:11:56.535745440 +0300 >>> +++ ./OFS/History.py2010-09-28 02:12:00.043764683

Re: [ZODB-Dev] "Versions aren't supported" message in Plone 4 upgrade

2010-09-20 Thread Chris Withers
On 09/09/2010 23:31, David Glick wrote: > I meant that we exported the database under Plone 3.x and reimported to > a new database in the same Zope instance (still Plone 3.x with the same > versions of all software). *Then* proceeded with a normal upgrade to > 4.0 using the normal method and the n

Re: [ZODB-Dev] Stripping version data from databases

2010-09-10 Thread Chris Withers
On 09/09/2010 15:10, Jim Fulton wrote: > I suspect we'll see more cases where people have databases that have > versioned records > that they don't know how to get rid of. (We had a case like this > internally, but we stopped > caring about the database in question before we had to deal with it. :

Re: [ZODB-Dev] attempting to undo an un-undoable transaction gives "NotImplementedError" with ZODB 3.9.5

2010-09-09 Thread Chris Withers
On 09/09/2010 21:47, Jim Fulton wrote: >> 2010-08-26 22:49:01 ERROR txn.-1338986496 Failed to abort resource >> manager:> object at 0x2d509f0> at 47516176> >> Traceback (most recent call last): >>File "transaction-1.1.0-py2.6.egg/transaction/_transaction.py", line >> 475, in abort >> rm.a

[ZODB-Dev] attempting to undo an un-undoable transaction gives "NotImplementedError" with ZODB 3.9.5

2010-08-26 Thread Chris Withers
Here's an example: 2010-08-26 22:49:01 ERROR txn.-1338986496 Failed to abort resource manager: at 47516176> Traceback (most recent call last): File "transaction-1.1.0-py2.6.egg/transaction/_transaction.py", line 475, in abort rm.abort(self) File "transaction-1.1.0-py2.6.egg/transacti

Re: [ZODB-Dev] [Zope-dev] Undo tab in Zope 2.12

2010-08-26 Thread Chris Withers
Chris Withers wrote: > ...often appears to be empty when it shouldn't be. > > Has anyone else observed this? The answer turned out to be ZODB 2.10b4 being used on the storage server side. I found a clue in the ChangeLog but it sure would be nice if there was some logging of som

[ZODB-Dev] tracking back to what code caused a conflicting object change

2010-07-28 Thread Chris Withers
te: Tue, 27 Jul 2010 11:45:26 +0100 From: Chris Withers To: zope list user Hi All, I have an old Zope 2.9.8 instance which has recently started throwing the following unresolved ConflictErrors: Traceback (innermost last): Module Zope2.App.startup, line 173, in zpublisher_exception

Re: [ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?

2010-06-18 Thread Chris Withers
Christian Theune wrote: > and a > mysterious bug WRT oid generation This one's new to me, where can I find out more? >>> >>> The one you reported quite a while ago that ended in mixed-up OID >>> results. >> >> Have you got a url to the tracker issue? > > I think it was this one: > htt

Re: [ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?

2010-06-12 Thread Chris Withers
Christian Theune wrote: > On 06/12/2010 09:47 AM, Chris Withers wrote: >> Christian Theune wrote: >>> On 06/11/2010 05:50 PM, Alan Runyan wrote: >>>>>> I suppose you could look at zeoraid. >>>>> Hadn't thought of that, will look at it, tha

Re: [ZODB-Dev] ZEORaid was Re: Restoring from repozo and reusing an index file?

2010-06-12 Thread Chris Withers
Christian Theune wrote: > On 06/11/2010 05:50 PM, Alan Runyan wrote: I suppose you could look at zeoraid. >>> Hadn't thought of that, will look at it, thanks! >> Anyone using ZEORaid in production? > > I hope not. There's still open issues with split brain situations IIUC, that can be mitig

Re: [ZODB-Dev] Restoring from repozo and reusing an index file?

2010-06-11 Thread Chris Withers
Paul Winkler wrote: > 3 should be pretty fast as its an incremental backup. > 4 may take a little while but iirc it's not too bad for this db - > maybe 10 minutes. > Most of the downtime is caused by rebuilding the index which takes way > longer (30 min?) > > So I had two questions: > > 1) wo

Re: [ZODB-Dev] The database root: It's a trap!

2010-05-12 Thread Chris Withers
Jim Fulton wrote: >> C. I'd really like to be able to configure what the root object is as >> database creation time. That way it can be application specific. > > That sounds reasonable, but doesn't address the default behavior. Default would be an OOBTree for me... Scales, no thinking or scary

Re: [ZODB-Dev] The database root: It's a trap!

2010-05-12 Thread Chris Withers
Jim Fulton wrote: > A. Change ZODB so that new databases have BTrees as root objects. > >This has the advantage that BTrees are scalable. It has a number >of down sides: > >- BTrees should only be used when keys are known to have a stable > ordering. If we had a scalable hash d

Re: [ZODB-Dev] Problem with handling of data managers that join transactions after savepoints

2010-05-11 Thread Chris Withers
Jim Fulton wrote: >>> I plan to implement A soon if there are no objections. >>> >>> Unless someone somehow convinced me to do D, I'll also add an >>> assertion in the Transaction.join method to raise an error if a >>> data manager joins more than once. >> Option A sounds sensible. It also means I

Re: [ZODB-Dev] RFC: compatibility of ZEO clients and servers running different Python versions

2010-04-20 Thread Chris Withers
Jim Fulton wrote: > Because of the complexity of the solution, I'm, sadly, inclined to > leave things as they are: > > - Can't use Python 2.4 clients with Python 2.5 or 2.6 servers > - Can use Python 2.6 clients with 2.4 servers. > - Can't use Python 2.5 or 2.6 servers with clients running ZODB 3.

  1   2   3   4   5   >