Re: [ZODB-Dev] Navigating inherently hierarchical data

2007-08-04 Thread Chris Withers
Tommy Li wrote: I don't understand how I can use ZODB to do this, however. From what I can gather from reading the manual, if I simply stored variables referring to the parent and children in every TreeNode, I'd end up storing the whole tree of TreeNodes's every time I wanted to store that

Re: [ZODB-Dev] Re: Navigating inherently hierarchical data

2007-08-06 Thread Chris Withers
Tommy Li wrote: Great. So when I store the parent node of a tree structure into zodb, all the children get recursively stored? Yes. Or do I need to manually store each of these Persistent subobjects? No. But having your node class inherit from Persistent will mean that altering a node

[ZODB-Dev] zodb tools for zodb as of zope 2.9.6?

2008-02-04 Thread Chris Withers
Hi All, Where can I find the zodb tools (fsrefs.py, fstest.py, etc) as appropriate for the zodb that ships with Zope 2.9.6? iirc, 2.9.6 was one of the releases that didn't ship with ZODBTools due to a bug, so I'm looking for places to hunt! cheers, Chris -- Simplistix - Content

[ZODB-Dev] data loss bug with mounted storages, cut'n'paste and pack

2008-02-12 Thread Chris Withers
Hi All, I believe I've bottomed out the bug at the centre of my POSKeyError report this morning and it's a real nasty. It's pretty convoluted but I'm fairly sure it's a bug. Imagine the following zope.conf setup: zodb_db main filestorage path $INSTANCE/var/UnPacked.fs

Re: [ZODB-Dev] corrupt filestorages - more info, still eep :-(

2008-02-12 Thread Chris Withers
Chris Withers wrote: I'm dealing with what I suspect are a couple of corrupt filestorages. One is packed every time content is loaded by a batch process and the other is never packed. The UnPacked storage is mounted as the root and then the Packed one is mounted in to /content. I forgot

[ZODB-Dev] corrupt filestorages - eep :-(

2008-02-12 Thread Chris Withers
Hi All, I'm dealing with what I suspect are a couple of corrupt filestorages. One is packed every time content is loaded by a batch process and the other is never packed. The UnPacked storage is mounted as the root and then the Packed one is mounted in to /content. Running fstest.py on both

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: It seems like all of these potential advantages are available through the use of multiple storages, but none of them really require database-level support for cross-storage references.On the other hand, it seems clear that cross-storage references make the system as a

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: Uh. I thought we were talking about reported POSKeyErrors that would not exist if there were no such thing as cross-storage references. I don't understand what there is to disagree with here. Are you saying that cross-storage references reduce the risk of other

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Shane Hathaway wrote: Chris Withers wrote: I did try this: data._p_jar = app._p_jar data._p_oid = oid app.x = data import transaction transaction.get().note('Fix POSKeyError') transaction.commit() ...but it didn't work. I don't know if the differences are significant. app.x

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Jim Fulton wrote: There's also a problem that we made cross-database references so easy to make that they are made accidentally. There's also no way to say actually, I want to *move* this object to this other storage. Semantically, that's what my customer was trying to do... make them

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Jim Fulton wrote: If someone can make something like this work without modifying ZODB, I won't object. The only thing I'd like is to be able to: - specify in policy that cross storage references should result in the target object being copied to the storage or - have an api to say where

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-14 Thread Chris Withers
Binger David wrote: On Feb 13, 2008, at 3:58 PM, Jim Fulton wrote: Note that, IMO, some of the best use cases for multi databases are separating catalog and session data from regular content. Could you say more about what the benefit of this separation is, and why cross-storage references

Re: [ZODB-Dev] Fixing POSKeyErrors :-)

2008-02-18 Thread Chris Withers
Christian Theune wrote: Okay, so I count two issues: - packing and multiple mounted storages - POSKeyErrors resulting in failure to load referring object rather than creation of a broken referred object Where would you like me to file these bug reports? How about the bugtracker? What's

Re: [ZODB-Dev] ZEO Newbie Question

2008-02-19 Thread Chris Withers
Kenneth Miller wrote: client to connect and I'm able to see the data, but once the second client is running, it can't see any more changes made by the first client. What version of zodb are you doing? Are you using a clients that have asyncore loops running? cheers, Chris -- Simplistix -

Re: [ZODB-Dev] ERROR ZODB.Connection Couldn't load state for 0x01

2008-03-10 Thread Chris Withers
Dylan Jay wrote: I have a few databases being served out of a zeo. I restarted them in a routine operation and now I can't restart due to the following error Do you use cross database references? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting -

Re: [ZODB-Dev] Re: ERROR ZODB.Connection Couldn't load state for 0x01

2008-03-11 Thread Chris Withers
Dylan Jay wrote: Chris Withers wrote: Dylan Jay wrote: I have a few databases being served out of a zeo. I restarted them in a routine operation and now I can't restart due to the following error Do you use cross database references? not to my knowledge. That doesn't mean you aren't

Re: [ZODB-Dev] Re: ZODB Benchmarks

2008-03-20 Thread Chris Withers
Roché Compaan wrote: Not yet, they are very time consuming. I plan to do the same tests over ZEO next to determine what overhead ZEO introduces. Remember to try introducing more app servers and see where the bottleneck comes ;-) Am I right in thinking the storage servers is still

[ZODB-Dev] anecdotal speed comparison with RelStorage

2008-03-25 Thread Chris Withers
Hey All, Just finished doing some rough speed tests having got RelStorage up and running against an Oracle 10g database using Oracle's InstantClient and cx_Oracle 4.3.3... The tests involved basically creating 10 folders, each with 10 pages in them, in a Plone site, using zope.testbrowser.

Re: [ZODB-Dev] anecdotal speed comparison with RelStorage

2008-03-26 Thread Chris Withers
Shane Hathaway wrote: Chris Withers wrote: FileStorage-over-ZEO managed the test in 3 mins 20 seconds Relatorage-to-Oracle managed the test in 3 mins 18 seconds Cool, although it's not clear the storage speed had a major impact. Well, the point was that RelStorage-to-Oracle was of similar

Re: [ZODB-Dev] pack script for RelStorage

2008-03-27 Thread Chris Withers
Shane Hathaway wrote: Can't you use the standard manage_pack() method? Yes, pack it about the same way you would pack a ZEO storage. OK, but I do that using fspack.py ;-) Is there anything similar for RelStorage or do I need to do a zopectl run / stepper script? cheers, Chris --

[ZODB-Dev] RelStorage Oracle Barf...

2008-03-28 Thread Chris Withers
Hi Shane, First real world error from the Oracle adapter: Module ZPublisher.Publish, line 125, in publish Module Zope2.App.startup, line 238, in commit Module transaction._manager, line 96, in commit Module transaction._transaction, line 395, in commit Module transaction._transaction,

Re: [ZODB-Dev] RelStorage Oracle Barf...

2008-03-28 Thread Chris Withers
Chris Withers wrote: Module relstorage.relstorage, line 323, in store Module relstorage.adapters.oracle, line 454, in store_temp DatabaseError: ORA-24813: cannot send or receive an unsupported LOB This was while trying to create a Plone object... Interestingly: - I could reproduce

Re: [ZODB-Dev] anecdotal speed comparison with RelStorage

2008-03-28 Thread Chris Withers
Shane Hathaway wrote: I'd be really interested to try some tests with multiple ZEO clients attached to a FileStorage versus multiple RelStorage client attached to Oracle. I suspect Oracle would win ;-) I am certainly interested in hearing the results of that test. Well, we tried this fairly

Re: [ZODB-Dev] Re: RelStorage Oracle Barf...

2008-04-01 Thread Chris Withers
Shane Hathaway wrote: http://ora-24813.ora-code.com/ That page suggests you may be running different versions of Oracle on the client and server. I'm sure there are other possibilities, of course. Indeed, hopefully Guy will take a look some time. I should stress that this error was

[ZODB-Dev] what could cause this kind of cache corruption?

2008-07-31 Thread Chris Withers
Hi All, I have a project with one zeo server, one zope client, all Zope 2.9.8. With the client running, I ran a script that did nothing other than scan through data (ie: no changes, no commits) using zopectl run from the instance home of the client. After this, the zope client's web

Re: [ZODB-Dev] what could cause this kind of cache corruption?

2008-07-31 Thread Chris Withers
Jens Vagelpohl wrote: Sounds logical to me. The running instance and your zopectl run-script work in the same space with identical configurations, so both will start using the same ZEO cache file. To me this sounds like a recipe for disaster. You should run your script in a dedicated

Re: [ZODB-Dev] what could cause this kind of cache corruption?

2008-07-31 Thread Chris Withers
Alan Runyan wrote: Chris, Are you seeing this error? Nope, don't think so. cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki:

Re: [ZODB-Dev] what could cause this kind of cache corruption?

2008-08-11 Thread Chris Withers
Christian Theune wrote: However, I was hoping for inclusion of at least the `storage-iterator-branch` in 3.9 and I asked a few times over the past months already for review but haven't gotten any responses at all so far. This branch has been used in production for a few month already. In

Re: [ZODB-Dev] Backing up Data.fs and blob directory

2008-09-08 Thread Chris Withers
Sidnei da Silva wrote: Keep in mind rsync is not erm, trivial to get going on Windows. Really? I've never had problems with cygwin... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk

[ZODB-Dev] NotImplementedError when attempting to undo transactions

2008-10-21 Thread Chris Withers
This is a little disconcerting: Failed to abort object: TransactionalUndo oid=object object at 0xb7d01470 Traceback (most recent call last): File /opt/Zope-2.11/lib/python/transaction/_transaction.py, line 549, in abort self.manager.abort(o, txn) File

Re: [ZODB-Dev] NotImplementedError when attempting to undo transactions

2008-10-24 Thread Chris Withers
Jim Fulton wrote: File /opt/Zope-2.11/lib/python/ZODB/DB.py, line 809, in abort raise NotImplementedError NotImplementedError You can ignore this error. It has no consequence. What does it mean? (It prevented my undo, which was a bit of a bummer :-S There's a bunch of stuff going on

Re: [ZODB-Dev] Blobs, Copies, and Exports (Zope 3)

2008-10-24 Thread Chris Withers
Christian Theune wrote: On Fri, 2008-10-24 at 09:51 -0400, Jim Fulton wrote: 2. I doubt that blobs have been factored into ZODB exports. This is, obviously, an oversight. They were factored in and we have tests. However, the initial pickle will empty them: copying blobs this way including

Re: [ZODB-Dev] Blobs, Copies, and Exports (Zope 3)

2008-10-24 Thread Chris Withers
Christian Theune wrote: On Fri, 2008-10-24 at 15:06 +0100, Chris Withers wrote: Christian Theune wrote: On Fri, 2008-10-24 at 09:51 -0400, Jim Fulton wrote: 2. I doubt that blobs have been factored into ZODB exports. This is, obviously, an oversight. They were factored in and we have tests

Re: [ZODB-Dev] NotImplementedError when attempting to undo transactions

2008-10-24 Thread Chris Withers
Carlos de la Guardia wrote: That's happened to me before. As Jim says, this error is not the culprit. If you check the error log on the ZMI you will find the *real* error. This *was* the error that showed up in the ZMI... Chris -- Simplistix - Content Management, Zope Python Consulting

Re: [ZODB-Dev] RFC: External Garbage Collection

2008-11-07 Thread Chris Withers
Jim Fulton wrote: I've posted a new proposal: http://wiki.zope.org/ZODB/ExternalGC That addresses multi-database garbage collection and can also be useful in other situations. Comments are welcome. I assume this would fix the following bug:

[ZODB-Dev] SAN/RHCS use for ZEO server?

2009-04-03 Thread Chris Withers
Hi All, I was just wondering if anyone had deployed any high availability solutions for ZEO storage servers using a SAN for the filestore used by the storage server (and any associated BLOB files)? If so, how and how's it panned out? Likewise, anyone uses Red Hat Cluster Suite to implement

[ZODB-Dev] error installing ZODB 3.9.0a12 with Python 2.6 on Windows

2009-04-03 Thread Chris Withers
buildout said easy_install barfed: Installing zeoinstance. Getting distribution for 'ZODB3'. error: Setup script exited with error: None An error occured when trying to install ZODB3 3.9.0a12.Look above this message f or any errors thatwere output by easy_install. While: Installing

Re: [ZODB-Dev] error installing ZODB 3.9.0a12 with Python 2.6 on Windows

2009-04-03 Thread Chris Withers
Wichert Akkerman wrote: Any ideas? buildout hides all compile errors unless you run it with -v (or -vv). buildout -vv didn't exactly provide much more help ;-) C:\Python26\python.exe -c from setuptools.command.easy_install import main; main() -mUNxd C:\buildout-eggs\tmpctofbd

Re: [ZODB-Dev] error installing ZODB 3.9.0a12 with Python 2.6 on Windows

2009-04-03 Thread Chris Withers
Adam GROSZER wrote: Hello Chris, You have a c compiler working on the machine? Yep, mingw I think... But that doesn't look like the normal you don't have a compile whine... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk

Re: [ZODB-Dev] SAN/RHCS use for ZEO server?

2009-04-03 Thread Chris Withers
Andrew Sawyers wrote: I'll let you know shortly; I'm waiting on the test hardware for exactly this. I will be using an EMC SAN and SRDF replication. I'm particularly interested in how you'll move the SAN from the primary to the secondary node in the even of primary node failure, and how

Re: [ZODB-Dev] SAN/RHCS use for ZEO server?

2009-04-03 Thread Chris Withers
Wichert Akkerman wrote: On 4/3/09 3:43 PM, Chris Withers wrote: I'm particularly interested in how you'll move the SAN from the primary to the secondary node in the even of primary node failure, and how you'll bring the secondary's zeo server up when that happens. Doesn't ZEORaid work

Re: [ZODB-Dev] error installing ZODB 3.9.0a12 with Python 2.6 on Windows

2009-04-03 Thread Chris Withers
Jim Fulton wrote: Andreas tells me Python 2.6 is the target for Zope 2.12. What kind of problems are there in this specific combination? The tests crash when built with the free ms compiler. If you want to help, you can debug this. Sadly, easier to install on Linux. Blame vmware for

[ZODB-Dev] TypeError: ('argument list must be a tuple') when trying load object from old zodb in Zope 2.12.0a1

2009-04-13 Thread Chris Withers
Forwarding this here in case it's related. This is ZODB 3.9.0a11 on Python 2.5.1. The problem only occurs for certain ParsedXML documents, the two I've found so far were created in 2004-2005, which I guess would have been Zope 2.7ish on Python 2.4. Any ideas what this means or where it's

[ZODB-Dev] Non-zeo version length but ZMI says There are no non-empty versions.

2009-04-13 Thread Chris Withers
Hi All, I get the following error when trying to open a filestorage .fs file with ZODB3-3.9.0a11: File /ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/config.py, line 154, in open **options) File /ZODB3-3.9.0a11-py2.5-linux-i686.egg/ZODB/FileStorage/FileStorage.py, line 185, in __init__

Re: [ZODB-Dev] Non-zeo version length but ZMI says There are no non-empty versions.

2009-04-14 Thread Chris Withers
Alan Runyan wrote: reported, https://bugs.launchpad.net/zodb/+bug/361184 maybe chris can take a stab at it *wink* That'll be a no I'm afraid ;-) (I simply don't have the knowledge :-S) Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk

[ZODB-Dev] ValueError: database_name already in databases

2009-04-20 Thread Chris Withers
Hi All, Is this a bug in ZODBMountPoint or ZODB? In any case, what's the correct tracker to report this in? Failed to mount database. type 'exceptions.ValueError' (database_name 'packed' already in databases) Traceback (most recent call last): File

[ZODB-Dev] ZopeUndo.Prefix import problem

2009-04-20 Thread Chris Withers
Hi All, Trying to go to the Undo tab in the ZMI for Zope 2.12.0a3 for one database I have gives the following on the storage server: ERROR ZEO.zrpc (6397) can't decode message: '(K\x05K\x00U\x08undoInfo(K\x00K\x14}U\x0bdescription(cZ...' -- ZEO.zrpc.Connection(S) (127.0.0.1:49396) Error

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-26 Thread Chris Withers
Laurence Rowe wrote: Jim Fulton wrote: Well said. A feature I'd like to add is the ability to have persistent objects that don't get their own database records, so that you can get the benefit of having them track their changes without incuring the expense of a separate database object.

Re: [ZODB-Dev] URGENT: ZODB down - Important Software Application at CERN

2009-05-27 Thread Chris Withers
Hanno Schlichting wrote: Hanno Schlichting recently posted a nice graph showing the persistent structure of a Plone Page object and it's 9 (!) sub-objects. http://blog.hannosch.eu/2009/05/visualizing-persistent-structure-of.html That graph isn't quite correct ;-) workflow_history has

Re: [ZODB-Dev] Multi-database garbage collection and database verification

2009-06-12 Thread Chris Withers
Jim Fulton wrote: As threatened, I've created a multi-database garbage collector: http://pypi.python.org/pypi/zc.zodbdgc Bonus: I've also included a multi-database verification script. This looks great :-) A couple of questions: - How do I know if my databases use 64-bit object ids?

Re: [ZODB-Dev] Multi-database garbage collection and database verification

2009-06-12 Thread Chris Withers
Jim Fulton wrote: A couple of questions: - How do I know if my databases use 64-bit object ids? You have to ask the storage author. All of the databases included with ZODB do. Well, all the DB I want to use this on are FileStorage, so that should be good to go, right? Chris --

Re: [ZODB-Dev] Temporary files not closed

2009-06-25 Thread Chris Withers
Izak Burger wrote: Before I dive into the code and attempt to find the problem, I thought I'd post first, partly to report the problem, but at the same time to find out if anyone has seen this and if perhaps a fix is already available. Never experienced this myself, what storage(s) are you

Re: [ZODB-Dev] Temporary files not closed

2009-06-25 Thread Chris Withers
Jim Fulton wrote: I haven't seen this and, looking at the code on the trunk, I see no reason why many savepoint temporary storages or ZEO transaction buffers would be kept open. Savepoint temporary storages are closed after use I wonder if Izak inadvertently has lots of savepoints

Re: [ZODB-Dev] CVE-2009-0668 and CVE-2009-0669: Releases to fix ZODB ZEO server vulnerabilities

2009-08-06 Thread Chris Withers
Hi Jim, Jim Fulton wrote: CVE-2009-0668 Arbitrary Python code execution in ZODB ZEO storage servers CVE-2009-0669 Authentication bypass in ZODB ZEO storage servers Where are the actual CVE entries for these? http://cve.mitre.org doesn't seem to know much about either of them... The

[ZODB-Dev] what could cause slowness checking if databases are alive?

2009-08-12 Thread Chris Withers
Hi All, This is all on Zope 2.9.8 for both the ZEO server and the app server clients. I have a customer with 2 app servers behind a hardware loadbalancer. The load balancer uses a hash of a url on each app server to determine if the app server is alive. Likewise, if the app server doesn't

Re: [ZODB-Dev] ZODB 3.8.3b1

2009-08-15 Thread Chris Withers
Jim Fulton wrote: On Sat, Aug 15, 2009 at 8:04 AM, Chris Withersch...@simplistix.co.uk wrote: Jim Fulton wrote: It would be great if people who have databases with version data would try the utility out. Where's tfm for the utility? :-) Use the -h option. There really isn't much to it.

Re: [ZODB-Dev] ZODB 3.8.3b1

2009-08-15 Thread Chris Withers
Jim Fulton wrote: If you install ZODB3 3.8.3b1 normally, you'll get a strip_versions script in your bin directory. I'm not familar with the Zope 2 buildout or whether you can use ZODB 3.8 with Zope 2.12. Hmm, okay, I'll play it safe then... If I have a normal source install of Zope 2.9.10,

Re: [ZODB-Dev] ZODB 3.8.3b1

2009-08-15 Thread Chris Withers
Jim Fulton wrote: $ bin/strip_versions -h usage: strip_versions input output Remove version records by copying input to output, stripping any version records seen in input. The input and output arguments are file-storage file names. If the -c option is used, then the input and output

Re: [ZODB-Dev] ZODB 3.8.3b1

2009-08-25 Thread Chris Withers
Jim Fulton wrote: Yup, where I assume you mean using the in-use storage as input. The script makes a new copy. After making the copy, you can change the server to use that. Of course, changes made after copying won't be in the copy, but if you just want to try the script out, you can just

[ZODB-Dev] instrumenting a ZEO server?

2009-08-26 Thread Chris Withers
Hi All, I'm wondering if there's any way to get a ZEO server to log about how it's performing? (eg: how many connections, how many clients are waiting, where they're waiting, etc) I'm trying to debug a problem where app servers are being dropped out of a load balancer pool for being

Re: [ZODB-Dev] instrumenting a ZEO server?

2009-08-26 Thread Chris Withers
Alan Runyan wrote: What version of ZODB are you using? Whatever ships with Zope 2.9.8 If you have zeo on in debugging What does this mean? How many clients are waiting: You should be able to see this in zeo log files Not currently... maybe this relates to your mystical zeo on in

Re: [ZODB-Dev] ZODB 3.9.0 released

2009-09-08 Thread Chris Withers
Jim Fulton wrote: The changes from ZODB 3.8: http://pypi.python.org/pypi/ZODB3/3.9.0#id1 There are some typos in this list: - should probably be `with` when describing the with statement - If control exists the block I'm guessing should be If control exits the block - effecient -

Re: [ZODB-Dev] ZODB 3.9.0 released

2009-09-09 Thread Chris Withers
Jim Fulton wrote: There are some typos in this list: - should probably be `with` when describing the with statement I don't know what you mean. Databases have a new method, transaction, that can be used with the Python (2.5 and later) with statement: - Databases have a new method,

[ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Chris Withers
Hi All, Anyone ever see this? Traceback (most recent call last): File /opt/Zope-2.9//bin/repozo.py, line 517, in ? main() File /opt/Zope-2.9//bin/repozo.py, line 510, in main do_backup(options) File /opt/Zope-2.9//bin/repozo.py, line 450, in do_backup reposz, reposum =

[ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Hi All, As the subject line says with the following definitions: - size = size on disk, in Mb - subtee = Zope folderish type things, in this case... cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk

Re: [ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Chris Withers
Adam GROSZER wrote: Hello, +1 on that. Sometimes copy operations (over the network) screw up some bits. Happens rarely but then it hits hard. Monday, September 14, 2009, 10:13:39 AM, you wrote: CT Hmm. Haven't seen that. My guess would be some low-level data corruption. CT Christian

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Jim Fulton wrote: ZODB doesn't provide any direct support because it has no notion of tree. Does it have any notion of object size on disk for a given object? In Zope 2, you could export the folder and look at the size of the export. Sadly, when you're trying to figure out which folder is

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Jens Vagelpohl wrote: Sadly, when you're trying to figure out which folder is using all the space on a disk that's running out of space, that's not practical... How about simply writing a script that crawls the ZODB and gives you paths and sizes? Indeed, and what's the magic incantation

[ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Hi All, I've got a ZODB that's ballooned and I'm now having problems packing with zeopack. I tried with zeopack, which appeared to just hang (still in top, but no cpu usage for an hour or two), so I ctrl-c'ed it. Now when I try, I just get the text Already packing back. There is a .old file

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Shane Hathaway wrote: There is a .old file there, but it was last modified over a week ago. The analysis phase of packing can take a long time. A week is too much, though. :-) Yeah, this is off the back of an automated pack/backup/unpack/verify to keep two storages in sync on two database

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Shane Hathaway wrote: Chris Withers wrote: I thought RelStorage was slow at packing? Not anymore. I optimized packing last December/January and it has been plenty fast ever since, AFAIK. Interesting. Got any metrics on its speed compared to FileStorage, including packing? cheers, Chris

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-18 Thread Chris Withers
Shane Hathaway wrote: Look at this chart in particular, which suggests that RelStorage with PostgreSQL or MySQL is several times faster than ZEO for heavy read loads: Cool, now all I need is something like zeoraid for postgres (or, at a push, mysql)... Chris -- Simplistix - Content

Re: [ZODB-Dev] Cleaning up a ZODB storage from 'versions'

2009-09-30 Thread Chris Withers
Jim Fulton wrote: Is there a way for cleaning up a storage and removing the related versions in some way? Yes. Use the ZODB 3.8 strip_versions script. I used this buildout to get this script available in isolation: [buildout] parts = fixer versions = versions [versions] ZODB3 = 3.8.3b1

Re: [ZODB-Dev] tid to old for invq in zeo server logs?

2009-10-20 Thread Chris Withers
Jim Fulton wrote: It means that the server wasn't able to provide quick invalidations because a client was disconnected too long. The message isn't very well worded, even aside from the misspelling. Has that been fixed in the latest zodb? If not, if I get a chance, I'm happy to patch...

Re: [ZODB-Dev] ZODB 3.9.3 released

2009-10-26 Thread Chris Withers
Jim Fulton wrote: situations when deleting items. I recommend that people running 3.9 update to the new release as soon as reasonable. (If you're using ZEO, this only applies to ZEO clients, but servers. Just wanted to check this last sentence: what needs updating? clients, servers or both?

[ZODB-Dev] why use zc.recipe.filestorage?

2009-10-30 Thread Chris Withers
Hi All, I'm wondering what the point of this recipe is? Why use: [storage] recipe = zc.recipe.filestorage [zeo] recipe = zc.zodbrecipes:server zeo.conf = filestorage 1 path ${storage:path} /filestorage ...when you could just use: [zeo] recipe = zc.zodbrecipes:server zeo.conf =

Re: [ZODB-Dev] why use zc.recipe.filestorage?

2009-10-30 Thread Chris Withers
Jim Fulton wrote: [zeo] recipe = zc.zodbrecipes:server zeo.conf = filestorage 1 path ${buildout:directory}/databases/Data.fs /filestorage I feel like I'm missing something... You aren't missing anything. The benefit is very small. Excellent. I shall not feel guilty about

[ZODB-Dev] zodb 3.8.3 strip_versions missing some versions?

2009-11-02 Thread Chris Withers
Hi All, I'm upgrading a Zope 2.9.8 project to Zope 2.12 + zeoraid. As I've done with previous projects, I run the .fs files through zodb 3.8.3's strip_versions script. For both the storages in this project, the script found no version records. As a first step, I'm currently running the 2.9.8

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-02 Thread Chris Withers
Christian Theune wrote: There's a size argument following the version which probably is not being passed as a keyword argument so it looks like a slight API incompatibility. I wonder whether ZEORaid should hack around that or whether the caller should be changed. Hmm... looking more closely

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-02 Thread Chris Withers
Christian Theune wrote: Would anyone object if I wrote a test and fixed this on the 3.9 branch of ZODB? I'm not really sure what to do about the trunk... I think that's the best idea. It's not exactly part of the policy of supporting old protocols but if we want to support and encourage 3rd

Re: [ZODB-Dev] Test time on 3.9 branch

2009-11-02 Thread Chris Withers
Jim Fulton wrote: Should I really expect the tests to take an hour to run on Windows? Yes. Erk. Is there a subset I can look to run when looking to patch the ZEOStorage308Adapter? However, there were some error-ish messages interspersed in the test output. Is this to be expected? You

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-02 Thread Chris Withers
Jim Fulton wrote: I'm fine with change the storage server to pass the history size argument as a keyword parameter. Okay, but where is the Z309 equivalent of ZEOStorage308Adapter's history method? cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting

Re: [ZODB-Dev] Test time on 3.9 branch - windows?

2009-11-02 Thread Chris Withers
Jim Fulton wrote: Should I really expect the tests to take an hour to run on Windows? Yes. Erk. Is there a subset I can look to run when looking to patch the ZEOStorage308Adapter? Feel my windows pain. Do the tests take much less time to run on Linux? If so, any idea as to why? Chris --

[ZODB-Dev] zeoraid backends dropping out

2009-11-11 Thread Chris Withers
Hi All, Since this is the first post to this list on the issue, the setup I have is: zeoraid1 and zeoraid2 are zeoraid servers zeo1 and zeo2 are normal storage servers serving two file storages: packed and unpacked. zeoraid1 and zeo1 are on one box, zeoraid2 and zeo2 are on the other both. A

Re: [ZODB-Dev] help with pinning zeo plone zodb3 to 3.8.1

2009-11-12 Thread Chris Withers
rafael wrote: I am experiencing the following problem: i need to pin zodb3 = 3.8.1 so I can pack (cross reference issue). I use the following mini buildout to get 3.8.1 on its own. In my case it was to run the strip_versions script, but if you run it, you'll see you have a bin/runzeo and

[ZODB-Dev] repozo uses md5 lib, which is deprecated in python 2.6

2009-11-18 Thread Chris Withers
Hi All, ZODB's repozo script uses the md5 library, which is deprecated in python 2.6, resulting in annoying emails from cron jobs. A few questions: - Where's the tracker for ZODB nowadays to report this issue? - Anyone know what to replace md5 with? The warning suggests hashlib, which landed

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-19 Thread Chris Withers
Jim Fulton wrote: What's wrong with the original idea of having the server pass the size as a keyword parameter? The ZEO protocol is irrelevant. Okay, from the top: So, a ZODB 3.9.3 ClientStorage calls the ServerStub's history method, this does an rpc call: self.rpc.call('history', oid,

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-19 Thread Chris Withers
Jim Fulton wrote: So why not add a history method to ZEOStorage that passes size as a keyword argument? I don't know what ZEOStorage is... zrpc.connection.Connection appears to despatch direct to the storage, please correct me if I'm wrong. In this case, that's a RAIDStorage. It doesn't have

Re: [ZODB-Dev] repozo, neither official nor supported, apparently...

2009-11-19 Thread Chris Withers
Jim Fulton wrote: There's nothing official or supported about a backup solution without automated tests. So I guess there isn't one. Right, so what does Zope Corp use? Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-19 Thread Chris Withers
Jim Fulton wrote: On Thu, Nov 19, 2009 at 6:07 PM, Chris Withers ch...@simplistix.co.uk wrote: Jim Fulton wrote: So why not add a history method to ZEOStorage that passes size as a keyword argument? I don't know what ZEOStorage is... Then we shouldn't be having this conversation. I'm

Re: [ZODB-Dev] ZODB 3.9.3 history call causing problems for storages that still accept version parameters

2009-11-20 Thread Chris Withers
Christian Theune wrote: Traceback (most recent call last): File ZODB3-3.9.3-py2.6-linux-i686.egg/ZEO/zrpc/connection.py, line 581, in handle_request ret = meth(*args) File gocept.zeoraid-1.0b6-py2.6.egg/gocept/zeoraid/storage.py, line 219, in history assert version is ''

Re: [ZODB-Dev] repozo, neither official nor supported, apparently...

2009-11-20 Thread Chris Withers
Jim Fulton wrote: On Thu, Nov 19, 2009 at 7:01 PM, Chris Withers ch...@simplistix.co.uk wrote: Jim Fulton wrote: There's nothing official or supported about a backup solution without automated tests. So I guess there isn't one. Right, so what does Zope Corp use? We use ZRS, of course

Re: [ZODB-Dev] 3.9.4 release?

2009-11-20 Thread Chris Withers
Jim Fulton wrote: I need to review the changes before the release. I'll probably reject the repozo change without an automated test. Are you serious? You'd rather have a broken tool than one that isn't broken on the basis that the existing tests aren't part of the test suite that gets run by

Re: [ZODB-Dev] 3.9.4 release?

2009-11-20 Thread Chris Withers
Tim Peters wrote: That's what he said -- and you made him repeat it several times by now. Yes, I find it hard to believe that someone would deliberately break something that someone else has taken the trouble to fix (and run the tests for!)... I've learned my lesson, I won't try and

Re: [ZODB-Dev] repozo, neither official nor supported, apparently...

2009-11-20 Thread Chris Withers
Jim Fulton wrote: On Fri, Nov 20, 2009 at 9:32 AM, Chris Withers ch...@simplistix.co.uk wrote: ... I'm not sure how much love repozo needs. It works, and it won't need changing until FileStorage's format changes, which I don't see happening any time soon. It just occurred to me that repozo

Re: [ZODB-Dev] repozo, neither official nor supported, apparently...

2009-11-20 Thread Chris Withers
Jeff Shell wrote: For what it's worth, I've dealt with broken / unloved / mildly-out-of-date ZODB scripts by making a new package that my little company uses internally. I don't agree with this because it's fragmenting a tiny development group even further, no to mention creating private

Re: [ZODB-Dev] 3.9.4 release?

2009-11-20 Thread Chris Withers
Jim Fulton wrote: +class RepozoTests(unittest.TestCase): + +def test_importability(self): +from ZODB.scripts import repozo + I appreciate your dedication to quality. That test passes under ZODB 3.9.3 with Python 2.6. ...but it will emit a deprecation warning, which I

Re: [ZODB-Dev] Zope 2.12 and blobs

2009-11-20 Thread Chris Withers
Jim Fulton wrote: No, we should tell them they should use a mostly untested tool to do backups that doesn't work with blobs. That should inspire confidence. Does Zope 2.12 use Blobs? Not by default, no. Chris -- Simplistix - Content Management, Batch Processing Python Consulting

Re: [ZODB-Dev] errors logged at info in ZEO's connection

2009-11-20 Thread Chris Withers
Jim Fulton wrote: It seems a bit bizarre that exceptions are logged at INFO rather than ERROR... Agreed. A simple fix and test on a branch would be appreciated. Done: http://svn.zope.org/ZODB/branches/chrisw-error_logging/ It would be great if this, too, could make it in a 3.9.4

[ZODB-Dev] client cache bugs when a client switches from one zeo server to another

2009-11-24 Thread Chris Withers
Hi All, So, I got the following errors when using a setup I'll describe shortly: o Module Products.ZCatalog.CatalogBrains, line 86, in getObject o Module OFS.Traversable, line 250, in restrictedTraverse o Module OFS.Traversable, line 226, in unrestrictedTraverse __traceback_info__: ([],

<    1   2   3   >