Re: [ZODB-Dev] ZODB memory problems (was: processing a Very Large file)

2005-05-21 Thread Dieter Maurer
DJTB wrote at 2005-5-21 13:00 +0200: ... I'm having problems storing large amounts of objects in a ZODB. After committing changes to the database, elements are not cleared from memory. You can control how many objects the ZODB cache may contain. Note, however, that the objects are usually

[ZODB-Dev] [Proposal] Size controlled ZODB cache

2005-06-02 Thread Dieter Maurer
Currently, the ZODB cache can only be controlled via the maximal number of objects. This makes configuration complex as the actual limiting factor is the amount of available RAM and it is very difficult to estimate the size of the objects in the cache. I therefore propose the implementation of

RE: [ZODB-Dev] ZEO client hangs when combined with other asyncore code

2005-06-22 Thread Dieter Maurer
Tim Peters wrote at 2005-6-21 14:56 -0400: ... [Dieter Maurer] If you happen to run your application on Linux (and use the GDB), I can provide detailed instructions on how to find out where your code hangs... That would be helpful! Also you already solved the main problem, I will share how I

[ZODB-Dev] [Danger] ZEOClientStorage does not detect lost ZEO connection

2005-06-24 Thread Dieter Maurer
During high availability cluster checks we observed that ZEOClientStorage's approach to detect connection loss to its server is unrealiable. It relies on the fact that the operating system reports a lost connection. However, by default, TCP does not garantee any notice for broken connections.

RE: Re: [ZODB-Dev] Handling more databases with zeo

2005-06-29 Thread Dieter Maurer
Tim Peters wrote at 2005-6-28 17:09 -0400: As before, I'd run a different ZEO server for each database. I'm not sure that what you're doing here will be supported for much longer (or really even _is_ supported anymore -- see my last msg). I do not know whether it is supported but it works

Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Jens Vagelpohl wrote at 2005-7-12 21:49 +0100: [Lennart] If it is the majoritys agreement that this answer is the best one, and that the refresh functionality should be looked at as something that may or may not work purely on luck, then I'll mark it as deprecated and up for removal in 2.10.

Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Jens Vagelpohl wrote at 2005-7-12 18:46 +0100: ... Due to the nastiness Refresh has to perform under the cover even Shane has to admit that it simply is not some make development faster panacea. Shane might (perhaps) admit it but I do not :-) There is not much nastiness that Refresh does:

Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Victor Safronovich wrote at 2005-7-12 18:17 +0600: My product has 'Auto refresh mode' option checked. I changed some source code of my product, after that autorefresh have been occured. And after autorefresh I got strange several errors: ... self._store_objects(ObjectWriter(obj),

Re: [ZODB-Dev] Savepoints are invalidated once they are used

2005-07-14 Thread Dieter Maurer
Jeremy Hylton wrote at 2005-7-11 18:37 -0700: IIRC, the old implementation of savepoints kept a copy of the index at the time the savepoint was taken so that you could rollback to it multiple times. I don't think there's any way to avoid such a copy. Maybe, we keep the original implementation (a

Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Lennart Regebro wrote at 2005-7-12 19:17 +0200: ... The refresh functionality has been unstable since it became a part of Zope core, and I have ended up not using it since it since then I never used auto refresh but always use the manual refresh and I am very happy with it. Recently, I

Re: [ZODB-Dev] Reloading product in 2.8

2005-07-14 Thread Dieter Maurer
Lennart Regebro wrote at 2005-7-12 20:22 +0200: ... well, if it happens to work for you, then good luck, but it is actually a piece of unsupported crap that hasn't worked for years and that nobody is bothered to fix bugs in. I have the feeling that we have an easy tendency to say such things:

RE: [ZODB-Dev] Savepoints are invalidated once they are used

2005-07-15 Thread Dieter Maurer
Tim Peters wrote at 2005-7-14 19:20 -0400: ... [Dieter] If we do, we can (usually) create a one with the state after the restore. [Tim] Sorry, I couldn't follow that sentence. After I have restored to a savepoint s, then the current state is the one saved in s, thus the sequence:

RE: [ZODB-Dev] Savepoints are invalidated once they are used

2005-07-16 Thread Dieter Maurer
Tim Peters wrote at 2005-7-15 16:06 -0400: ... Anyway, for reasons explained before, making a savepoint would still need to copy the index, so I don't see that this would save anything worth saving. It might allow skipping an index copy when a _rollback_ is done (at the cost of clumsier coding

Re: [ZODB-Dev] detecting changes to objects in a database

2005-08-01 Thread Dieter Maurer
Arthur Peters wrote at 2005-7-31 23:07 -0500: I am working on a non-Zope app that needs to be notified when objects in the database change (to update the GUI). How can I get this notification in all cases including when the object is ghosted? I would really like to avoid polling. The ZODB informs

Re: [ZODB-Dev] Ordering before commit hooks

2005-08-24 Thread Dieter Maurer
Jim Fulton wrote at 2005-8-23 14:22 -0400: ... For example, you could have gotten to the same end here with the old method, by registering your actions with an object of your own creation, and registering just one commit hook with the transaction, where that one hook looked at the actions you

Re: [ZODB-Dev] Ordering before commit hooks

2005-08-26 Thread Dieter Maurer
Jim Fulton wrote at 2005-8-24 16:08 -0400: ... I have a *package* (similar to Archetypes SQLStorage) that wants to go after everything that might possibly change attributes whether those changes are done by another component from me or any package developped by someone else. So again, you

Re: [ZODB-Dev] Setting _p_changed on a ghost

2005-08-28 Thread Dieter Maurer
Tim Peters wrote at 2005-8-26 14:59 -0400: ... Doing obj._p_changed = True when obj is a ghost appears to be senseless (what could a user possibly intend by doing this?) I met this strange behaviour and considered it a bug. What I wanted to do: use a ZODB object to synchronize

Re: [ZODB-Dev] Setting _p_changed on a ghost

2005-08-31 Thread Dieter Maurer
Tim Peters wrote at 2005-8-26 14:59 -0400: ... I would like to make it an error (raise a ValueError exception) to attempt to set obj._p_changed to a true value when obj is a ghost. Does anyone object? I came along an argument against this change: Assume, you have a persistent object o with a

Re: [ZODB-Dev] Problems when Conflict Resolution import code that try to open client storage

2005-09-21 Thread Dieter Maurer
Paolo Linux wrote at 2005-9-21 11:23 +0200: ... I had zodb hanging when It had to solve Conflict situations. It turned out to be that it tried to import some modules that tried to instantiate ClientStorage... I easily fixed the hurting code... Lesson learned: - never automatically register in

[ZODB-Dev] Re: afterCommitHook

2005-09-25 Thread Dieter Maurer
Florent Guillaume wrote at 2005-9-24 10:10 +0200: Dieter Maurer [EMAIL PROTECTED] wrote: Thus, promiss the user that any persistent changes he may try will be lost: Put the afterCommit in its own transaction and abort it afterwards. I should have added will be lost, unless he itself

[ZODB-Dev] [Enhancement] Common cache for mounted connections

2005-09-30 Thread Dieter Maurer
We have the need to mount dozens, if not hundreds of ZODBs in a single Zope instance. With the current implementation, each ZODB connection has its own independent cache. This makes it very difficult to use the available memory efficiently for caching. As the caches are independent, a heavily

Re: [ZODB-Dev] [Enhancement] Common cache for mounted connections

2005-09-30 Thread Dieter Maurer
Toby Dickenson wrote at 2005-9-30 16:40 +0100: On Friday 30 September 2005 11:40, Dieter Maurer wrote: We have the need to mount dozens, if not hundreds of ZODBs in a single Zope instance. With the current implementation, each ZODB connection has its own independent cache. This makes it very

Re: [ZODB-Dev] Blobs revisited...

2005-10-08 Thread Dieter Maurer
Chris McDonough wrote at 2005-10-5 09:58 -0400: ... savepoint difficulty ... Sure. BlobStorage is a wrapper around an existing storage. The wrapped storage's connections might support savepoints, which implies that they support rollback. But supporting rollback for a BlobDataManager is

Re: Re[2]: [ZODB-Dev] know the state of an object

2005-10-19 Thread Dieter Maurer
Victor Safronovich wrote at 2005-10-18 13:18 +0600: ... May be usefull to add some simple workflow to Persistent class with states: GHOST UPTODATE CHANGED STICKY transitions: Most of the transitions are there (though through strange names) activate --

Re: [ZODB-Dev] Weird errors with Zope 2.7.7

2005-10-19 Thread Dieter Maurer
Chris Withers wrote at 2005-10-19 13:53 +0100: ... from binascii import unhexlify def oid_unrepr(oidr): if oidr.startswith('0x'): oidr = oidr[2:] h = unhexlify(oidr) return '\x00'*(8-len(h))+h def oidr2obj(app,oidr): return app._p_jar[oid_unrepr(oidr)] First

Re: [ZODB-Dev] Slow Zeo load times after upgrade to ZODB 3.4

2005-10-21 Thread Dieter Maurer
Erik A. Dahl wrote at 2005-10-21 08:56 -0400: I just upgraded a client site from zope 2.6.2 to 2.8.1 (which I think has ZODB 3.4.1). After the upgrade I'm seeing very slow initial load times for objects from the zeo server (much slower than before). I understand that the objects need to be

Re: [ZODB-Dev] question about connections

2005-10-21 Thread Dieter Maurer
Victor Safronovich wrote at 2005-10-21 14:54 +0600: ... app = Zope.bobo_application() try: self.running = True while self.running: get_transaction().begin() try: scheduler = app.unrestrictedTraverse(

Re: [ZODB-Dev] Slow Zeo load times after upgrade to ZODB 3.4

2005-10-22 Thread Dieter Maurer
Erik A. Dahl wrote at 2005-10-21 15:43 -0400: The ZODB cache lives in RAM and is a Python object cache. It is a per connection cache. The ZEO client cache lives on disk and is a pickle cache (it caches the object states not the objects themselves). It is shared by all connections to a single

Re: [ZODB-Dev] Slow Zeo load times after upgrade to ZODB 3.4

2005-10-22 Thread Dieter Maurer
Paul Winkler wrote at 2005-10-21 16:24 -0400: ... Why do you want to use the same zope.conf, and how can that possibly work anyway? You can't run two servers on the same port! He might use the same configuration file on different hosts. And I posted a ZConfig extension that allows to read the

Re: [ZODB-Dev] request on restrictedTraverse

2005-11-02 Thread Dieter Maurer
Daniel wrote at 2005-10-31 09:52 +0100: ... It seems that the restrictedTraverse method changes the request, doesn't it? It uses locally its own request object. You will see it only in __bobo_traverse__ methods during the restrictedTraverse. All objects (in the standard acquisition context)

Re: [ZODB-Dev] ZODB for context objects

2005-11-02 Thread Dieter Maurer
Andre Meyer wrote at 2005-10-25 23:12 +0200: I am not sure whether ZODB is the right thing to use for my needs. So, what do I want? I have a number of threaded objects that have a context as a separate entity. The object and context need to be separated to allow for the threads to run

Re: [ZODB-Dev] oids, numbers, hex and 8-byte strings

2005-11-02 Thread Dieter Maurer
Chris Withers wrote at 2005-10-25 13:11 +0100: ... The alias I suggested is purely to mirror that of the oid_repr function, and I would include it directly above of below the oid_repr function with bigger documentation clues for blind people like me ;-) Does anyone have any non-pedantic

Re: [ZODB-Dev] ZConfig Extension?

2005-11-02 Thread Dieter Maurer
Chris Withers wrote at 2005-10-25 12:29 +0100: Dieter Maurer wrote: And I posted a ZConfig extension that allows to read the environment (thus using environment variables in the configuration file). Did you talk to the ZConfig maintainer about merging this? I announced it and I think I have

RE: [ZODB-Dev] oids, numbers, hex and 8-byte strings

2005-11-03 Thread Dieter Maurer
Tim Peters wrote at 2005-11-1 15:00 -0500: ... I don't want to see any renaming, or aliasing, until the docs (comments docstrings) are clear. Usually, I am convinced that well chosen names form the largest part of the required documentation. That's why I sometimes call things bugs (do not

RE: [ZODB-Dev] strange savepoint behavior

2005-11-06 Thread Dieter Maurer
Tim Peters wrote at 2005-11-3 12:31 -0500: ... But those details are all artifacts of exactly how your test case is written. As a general rule, if you do a rollback _beyond_ the creation of a new persistent object P, you shouldn't expect P to be in a useful state anymore (and, as you've seen,

Re: [ZODB-Dev] misleading exception

2005-11-08 Thread Dieter Maurer
Jim Fulton wrote at 2005-11-6 11:22 -0500: Dieter Maurer wrote: You should file a bug report: http://www.zope.org/Collectors/Zope;, category database. This way, Tim can fix the hole in the ConnectionStateError check. It's a mistake to rely on Tim alone to fix bugs. Maybe, it is a mistake

Re: [ZODB-Dev] custom mount point container type

2005-11-15 Thread Dieter Maurer
Tarek Ziadé wrote at 2005-11-14 17:59 +0100: ... I am trying to create my own mount point container type, do I just have to base it on ZODBMountPoint.Mount.MountPoint, MountPoint is not a mount point container but the actual mount point. In what way should your new mount point container be

Re: [ZODB-Dev] POSKeyError tracing

2005-11-18 Thread Dieter Maurer
Paolo Linux wrote at 2005-11-17 10:23 +0100: ... I suspect that in some cases we do not manage Conflict Error (we retry for 3 times but then we give up...). Could this be connected with POSKeyErrors? It should not. Because giving up should mean you abort the transaction and your transaction

RE: [ZODB-Dev] Migration of obsolete BTrees to ZOPE 2.8

2005-11-18 Thread Dieter Maurer
Tim Peters wrote at 2005-11-18 10:02 -0500: [Margie Watkins CONT] The issue http://www.zope.org/Collectors/Zope/1695 asks for a statement from users who are affected by the missing modules. Unfortunately, that's become out of sight, out of mind since then -- the issue was opened 9 months ago,

Re: [ZODB-Dev] Multiple threads connecting to ZODB

2005-11-25 Thread Dieter Maurer
Alex Couper wrote at 2005-11-25 16:30 +: I am having trouble creating connections from different threads to my zodb. In the main thread, I create/get the database using this: storage=FileStorage('my.fs') db=DB(storage) The main thread subsequently creates other threads, and I would like

Re: [ZODB-Dev] Different weird errors with Zope 2.8.4

2005-11-29 Thread Dieter Maurer
Chris Withers wrote at 2005-11-29 14:33 +: Hot on the heals of my posts about Shouldn't load state for comes the eagerly awaited sequel Couldn't load state for ;-) ... File lib/python/ZEO/zrpc/connection.py, line 531, in call r_flags, r_args = self.wait(msgid) File

Re: [ZODB-Dev] Searching for Objects

2005-12-06 Thread Dieter Maurer
Chris Spencer wrote at 2005-12-4 16:32 -0500: Is there an existing way to search for objects in ZODB based on criteria? The ZODB by itself does not provide searching facilities. There are applications on top of the ZODB that do: * Andreas already told you about Zope's ZCatalog * there

Re: [ZODB-Dev] Deferred notifing from one thread to another.

2005-12-09 Thread Dieter Maurer
Victor Safronovich wrote at 2005-12-9 14:33 +0500: ... But when T2 still not commiting its transaction, dispatcher not see new element :(. Any transaction will only see committed changes. Only at transaction boundaries will a connection cache be updated for modifications by other transactions.

Re: [ZODB-Dev] lazy BTreeItems

2005-12-09 Thread Dieter Maurer
Victor Safronovich wrote at 2005-12-9 12:05 +0500: Hello zodb-dev! Is this a correct behaviour? from BTrees.IIBTree import IIBTree t = IIBTree(map(None,range(5),range(5))) list(t.keys()) [0, 1, 2, 3, 4] k = t.keys() list(k) [0, 1, 2, 3, 4] del t[0] list(k) [] BUT i = iter(t.keys()) del

RE: [ZODB-Dev] OID length

2005-12-09 Thread Dieter Maurer
Tim Peters wrote at 2005-12-9 10:46 -0500: ... I'm not sure where the code actually relies on stringness The cPickleCache explicitely checks for stringness. Not, that this were necessary. It just does. -- Dieter ___ For more information about ZODB,

Re: Re[2]: [ZODB-Dev] Deferred notifing from one thread to another.

2005-12-11 Thread Dieter Maurer
Victor Safronovich wrote at 2005-12-10 13:37 +0500: ... Have you some situations, when you need to know if transaction of thread1 commited, but you are in thread2? In other words How can i receive the signal that transaction in another thread commited( or aborted, but

RE: [ZODB-Dev] More exciting ZODB errors ;-)

2005-12-15 Thread Dieter Maurer
Tim Peters wrote at 2005-12-14 12:05 -0500: ... return self.rpc.call('tpc_begin', id, user, descr, ext, tid, status) File lib/python/ZEO/zrpc/connection.py, line 536, in call raise inst # error raised by server StorageTransactionError: Multiple simultaneous tpc_begin requests

Re: [ZODB-Dev] sessions in the presence of conflicts

2005-12-15 Thread Dieter Maurer
Dennis Allison wrote at 2005-12-14 12:58 -0800: ... _p_resolveConflict(self, oldState, savedState, newState) returns the state of the object after resolving different changes. The arguments are: oldState -- state of the object at the beginning of the current transaction

RE: [ZODB-Dev] Re: Connection pool makes no sense

2005-12-29 Thread Dieter Maurer
Tim Peters wrote at 2005-12-29 11:28 -0500: ... Or there are no strong reference to `obj`, but `obj` is part of cyclic garbage so _continues to exist_ until a round of Python's cyclic garbage collection runs. And this is *VERY* likely as any persistent object in the cache has a (strong, I

Re: [ZODB-Dev] Query Regrading ZODB FileStorage(.fs file)

2005-12-30 Thread Dieter Maurer
Monica chopra wrote at 2005-12-29 18:17 -0800: I have recently jumped into python and ZODB . I am quiet familar to syntax and everything but there is one Issue . when we create a .fs file or say data.fs and saves in it some objects.If we open this file in notepad or other editior. It shows

RE: [ZODB-Dev] Re: Connection pool makes no sense

2005-12-30 Thread Dieter Maurer
Tim Peters wrote at 2005-12-29 12:59 -0500: ... [Tim Peters] It means that _ConnectionPool no longer has a reason to remember anything about that Connection.  Application code can continue keeping it alive forever, though. [Denis Markov] But what about RDB-Connection what stay in cache

Re: [ZODB-Dev] Query Regrading ZODB FileStorage(.fs file)

2005-12-31 Thread Dieter Maurer
Please always stay on the list! Monica chopra wrote at 2005-12-30 19:22 -0800: ... Thanks for responding but can clarify more about how to do that. How can i make an ecrypted file. ... Dieter Maurer [EMAIL PROTECTED] wrote: Monica chopra wrote at 2005-12-29 18:17 -0800: ... It would

RE: [ZODB-Dev] Re: Connection pool makes no sense

2005-12-31 Thread Dieter Maurer
Tim Peters wrote at 2005-12-30 14:51 -0500: [Dieter Maurer] They did not tell us about their application. But, Zope's database adapters work like this. They use the ZODB cache (and its pool) as an RDB connection pool. Thus, if the ZODB caches are not released, the RDB connections won't. I

RE: [ZODB-Dev] Re: Connection pool makes no sense

2006-01-01 Thread Dieter Maurer
David Rushby wrote at 2005-12-30 11:14 -0800: ... Since ZODB doesn't know whether the connection it's releasing from the pool is still in use, I don't know whether resetting the connection's cache is appropriate as a general solution. But it fixes a definite problem with Zope's behavior as a

Re: [ZODB-Dev] ZODB with a Relational Database

2006-01-03 Thread Dieter Maurer
José Carlos Senciales wrote at 2006-1-3 12:28 +0100: I´m wondering if i can use ZODB without files ´Data.fs´ and use a ralational database like Mysql to save the objects... is this possible?? Maybe, you can look at APE? -- Dieter ___ For more

RE: [ZODB-Dev] Re: Connection pool makes no sense

2006-01-03 Thread Dieter Maurer
Tim Peters wrote at 2006-1-2 15:05 -0500: I'm still baffled by how you get into this state to begin with. Florent explained why earlier, and I didn't see a reply to this part: [Florent Guillaume] By itself Zope never uses more than one connection per thread, and the number of thread is

Re: [ZODB-Dev] Searching/wo/Zope

2006-01-05 Thread Dieter Maurer
Tamas Hegedus wrote at 2006-1-4 14:38 -0500: You have released the ZODB/ZEO as a stand alone package. But there is no stand alone searching possibility. Why? This just does not make any sense. Yous suggested and I tried Kevin Dangoor's stand alone ZCatalog. It uses the Zope source just remove

Re: [ZODB-Dev] Another interesting ZODB cache inconsistency

2006-01-15 Thread Dieter Maurer
Jim Fulton wrote at 2006-1-13 17:54 -0500: ... This means that interrupting ZEO while it is sending invalidation messages can cause inconsitent states in the ZODB caches of its clients. What do you mean by inconsistent states? Do you mean inconsistent between clients? The inconsistent ZEO

Re: [ZODB-Dev] Weird ConflictErrors with Zope 2.8.4 - It's been a while ; -)

2006-01-24 Thread Dieter Maurer
Chris Withers wrote at 2006-1-24 13:41 +: Okay, now that we have 2.8.4 in place, we get proper reporting of ConflictErrors and today we started seeing one happening over and over again which looked roughly as follows: Traceback (most recent call last): File

Re: [ZODB-Dev] ioerror on mounting db (zope mounpoint)

2006-01-25 Thread Dieter Maurer
Jürgen Herrmann wrote at 2006-1-25 11:00 +0100: hi! after upgrading to zope 2.9 i receeive the following errors for about 20-30s after restart: 2006-01-25 10:54:16 ERROR ZODB.lock_file Error locking file /home/bliss/zope/var/bliss_data.fs.lock Traceback (most recent call last): File

Re: [ZODB-Dev] ZEO.ClientStorage replication: invalidations don't work

2006-01-26 Thread Dieter Maurer
Mihai Ilinca wrote at 2006-1-26 15:10 +0200: ... Trough 'slave' I only do read operations at the moment. If I make changes to the stored data trough the 'main' client, the 'slave' clients don't see it. They only see the data as it was when the ClientStorage was constructed. If I restart a

Re: [ZODB-Dev] runzeo.py options?

2006-02-08 Thread Dieter Maurer
Jeff Bauer wrote at 2006-2-8 08:59 -0600: Where are the runzero.py options documented? At the top of the source file .../ZEO/runzeo.py -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev

Re: [ZODB-Dev] packaging zodb in eggs.

2006-03-03 Thread Dieter Maurer
Chris McDonough wrote at 2006-3-2 22:03 -0500: ... The ZODB egg might include both ZODB and ZEO (these are linked inextricably) which depends on the persistence distribution. Transaction should be dependency-free (but probably isn't, I haven't gotten that far yet). While ZEO is of no use

Re: [ZODB-Dev] Re: Historical Persistent References -- Feedback Wanted!

2006-03-14 Thread Dieter Maurer
Matt Hahnfeld wrote at 2006-3-13 14:33 -0500: ... I found your code a while back and actually looked it over quit a bit before I started coding the solution I attached. Would you be able to explain how it works? It works through a special HistoryJar (for some unknown reason, the ZODB uses jar

Re: [ZODB-Dev] ImportError results in AttributeError later

2006-03-15 Thread Dieter Maurer
Thomas Güttler wrote at 2006-3-15 11:30 +0100: ... But I would like to know, why the __import__ does not raise an ImportError, but instead returns a half imported module. A buglet in Python, fixed from version 2.4 on. -- Dieter ___ For more

Re: [ZODB-Dev] Zodb with Yaml

2006-03-19 Thread Dieter Maurer
Chris McDonough wrote at 2006-3-18 20:58 -0500: If yaml (or pysyck) supports all of the features of pickle, it's theoretically possible. You would need to track down all of the places in ZODB where the code serializes and deserializes data from pickle format and write alternate

Re: [ZODB-Dev] Extenuated memory

2006-03-22 Thread Dieter Maurer
Antonio Beamud Montero wrote at 2006-3-22 10:56 +0100: ... If I exectue a .minimizeCache(), then the output is: [['request.Request', 504]] But I can't minimize more the cache... Looks as if your application keeps references to your request.Request objects. Check where and get rid of them. --

Re: [ZODB-Dev] Extenuated memory

2006-03-24 Thread Dieter Maurer
Antonio Beamud Montero wrote at 2006-3-23 09:26 +0100: ... But my server doesn't free any memory, now it uses 55Mb of resident size... How I can debug this references? It has no chance -- and this is neither the fault of Python nor of the ZODB but of C's standard memory management: C does

Re: [ZODB-Dev] Mindnumbing POSkeyError

2006-04-01 Thread Dieter Maurer
Stoons - Headnet wrote at 2006-3-29 13:06 +0200: ... Of cause I have tried to del / manage_delObjects it, but both times ZODB.POSException.POSKeyError: 0x06f21b was returned in both cases I ran the following: === o = app['kristendom']['medier']['billeder']['image.2006-03-19.5540330656']

Re: [ZODB-Dev] [Performance] excessive time in Connection.afterCompletion

2006-04-02 Thread Dieter Maurer
Hi Tim, Tim Peters wrote at 2006-3-31 18:48 -0500: ... What is the motivation to request synchronized transaction completion by default? Primarily so we could stop seeing endless threads on zodb-dev asking why ZEO clients that didn't happen to modify any objects never saw current data. From

Re: [ZODB-Dev] Re: [Zope-dev] Re: 64-bit BTrees

2006-04-18 Thread Dieter Maurer
Chris Withers wrote at 2006-4-18 08:34 +0100: ... If having two isn't acceptable, then why do we have an I and O BTree's, not to mention the special ones used for in-memory ZODB indexes? Surely we should just have one BTree class? Using I versus O BTrees makes a huge difference for mass

Re: [ZODB-Dev] Discrete synchronization with ZEO

2006-04-24 Thread Dieter Maurer
Arve Knudsen wrote at 2006-4-23 22:44 +0300: I already use ZODB as a local persistency service for my Python application, now I'm considering ZEO as a distributed data source. The thing is I don't want to keep my local object cache synchronized at all times with the remote database, my

Re: [ZODB-Dev] world gone wrong after write conflict

2006-04-24 Thread Dieter Maurer
John Belmonte wrote at 2006-4-22 19:13 -0400: ... I'm attempting to dust off my old high-write ZODB/ZEO app, and am having a hard time tracking down a problem. Something goes very wrong in the case where I catch a write conflict, abort the transaction, and attempt a retry. Frequently it appears

Re: [ZODB-Dev] Getting all OIDs from a storage.

2006-04-24 Thread Dieter Maurer
Jim Fulton wrote at 2006-4-24 10:48 -0400: ... I would like to have this. I can't find a proposal for it, so I guess we should start with that, I'll note that I'd like to have an interface like: oidIterator(class_glob='*') Returns an iterable of object ids whose dotted class names

Re: [ZODB-Dev] Discrete synchronization with ZEO

2006-04-25 Thread Dieter Maurer
Arve Knudsen wrote at 2006-4-25 17:32 +0300: ... How should I ensure local ownership of objects from the ClientStorage, by deep copying them? The ZODB does not have any ownership concept. Zope, on the other hand has two: executable ownership and local role ownership. Executable ownership

Re: [ZODB-Dev] Discrete synchronization with ZEO

2006-04-26 Thread Dieter Maurer
Arve Knudsen wrote at 2006-4-25 22:07 +0300: I must say I'm not an expert on all things ZODB/ZEO, but what I meant by ownership is the management of persistent objects. The cache management logic of ZODB can the way I understand it ghostify objects, what happens if I insert an object from

Re: [ZODB-Dev] Discrete synchronization with ZEO

2006-04-27 Thread Dieter Maurer
Arve Knudsen wrote at 2006-4-26 21:53 +0300: ... Well, this is pretty obvious if I deep copy the objects of course. What happens if I insert references to objects contained in the ZEO-served database into another database, though, will they be controlled by this second database (or the

Re: [ZODB-Dev] Missing Content

2006-05-01 Thread Dieter Maurer
Michael Havard wrote at 2006-5-1 06:57 -0500: Python 2.3.5, Zope 2.8.4, Plone 2.1.2, and a host of other products. ZEO configuration. Persistent disk caching is on. Yesterday everything was fine. All the content was there and accessible. Today I try to log in and the server doesn't respond. I go

Re: [ZODB-Dev] tid error with ClientStorage cache in Zope 2.9.2

2006-05-01 Thread Dieter Maurer
Chris Withers wrote at 2006-5-1 11:34 +0100: ... ...well, there are .zec files in the var directory, so I guess I must be. What controls whether a persistent or temporary client cache is used? Your Zope configuration file, of course ;-) It is the client option of the zeoclient section. You

Re: [ZODB-Dev] AssertionError on multithread ZEO app

2006-05-25 Thread Dieter Maurer
Pedro Werneck wrote at 2006-5-23 20:24 -0300: ... The problem is... I can't see the committed changes from the main thread, unless I do a commit there too, which seems strange. You see modifications only after crossing a transaction boundary. In the same transaction, you see only the

Re: [ZODB-Dev] Getting an up to date instance of a persistent object

2006-05-25 Thread Dieter Maurer
Pascal Peregrina wrote at 2006-5-24 09:57 +0100: ... I have a question regarding what triggers the reloading of an object state from ZODB (in order to set the most up to date one). There is only one reason: that it is not in the cache or a ghost. At transaction boundaries (and only there!)

Re: [ZODB-Dev] AssertionError on multithread ZEO app

2006-05-26 Thread Dieter Maurer
Please stay on the mailing list -- readded. Pedro Werneck wrote at 2006-5-25 21:33 -0300: ... You see modifications only after crossing a transaction boundary. In the same transaction, you see only the modification made by this transaction (not by others, even committed). OK... so, how can I

Re: [ZODB-Dev] Resource Managers not getting collected?

2006-05-27 Thread Dieter Maurer
Hi Sidnei, Sidnei da Silva wrote at 2006-5-26 13:37 -0300: On Thu, May 25, 2006 at 02:41:32AM -0300, Sidnei da Silva wrote: | Anyone experienced this? Any clues about what to do to get more | information? /me still expecting Dieter Maurer's I've fixed this in my private copy of Zope by doing X,

Re: [ZODB-Dev] Clustering ZEO Server with drbd

2006-06-03 Thread Dieter Maurer
[EMAIL PROTECTED] wrote at 2006-5-30 15:17 -0700: I would be very helpful if someone could summarize what needs to be done to get this to work. I don't have any experience with the ZEO server so any input would be greatly appreciated. There is a standard FileStorage documentation which tells

[ZODB-Dev] [Warning] potential ZODB cache inconsistency after client reconnect

2006-06-12 Thread Dieter Maurer
I fear we may get a ZODB cache inconsistency after a client disconnect. The current reconnection protocol essentially works as follows: * ClientStorage detects a broken ZEO connection and starts a reconnect thread * The reconnect thread tries to reconnect to the ZEO server and

Re: [ZODB-Dev] proposed comments

2006-06-21 Thread Dieter Maurer
Florent Guillaume wrote at 2006-6-20 16:35 +0200: Is it ok if I add the following comments to Connection's docstring? I do not think they should become part of the class' docstring. Having comments at the place where the attributes are introduced (which are used for other attributes) would be

Re: [ZODB-Dev] ZEO LoadEx?

2006-06-21 Thread Dieter Maurer
Chris Withers wrote at 2006-6-20 07:31 +0100: ... But, there also seems to be (to my untrained eye) random periods of slowness from then onwards, again, without any apparent signs of excess load, memory usage or disk i/o. I'm somewhat flumoxed as to where to go from here on the debugging

Re: [ZODB-Dev] What is the best document about ZODB internals?

2006-06-22 Thread Dieter Maurer
Eino Mäkitalo wrote at 2006-6-22 13:06 +0300: What is the best document about ZODB internals? Questions with respect to the best are always very difficult... I can and will read code but are there any good documentation to understand ideas why it's implemented that way. There are the

Re: [ZODB-Dev] Re: What makes the ZODB slow?

2006-06-23 Thread Dieter Maurer
Jean Jordaan wrote at 2006-6-23 16:24 +0200: ... write conflicts by large transactions ... To mitigate this, we want to create a savepoint and then commit more often while iterating and changing workflow, rolling back to the savepoint if necessary. I fear this will not work -- at least not when

Re: [ZODB-Dev] What makes the ZODB slow?

2006-06-23 Thread Dieter Maurer
Roché Compaan wrote at 2006-6-22 21:53 +0200: ... What overhead does undo add to performance? Very few -- apart from a fast growing storage file. However, the log behaviour of FileStorage means that you get a very different notion of locallity. In a relational database, records in the same

Re: [ZODB-Dev] Advice needed

2006-06-23 Thread Dieter Maurer
Adam Groszer wrote at 2006-6-23 14:27 +0200: ... Some fears they are having and I can't find unambiguous information: - Is ZODB a good choice for this app? It depends... At least careful design is necessary! The most problematic aspects of the ZODB are write conflicts. When two concurrent

Re: [ZODB-Dev] Advice needed

2006-06-24 Thread Dieter Maurer
Roché Compaan wrote at 2006-6-24 08:53 +0200: ... I am curious what other strategies besides QueueCatalog you employ? Careful design with respect to the granularity and locality of persistent objects: Move groups of large and rarely used attributes out into persistent subobjects. Move

Re: [ZODB-Dev] Re: What makes the ZODB slow?

2006-06-24 Thread Dieter Maurer
Roché Compaan wrote at 2006-6-23 19:04 +0200: ... In a test where one commits an instance of a Persistent subclass that have only 2 string attributes, 300 objects per second are created on average. Writing the exact same strings to a two column table in an RDBMS, yields more than 3000 records per

Re: [ZODB-Dev] Advice needed

2006-06-25 Thread Dieter Maurer
Roché Compaan wrote at 2006-6-25 11:13 +0200: ... Careful design with respect to the granularity and locality of persistent objects: Move groups of large and rarely used attributes out into persistent subobjects. Will this lead to smaller transaction sizes for objects that store large

[ZODB-Dev] Re: More exciting ZODB errors ;-)

2006-06-26 Thread Dieter Maurer
Hallo Florent, Florent Guillaume wrote at 2006-6-26 00:53 +0200: Dieter Maurer wrote: Chris Withers wrote at 2005-12-14 16:23 +: ... File lib/python/ZODB/Connection.py, line 788, in _setstate_noncurrent assert end is not None AssertionError This means that the latest

Re: [ZODB-Dev] Re: What makes the ZODB slow?

2006-06-27 Thread Dieter Maurer
Chris Withers wrote at 2006-6-27 09:56 +0100: Dieter Maurer wrote: PostGres does use looks, lots of them and for different purposes. Could ZODB use locks to gain a similar performance boost? Maybe, but it would be a really big change... However, as I explained in an earlier message, the major

Re: [ZODB-Dev] ZODB Conflict Errors

2006-06-29 Thread Dieter Maurer
[EMAIL PROTECTED] wrote at 2006-6-29 11:05 -0400: ... 2006-06-28T17:18:38 INFO ZODB.Conflict database conflict error (oid 0x0300bc, class BTrees._OOBTree.OOBTree, serial this txn started with 0x036671be9feda500 2006-06-28 21:18:37.483195, serial currently committed 0x036671bea30a9dee

Re: [ZODB-Dev] Adopting ZODB

2006-07-13 Thread Dieter Maurer
Christian Theune wrote at 2006-7-13 17:13 +0200: ... Is there any ability in ZODB to retroactively wrap objects in a persistence mechanism instead of having to rewrite an entire library to use the Persistent class? You can always persist (almost) any object, even if it does not subclass from

Re: [ZODB-Dev] Threads and Connections

2006-07-26 Thread Dieter Maurer
Chris S wrote at 2006-7-25 23:40 -0400: The problem is the only objects I need to grab are the ones loaded by other connections. I use Python's garbage collector to track references to instances of classes that have been updated. Use a customized ZODB.Conntection.Connection class instead.

Re: [ZODB-Dev] Threads and Connections

2006-07-29 Thread Dieter Maurer
Chris S wrote at 2006-7-26 16:03 -0400: ... When a file containing source code is modified, I want to reload it using Python's reload() function, then update classes with the new definitions. I have application-level logic to ensure the objects are in a sane-state during update. The only problem

  1   2   3   >