[ZODB-Dev] Before-abort hook?
Hi all, The ExtFile product I maintain keeps its payload on the filesystem. All operations are performed using temporary files which are then saved or deleted on transaction commit or abort respectively. To this end an ExtFile provides _finish and _abort methods which are hooked up to a TM. Since Zope 2.8 (MVCC) I have a problem with aborts. On abort the machinery apparently nukes an ExtFile's persistent attributes, and I lose all information I need to clean up the external resources (filename, path, etc). What I need is a way to guarantee my cleanup code is called *before* the persistent part of my object vanishes. I currently manipulate the transaction's _resources list directly [1], but that's obviously not desirable. I have seen synchronizers, but I am unable to tell whether a beforeCompletion will end up to be a commit or abort. We now have before-commit hooks, and it seems what I need is a before- abort hook as well. Thoughts appreciated, Stefan [1] http://dev.plone.org/collective/browser/ExtFile/trunk/TM.py -- Anything that happens, happens. --Douglas Adams ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Extenuated memory
El jue, 23-03-2006 a las 15:02 -0500, Tim Peters escribió: [Antonio Beamud Montero] But my server doesn't free any memory, now it uses 55Mb of resident size... How I can debug this references? Many layers are involved in memory management, from ZODB through Python down to your platform C's malloc/free implementation and ending in your OS kernel Unless you become knowledgable about all of them (and vital details vary wildly across specific platform C and OS versions), chances are decent you'll never find a correct answer. I think I will remove ZODB from my server and use a SQL Object mapper like SQLObject to see if I can get an stable server, or the dbapi directly.. my experience with ZODB and threads has been a nightmare :( A lot of thanks for your help, but this server is in a production/stable system, and use unstable systems give me fear... (what's the next problem?) Greetings. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Before-abort hook?
Stefan H. Holek schrieb: Hi all, The ExtFile product I maintain keeps its payload on the filesystem. All operations are performed using temporary files which are then saved or deleted on transaction commit or abort respectively. Do you remember, when you use the tempfile module, you get a filehandle with a nameless file which automatically vanishes if you close the handle. Maybe this solves the problem for you? Regards Tino ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] edge case question about _p_deactiveate and modified objects
On Mar 24, 2006, at 9:11 AM, Chris Withers wrote: Hi All, What happens if you call _p_deactivate() on a modified object but before the modifications have been committed? Do the modifications get lost or will they still be saved when the next commit happens after the _p_deactivate()? Under normal operation, the _p_deactivate call is ignored on a modified object, so the changes are not lost (but _p_deactivate accomplished nothing). However, there is currently a bug that if you call _p_deactivate on a new persistent object that has a database connection then the whole thing will just be lost. I have a TODO to make a small test case of this and put it in the collector. AIUI from Jim, fixing this reliably will require some API changes. Gary ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Weird ZODB error on Zope startup :-(
[Chris Withers] Anyone ever see this error before and have a clue what it means? File lib/python/Zope2/Startup/__init__.py, line 47, in startup starter.setConfiguration(cfg) File lib/python/Zope2/App/startup.py, line 100, in startup File lib/python/Products/ZODBMountPoint/Mount.py, line 226, in close File lib/python/ZODB/Connection.py, line 240, in close ZODB.POSException.ConnectionStateError: Cannot close a connection joined to a transaction It means that some object in the Connection was in a modified state when an attempt to close the Connection was made. The current transaction must be committed or aborted first -- ZODB can't guess whether the pending changes should be committed or thrown away, so it won't let the app close the Connection until the app decides that for itself. Which version of Zope? Note that ZODBMountPoint isn't part of ZODB -- it's part of Zope, and gets rewritten a lot ;-) That's why the Zope version is important. If you want to make progress, look in SVN history for who rewrote it most recently for the version of Zope you're using. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Extenuated memory
Antonio Beamud Montero wrote at 2006-3-23 09:26 +0100: ... But my server doesn't free any memory, now it uses 55Mb of resident size... How I can debug this references? It has no chance -- and this is neither the fault of Python nor of the ZODB but of C's standard memory management: C does not use memory compaction (if it would, development would be much more difficult, as any memory use would need communication with the compacter (in the form of: I am currently using this memory, do not move it aways while I do). Thus, uses memory snippets cannot be moved in order to have all free memory at the end of the heap. *nix has 2 ways to allocate memory: * the brk system call that can add and remove memory from the end of the data/heap segment * and mmap which can add large memory blocks to the process address space. mmap is only reasonable for large blocks, smaller blocks are usually allocated via the brk system call -- and this memory can only be given back to the OS if a contiguous block is free at the end of the data/segment heap. The C level memory management decides when to give memory back this way (and how large the free block must be before it does). As occupied memory cannot be moved, its chances to give back large chunks are limited... This problem plages *ALL* C programs that intermittendly require large amounts of memory (but know in very large blocks which could use mmap). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
