Re: [ZODB-Dev] Re: Multi-Threading Transactions
--On 13. Oktober 2005 02:44:34 -0400 Chris Spencer [EMAIL PROTECTED] wrote: I understand that, but my point was when you call transaction.commit(), you don't necessarily know what you're committing. One thread may be ready to commit. Another may not be. If one thread calls transaction.abort(), they may be aborting changes made by another thread. You'd have to write a complicated coordination mechanism to handle multi-threaded transactions to be sure. If threading is limited to connections, then wouldn't in make sense if transactions were as well? I have no idea what you mean. Once again: when you are using multiple threads to write to the ZODB (as Zope does all the time) then usually every request is done with a dedicated transaction. The work within the threads happens isolated. Only at commit time an error might occurs when multiple thread write to the *same* object or one writes and one reads (already solved by using MVCC in Zope 2.8). NO idea what you mean by complicated coordination. Hopefully you don't write code that spawns up threads that deal will the ZODB in an insane way. This is tricky and you should really know what you are doing in this casethis is for experienced developers only -aj pgpOG6TpLHYjw.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Fixing Broken Objects?
--On 13. Oktober 2005 02:54:51 -0400 Chris Spencer [EMAIL PROTECTED] wrote: But what if the client doesn't know what sources it's supposed to have? What if it only has a limited understanding of the objects, and needs to load certain objects on the server from scratch. Is this really a necessary limitation? I realize I may be using Zope in a way it was not designed for, but wouldn't it be useful to persist the entire object, source code as well as state? Sorry, this is a stupid assumption. The ZODB stores Python pickles and not any kind of source. Means when you load objects from the ZODB the corresponding classes in terms of sources must be present to unpickle the data. This means for you: every ZEO client need the full source. This is not an implementation issue but just an issue of your ZEO installation: when you load data from the ZODB then the sources must be present, otherwise your objects will be broken. Point. -aj pgpEXomgDqMlp.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Multi-Threading Transactions
Chris Spencer wrote: I understand that, but my point was when you call transaction.commit(), you don't necessarily know what you're committing. Yes you do, each thread has its own connection to the database, and this connection has an independent view of the database from any other thread. Calling commit on one connection does squat on any other connections ;-) Now, unless you're writing your own threading code, or your own connection code, you don't need to worry 'cos Zope does the right thing. If you are writing your own threading or connection code, you're either god-like, or an idiot. 99.5% percent of people who _think_ they need to write connection or threading code fall into the latter category... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Fixing Broken Objects?
Andreas Jung wrote: Sounds like a total insane idea to me to use ZEO to distribute code :-) Yeah, 'cos ZODB-based Script (Python)'s and ZPT's aren't code :-P Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] know the state of an object
Hi all, this my first post to this list, and first of all thanks for your work. I'm developing a ZODB based Collection Management software, and, for a bunch of reasons, i have to know the list of modified objects before the current transaction commit. Looking around seems there is no a public API to obtains this list from the connection object, so i overload the __setattr__ method in my base-class and keep track of the modified objects But that's the problem, __setattr__ is called also when ZODB loads the objects from the storage and reconstructs it, so is there a way to know the state of the object (unpickilng, modified, clean etc etc)? thank you david ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Fixing Broken Objects?
--On 13. Oktober 2005 09:46:43 +0100 Chris Withers [EMAIL PROTECTED] wrote: Andreas Jung wrote: Sounds like a total insane idea to me to use ZEO to distribute code :-) Yeah, 'cos ZODB-based Script (Python)'s and ZPT's aren't code :-P We are basically talking about the common usecase...Scripts and ZPTs are somewhat different for known reasons...the ZODB is mostly used as storage for pickled data and usually not for code objects. -aj pgpwnVkptIelD.pgp Description: PGP signature ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Multi-Threading Transactions
On 10/13/05, Chris Spencer [EMAIL PROTECTED] wrote: I understand that, but my point was when you call transaction.commit(), you don't necessarily know what you're committing. One thread may be ready to commit. Another may not be. If one thread calls transaction.abort(), they may be aborting changes made by another thread. You'd have to write a complicated coordination mechanism to handle multi-threaded transactions to be sure. If threading is limited to connections, then wouldn't in make sense if transactions were as well? The default transaction manager maintains a separate transaction for each thread. When you call transaction.commit() from thread T1, it commits the transaction for thread T1. It has no effect on other thread's running transactions. Jeremy ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Multi-Threading Transactions
[Jeremy] The default transaction manager maintains a separate transaction for each thread. When you call transaction.commit() from thread T1, it commits the transaction for thread T1. It has no effect on other thread's running transactions. My apologies if this has been mentioned already (I'm reading this via the Communigate interface, which as at least Jeremy knows too well wink makes following a discussion virtually impossible): the intended way to use ZODB is to open a distinct Connection for each thread. Then if, e.g., threads T1 and T2 both do the_thread_Connection.root()['whatever'] = whatever2 T1 and T2 end up with distinct _in-memory_ copies of the same persistent object. They don't interfere with each other. Whichever thread commits first wins. The second thread to commit will get a (write) conflict error if both threads modified the same persistent object, although there's a possiblity to supply conflict resolution methods too. Chris, is this getting clearer? One thread == one Connection == one transaction. The in-memory materialization of a persistent object loaded from one thread via its connection should _never_ be touched (or even looked at) by any other thread. That last part is a matter of programmer discipline, but is really quite easy in practice. If the programmer doesn't go out of their way to violate it, then each thread naturally sees its own self-consistent view of database state, from its own connection, and in its own transaction. The only point of intersection is commit() time. Now you can break that easy model by sharing in-memory persistent objects across threads, or by trying to use a single Connection in more than one thread. ZODB is not intended to be used in such ways, and any number of bad things will happen if you do them. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] ZODB 3.2.10 final released
I'm pleased to announce the release of ZODB 3.2.10 final. This corresponds to the ZODB in Zope 2.7.8 final. You can download a source tarball or Windows installer from: http://zope.org/Products/ZODB3.2 Note that there are two Windows installers, for Python 2.3 (2.3.5 is recommended) and Python 2.4 (2.4.2 is recommended). ZODB 3.2.10 contains a few bugfixes relative to 3.2.9, all in obscure error cases. The most serious is a workaround for what appears to be a rare race bug in Microsoft's implementation of socket binding on Windows platforms. See the NEWS file for details: http://zope.org/Products/ZODB3.2/NEWS The current status of all active ZODB lines can be seen here: http://www.zope.org/Wikis/ZODB/CurrentStatus ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: Fixing Broken Objects?
Chris Spencer wrote: I noticed that if a ZEOServer client doesn't have the definition for an object's class, it loads it as a broken object. Is there any way to fix the broken object by obtaining the class source from the server? Others have commented on the status quo that your client must contain the correct code classes for your pickles. However your question of loading code from a remote location is valid, and another Python remote object system (Pyro) that can be configured, with some restrictions, to load code over the network: http://pyro.sourceforge.net/ in particular, see the section on mobile code: http://pyro.sourceforge.net/manual/7-features.html#mobile I think twisted has a similar module but I don't know if it does the mobile code concept. I've never heard of anyone doing it, but there might be some synergy with combining pyro's mobile code with ZODB and ZEO. -Michel ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Fixing Broken Objects?
Andreas Jung wrote: ... Sorry, this is a stupid assumption. No. It was an incorrect assumption. Jim -- Jim Fulton mailto:[EMAIL PROTECTED] Python Powered! CTO (540) 361-1714http://www.python.org Zope Corporation http://www.zope.com http://www.zope.org ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
RE: [ZODB-Dev] know the state of an object
[dvd] this my first post to this list, and first of all thanks for your work. I'm developing a ZODB based Collection Management software, and, for a bunch of reasons, i have to know the list of modified objects before the current transaction commit. Looking around seems there is no a public API to obtains this list That's true. from the connection object, You didn't say which version of ZODB you're using. Since you believe a Connection keeps track of which objects have been modified, I'll assume you're using ZODB 3.3 or later (which is either true, or your belief about who keeps track of modified objects is false ;-)). so i overload the __setattr__ method in my base-class and keep track of the modified objects But that's the problem, __setattr__ is called also when ZODB loads the objects from the storage and reconstructs it, so is there a way to know the state of the object (unpickilng, modified, clean etc etc)? The good news is that obj._p_state reveals the current state of a persistent object. The possible values are exposed by the `persistent` module: import persistent persistent.GHOST -1 persistent.UPTODATE 0 persistent.CHANGED 1 persistent.STICKY 2 The possible state transitions are explained in persistent/interfaces.py, where UPTODATE is called saved. For reasons that escape me, it doesn't admit to the existence of _p_state (maybe that's not an official part of the API?). The bad news is that, during the transition from GHOST to UPTODATE (your ZODB loads the objects from the storage and reconstructs it), the state is temporarily set to CHANGED _before_ code is invoked to materialize the object's state (and is set to UPTODATE after that's done). This is to prevent runaway recursion, but no matter: while an object is being unghostified, it (untruthfully) claims to be in the CHANGED state, not the GHOST state you might expect. So, in the end, I don't see any hope for you via this route, short of this: register an object as changed in your __setattr__ without worrying at all about _why_ __setattr__ was called. Later, when you do something with your list of modified objects, simply ignore any whose state at that time is UPTODATE. Those are exactly the objects whose state got materialized but didn't change thereafter. ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: Fixing Broken Objects?
Michel Pelletier wrote: However your question of loading code from a remote location is valid, and another Python remote object system (Pyro) that can be configured, with some restrictions, to load code over the network: http://pyro.sourceforge.net/ in particular, see the section on mobile code: http://pyro.sourceforge.net/manual/7-features.html#mobile Thank you, that's exactly what I was looking for. Chris ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
