Re: [ZODB-Dev] Re: KeyError / POSKeyError
The broken object is a 1gb plone instance. Which is what I'm trying to recover. -Tim On 3/26/07, Dieter Maurer [EMAIL PROTECTED] wrote: Tim Tisdall wrote at 2007-3-23 20:08 -0400: ... broken objects ... Another possibility would be to try to delete the broken object(s). There is a HowTo around (on zopelabs, I think) on how to delete objects the inofficial way (in cases when the standard way is not workable, e.g. for broken objects). -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Blob test failure on FreeBSD
Gary Poster wrote: FWIW, It smells more like a 64 bit vs. 32 bit machine/OS question, but that's just on the basis of fly-by-diagnosis. :-) Yep, looks like it. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Blob test failure on FreeBSD
Christian Theune wrote: Who's the owner of the machine? You might know him. A guy by the name of Thomas Lotze. :) -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Blob test failure on FreeBSD
Am Dienstag, den 27.03.2007, 12:44 -0400 schrieb Benji York: Christian Theune wrote: Who's the owner of the machine? You might know him. A guy by the name of Thomas Lotze. :) Nope. That's the buildbot that has been offline for ages. ;) The failing one is 'tmiddleton'. :) -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Blob test failure on FreeBSD
Christian Theune wrote: Am Dienstag, den 27.03.2007, 12:44 -0400 schrieb Benji York: Christian Theune wrote: Who's the owner of the machine? You might know him. A guy by the name of Thomas Lotze. :) Nope. That's the buildbot that has been offline for ages. ;) The failing one is 'tmiddleton'. :) Indeed! I read the wrong column. I guess I need to ping Thomas about restarting his slave, or just remove it. To answer your original question: that machine is 64-bit. tmiddleton is Tim Middleton ([EMAIL PROTECTED]). I've copied him on this message. -- Benji York Senior Software Engineer Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: KeyError / POSKeyError
Tim Tisdall wrote at 2007-3-27 09:17 -0400: The broken object is a 1gb plone instance. Which is what I'm trying to recover. You may try to find the (non broken) persistent subobjects of the broken objects and relink them to a new object. Then you can delete the broken object. Whether you have a chance to succeed depends on how badly the content is destroyed. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: KeyError / POSKeyError
When I try to access any part of the plone instance I'm given the same exceptions I mentioned before. I can't seem to access any subobjects. Am I doomed? -Tim On 3/27/07, Dieter Maurer [EMAIL PROTECTED] wrote: Tim Tisdall wrote at 2007-3-27 09:17 -0400: The broken object is a 1gb plone instance. Which is what I'm trying to recover. You may try to find the (non broken) persistent subobjects of the broken objects and relink them to a new object. Then you can delete the broken object. Whether you have a chance to succeed depends on how badly the content is destroyed. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: KeyError / POSKeyError
On Tue, Mar 27, 2007 at 02:59:51PM -0400, Tim Tisdall wrote: When I try to access any part of the plone instance I'm given the same exceptions I mentioned before. I can't seem to access any subobjects. Am I doomed? Are you getting these exceptions via the ZMI? You should also try exploring in zopectl debug. That might help you determine whether the plone instance itself is broken, or some vital sub-object. As long as you don't call transaction.commit() carelessly, this is safe. more here: http://docs.neuroinf.de/programming-plone/debug#1-6 -- Paul Winkler http://www.slinkp.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: KeyError / POSKeyError
That's what I've been using... the zopectl debug. It basically says that there is a KeyError: 'fammed-old' when I try to access a part of the object. The traceback seems to indicate that it's trying to connect to that object but isn't able to. I can get a list via dir(obj), but that seems to be about it. -Tim On 3/27/07, Paul Winkler [EMAIL PROTECTED] wrote: On Tue, Mar 27, 2007 at 02:59:51PM -0400, Tim Tisdall wrote: When I try to access any part of the plone instance I'm given the same exceptions I mentioned before. I can't seem to access any subobjects. Am I doomed? Are you getting these exceptions via the ZMI? You should also try exploring in zopectl debug. That might help you determine whether the plone instance itself is broken, or some vital sub-object. As long as you don't call transaction.commit() carelessly, this is safe. more here: http://docs.neuroinf.de/programming-plone/debug#1-6 -- Paul Winkler http://www.slinkp.com ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: KeyError / POSKeyError
Maybe I missed mentioning it, but if there was no fammed-old object then it would give a KeyError: 'fammed-old'. If I created a new plone instance in the spot where it was expecting, it would instead give me a POSKeyError. I really don't know anything about Python... how do I create a persistent object? -Tim On 3/27/07, Dieter Maurer [EMAIL PROTECTED] wrote: Tim Tisdall wrote at 2007-3-27 15:31 -0400: That's what I've been using... the zopectl debug. It basically says that there is a KeyError: 'fammed-old' when I try to access a part of the object. The traceback seems to indicate that it's trying to connect to that object but isn't able to. But, that should not result in a KeyError (but something different). Once, you have also spoken about a POSKeyError. This POSKeyError may be more tractable. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: [Zope3-dev] Re: Community opinion about search+filter
Jim Fulton wrote at 2007-3-26 15:55 -0400: ... On Mar 26, 2007, at 3:28 PM, Dieter Maurer wrote: Jim Fulton wrote at 2007-3-25 09:53 -0400: On Mar 25, 2007, at 3:01 AM, Adam Groszer wrote: MF I think one of the main limitations of the current catalog (and MF hurry.query) is efficient support for sorting and batching the query MF results. The Zope 3 catalog returns all matching results, which can then MF be sorted and batched. This will stop being scalable for large MF collections. A relational database is able to do this internally, and is MF potentially able to use optimizations there. What evidence to you have to support this assertion? We did some literature search on this a few years ago and found no special trick to avoid sorting costs. I know of 2 approaches to reducing sort cost: 1. Sort your results based on the primary key and therefore, pick your primary key to match your sort results. In terms of the Zope catalog framework, the primary keys are the document IDs, which are traditionally chosen randomly. You can pick your primary keys based on a desired sort order instead. A variation on this theme is to use multiple sets of document ids, storing multiple sets of ids in each index. Of course, this approach doesn't help with something like relevance ranks. 2. Use an N-best algorithm. If N is the size of the batch and M is the corpus size, then this is O(M*ln(N)) rather than O(M*ln(M)) which is a significant improvement if N M, but still quite expensive. The major costs in sorting are usually not the log(n) but the very high linear costs fetching the sort keys (although for huge n, we will reach the asymptotic limits). Right. The problem is the N not the log(N). :) Under normal conditions, a relational database can be far more efficient to fetch values either from index structures or the data records than Zope -- as * its data representation is much more compact * it often supports direct access * the server itself can access and process all data. With the ZODB, the data is hidden in pickles (less compact), there is no direct access (instead the complete pickle need to be decoded) The catalog sort index mechanism uses the un-index data structure in the sort index to get sort keys. This is a pretty compact data structure. The data usually is in IOBuckets which contain 45 values on the average. In a corresponding relational index structure, you could have several hundreds of values. and all operations are done in the client (rather than in the server). Which is often fine if the desired data are in the client cache. It avoids making the storage a bottleneck. Yes, but the if is important. Quite often, some operations flush almost all objects from the cache (partly because the cache is controlled by the number of objects and not by their size) and after that, filling the cache again takes ages. Moreover, a relational database can (and usually does) use caching as well. It is not restricted to a cliend side only technique. I know that most relational database backends have a different architecture than ZEO: use one process (or at least one thread) per connection such that several activities can interleave the high IO wait times. The one thread ZEO architecture must take more care not to become the bottleneck. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Blob test failure on FreeBSD
Ok, I'll try to figure out whether this happens on 64bit linux tomorrow. After that I can try 32bit Freebsd, otherwise I have to guess. Christian Am Dienstag, den 27.03.2007, 14:16 -0400 schrieb Benji York: Christian Theune wrote: Am Dienstag, den 27.03.2007, 12:44 -0400 schrieb Benji York: Christian Theune wrote: Who's the owner of the machine? You might know him. A guy by the name of Thomas Lotze. :) Nope. That's the buildbot that has been offline for ages. ;) The failing one is 'tmiddleton'. :) Indeed! I read the wrong column. I guess I need to ping Thomas about restarting his slave, or just remove it. To answer your original question: that machine is 64-bit. tmiddleton is Tim Middleton ([EMAIL PROTECTED]). I've copied him on this message. -- gocept gmbh co. kg - forsterstraße 29 - 06112 halle/saale - germany www.gocept.com - [EMAIL PROTECTED] - phone +49 345 122 9889 7 - fax +49 345 122 9889 1 - zope and plone consulting and development signature.asc Description: Dies ist ein digital signierter Nachrichtenteil ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
