[Zope-dev] Zope threads on 2.7.0, python 2.3.3
Hi, I am hitting my head agains wall - and witnessing strange behaviour, where after time most of load focuses only to one thread. I am not sure what is the cause of this, but so far we have seen different things before load has concentrated to one single thread. After reading thread: http://www.gossamer-threads.com/lists/zope/dev/24230?page=last I started to wonder whether that signalling behaviour is also a cause for our problems - since we are running Debina linux without NPTL ( Linux 2.4.26 #1 SMP Thu Apr 22 11:16:14 EEST 2004 i686 unknown ). When zope is started, it nicely starts multiple threads and when tested with ab - each thread gets their share of the work. We saw in event log previously exceptions that were never caught - and thought if those were the source of the problem. Eventhough we added ugly try catch code around this problematic code - we still notice load concentration to this one thread. I will try to go and look with gdb those no longer working threads - but would like to know if anyone could give any pointers on what to look for or what to check. -huima ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug or feature?
Paul Winkler wrote: On Mon, May 24, 2004 at 09:50:31AM +0200, Martijn Faassen wrote: Yup, it's the help system. This is very odd. Did you see the message I sent to formulator-dev a few days ago? No, sorry, just taking a look at it. I spent some time tracing the source of the ReadOnlyErrors and was *sure* it was coming from initializeFieldForm. Today I am trying again and it seems that indeed, the calls to registerHelp and registerFieldHelp are the cause, and initializeFieldForm is not writing to the ZODB after all. I wonder what the heck I was doing differently. Sigh. Hmm. How do you detect the cause anyway? Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug or feature?
On Tue, May 25, 2004 at 04:13:18PM +0200, Martijn Faassen wrote: Paul Winkler wrote: On Mon, May 24, 2004 at 09:50:31AM +0200, Martijn Faassen wrote: Yup, it's the help system. This is very odd. Did you see the message I sent to formulator-dev a few days ago? No, sorry, just taking a look at it. I spent some time tracing the source of the ReadOnlyErrors and was *sure* it was coming from initializeFieldForm. Today I am trying again and it seems that indeed, the calls to registerHelp and registerFieldHelp are the cause, and initializeFieldForm is not writing to the ZODB after all. I wonder what the heck I was doing differently. Sigh. Hmm. How do you detect the cause anyway? just commenting out various stuff in initialize() and restarting zope to see if I get ReadOnlyError. This is what is puzzling; if I use the same technique today, the error seems clearly related to the registerHelp and registerFieldHelp calls... but last week I concluded that those were not the problem, and I don't recall what made me think that. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug or feature?
Paul Winkler wrote: On Mon, May 24, 2004 at 06:55:02PM +0200, Dieter Maurer wrote: Content-Description: message body and .signature Martijn Faassen wrote at 2004-5-24 09:50 +0200: ... I know this has been reported before but I haven't looked into it yet. I'm wondering how to handle Formulator upgrades though -- just checking if you've been registered in a previous startup is not enough if the help text changes between releases, is it? It's been a long time since I worked with the Zope help system.. I attach my previous patch. It may no longer work, though. It doesn't apply cleanly to Formulator 1.6.2 (the one that ships with plone 2.0-final). But it's easy to apply the rejects by hand. Updated patch attached, can you verify that this does what your patch intended? I applied this, but it doesn't solve the ReadOnlyError :-( I'm not sure whether the patch ever could've worked. Firstly, the rich comparison operations were never called; I think perhaps due to some limitation in ExtenionClass. __cmp__ is called, but instead I just made the method explicit. Possibly this is better in Zope 2.7; I'm still testing with 2.6.4. Then there's another problem; I think FieldHelpTopic is basically compared before Formulator (or the help system?) is fully initialized, so no groups or fields are found to compare (causing it to always succeed). I'll try to revise the strategy to fix this problem.. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
Tim Peters wrote: [Martijn Faassen] ... I'm not sure whether the patch ever could've worked. Firstly, the rich comparison operations were never called; I think perhaps due to some limitation in ExtenionClass. ExtensionClass doesn't play well with many newer Python class features. Rich comparisons are one of them. That's what I figured, thanks Tim for confirming it. Anyway, I think I fixed at least the matter for field help topics in a much simpler way, though I don't know whether other sections don't do ZODB writes either on startup. If it's in the normal Zope help registration then we have a bigger problem. :) The fix is in the latest Formulator CVS; Paul, please test it if you can and let me know if you still see the untowards behavior. Regards, Martijn ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
On Tue, 2004-05-25 at 12:48, Martijn Faassen wrote: Anyway, I think I fixed at least the matter for field help topics in a much simpler way, though I don't know whether other sections don't do ZODB writes either on startup. If it's in the normal Zope help registration then we have a bigger problem. :) DEATH TO THE HELP SYSTEM! - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope threads on 2.7.0, python 2.3.3
Heimo Laukkanen wrote at 2004-5-25 14:54 +0300: ... witnessing strange behaviour, where after time most of load focuses only to one thread. ... After reading thread: http://www.gossamer-threads.com/lists/zope/dev/24230?page=last I started to wonder whether that signalling behaviour is also a cause for our problems - since we are running Debina linux without NPTL ( Linux 2.4.26 #1 SMP Thu Apr 22 11:16:14 EEST 2004 i686 unknown ). I would be surprised... What does Control_Panel -- Debug information tells you about the use of your connections (at the bottom of the page)? -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Explicit Acquisition within Acquistion.Implicit derived class
Wyatt Anderson wrote at 2004-5-24 13:01 -0400: In a Python Product class, named TestAcquisition, I derive from Folder (which in turn gives me Acquistion.Implicit). This class defines an attribute which can dynamically be set to be acquired from nested TestAcquisition objects. In order to do this I test whether the attribute can be acquired (by checking to see if aq_parent has the attribute). In the top-level TestAcquisition object, this attribute should not be acquirable, and thus it will set it self.attribute = 1 In sub-objects of TestAcquisition, by default the attribute will be acquired (which I test and do in manage_afterAdd) by setting self.aq_explicit.attribute = Attribute.Acquired Acquisition (and explicit acquisition) works differently from what you expect. If you have an acquisition wrapped obj, then obj.aq_explicit rewraps obj into an explicit acquisition wrapper. self.aq_explicit.XXX = ... has the same effect as self.XXX = ... as assignment *always* effects the base object. Only lookup works differently for implicit and explicit acquisition wrappers. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
Chris McDonough wrote: DEATH TO THE HELP SYSTEM! I'm sure I remember reading that about the OLD help system ;-) Chris - Zope.org anyone? -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Announce: May Zope Bug Day!
Hi All, I'm pleased to announce the first a series of monthly bug days: What? Zope Bug Day working from http://collector.zope.org/Zope When? Friday 28th May - from when you wake up until when the day ends! Where? #zope-dev on irc.zope.org Come along and help if you've got a chance! Read this if you want to know how: http://dev.zope.org/CVS/BugDays cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
On Tue, 2004-05-25 at 14:45, Chris Withers wrote: Chris McDonough wrote: DEATH TO THE HELP SYSTEM! I'm sure I remember reading that about the OLD help system ;-) This *is* the old help system. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
On Tue, May 25, 2004 at 06:48:56PM +0200, Martijn Faassen wrote: Tim Peters wrote: [Martijn Faassen] ... I'm not sure whether the patch ever could've worked. Firstly, the rich comparison operations were never called; I think perhaps due to some limitation in ExtenionClass. ExtensionClass doesn't play well with many newer Python class features. Rich comparisons are one of them. That's what I figured, thanks Tim for confirming it. Anyway, I think I fixed at least the matter for field help topics in a much simpler way, though I don't know whether other sections don't do ZODB writes either on startup. If it's in the normal Zope help registration then we have a bigger problem. :) The fix is in the latest Formulator CVS; Paul, please test it if you can and let me know if you still see the untowards behavior. It seems good, thanks!! Now I've discovered a few more products that are read-only-incompatible :-( CMFCalendar and (eek) CMFCore. I'll look into those and see if a similar fix will help. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Zope threads on 2.7.0, python 2.3.3
On Tue, 25 May 2004 20:21:21 +0200, Dieter Maurer [EMAIL PROTECTED] wrote: I would be surprised... What does Control_Panel -- Debug information tells you about the use of your connections (at the bottom of the page)? At the moment it said that only one opened connection and others were none. I have no terminal access to the machines at the moment to check how threads are behing at the moment. However I did try to look at the processes with gdb - however I need to do more reading on gdb manual to provide more info beyond just backtrace on the process status. I'm not familiar on Python internals so any pointers on what to expect or what not to expect would be appreciated. Below is first output from ps -auxfww | grep portaali where portaali is the name of the user who owns the processes. Below we will see that each thread has started at the same time - but thread with pid 14553 has done most of the work. And while putting more load and looking at top - that is also the only thread that gets any percentage of process time. I have no information on when this has happened since event log does not tell anything peculiar. portaali 14530 0.0 0.2 6116 4432 ?S12:12 0:00 /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/portaali/ ContentManagement-1.0/zope-2.7.0/lib/python/zdaemon/zdrun.py -S /opt/portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/ zopeschema.xml -b 10 -d -f -s /usr/local/portaali/zope_instance/var/zopectlsock -x 0,2 -z /usr/local/portaali/zope_instance /usr/local/portaali/zope_instance/bin/runzope portaali 14531 0.2 16.6 365220 345168 ? S12:12 0:46 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14532 0.0 16.6 365220 345168 ? S12:12 0:00 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14553 13.0 16.6 365220 345168 ? S12:12 34:40 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14552 0.1 16.6 365220 345168 ? S12:12 0:18 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14548 0.8 16.6 365220 345168 ? S12:12 2:16 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14545 0.1 16.6 365220 345168 ? S12:12 0:17 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf portaali 14547 0.2 16.6 365220 345168 ? S12:12 0:36 \_ /opt/portaali/ContentManagement-1.0/python-2.3.3/bin/python /opt/ portaali/ContentManagement-1.0/zope-2.7.0/lib/python/Zope/Startup/run.py -C /usr/local/portaali/zope_instance/etc/zope.conf And then backtrace with gdb from those quiet threads ~# gdb program 14552 backtrace #0 0x4007b87e in sigsuspend () from /lib/libc.so.6 #1 0x4001e879 in __pthread_wait_for_restart_signal () from /lib/libpthread.so.0 #2 0x4001fee1 in sem_wait@@GLIBC_2.1 () from /lib/libpthread.so.0 #3 0x080c4830 in PyThread_acquire_lock (lock=0x861a908, waitflag=1) at Python/thread_pthread.h:406 #4 0x080c76c8 in lock_PyThread_acquire_lock (self=0x417c4840, args=0x4017002c) at ./Modules/threadmodule.c:63 #5 0x080e0e3f in PyCFunction_Call (func=0x41c7c7cc, arg=0x4017002c, kw=0x0) at Objects/methodobject.c:73 #6 0x080a3beb in call_function (pp_stack=0xbe7fb904, oparg=0) at Python/ceval.c:3439 #7 0x080a22f6 in eval_frame (f=0x9b5ea24) at Python/ceval.c:2116 #8 0x080a31e2 in PyEval_EvalCodeEx (co=0x40442c20, globals=0x40445acc, locals=0x0, args=0x8bdb024, argcount=1, kws=0x8bdb028, kwcount=1, defs=0x4047eb18, defcount=5, closure=0x0) at Python/ceval.c:2663 #9 0x080a55ec in fast_function (func=0x404a9924, pp_stack=0xbe7fba94, n=3, na=1, nk=1) at Python/ceval.c:3529 #10 0x080a3c71 in call_function (pp_stack=0xbe7fba94, oparg=256) at Python/ceval.c:3458 Another thread ( 14548 ) #0 0x401153c4 in read () from /lib/libc.so.6 #1 0x40029ae0 in __DTOR_END__ () from /lib/libpthread.so.0 #2 0x412474f0 in nttrd () from /opt/portaali/comp/oracle/lib/libclntsh.so.9.0 #3 0x410fdcf8 in nsprecv () from /opt/portaali/comp/oracle/lib/libclntsh.so.9.0 #4 0x41101ac5 in nsrdr () from /opt/portaali/comp/oracle/lib/libclntsh.so.9.0 #5
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
On Tue, 2004-05-25 at 15:51, Chris Withers wrote: Chris McDonough wrote: I'm sure I remember reading that about the OLD help system ;-) This *is* the old help system. Oh? What was the on before it then? I'm afraid you've invented an earlier help system in your mind. - C ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Read-only root database doesn't work ... bug orfeature?
On Tue, May 25, 2004 at 03:46:16PM -0400, Paul Winkler wrote: Now I've discovered a few more products that are read-only-incompatible :-( CMFCalendar and (eek) CMFCore. I'll look into those and see if a similar fix will help. False alarm, apparently caused by me moving my product directories around while testing. registerHelp checks the directory mod time and returns immediately unless it's more recent than the last registration time. My moving stuff around triggered that. So after one startup with read-only off, I restarted with read-only on and presto, all my products work. Whew! -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
