Re: [Zope-dev] Core dumps - How to find out the bug?
Dario - I had faced similar problems ... try the following thread for help http://aspn.activestate.com/ASPN/Mail/Message/zope-Dev/940742 on how to trap the error on a running process. For linux, you need the command strace ... here is one url http://www.pugcentral.org/howto/truss.htm Once you have the core file, you should invoke gdb like this ... gdb program core where program is the path to the python executable and core is the core file. Then, type info threads to see which thread and where the problem occured in zope/python. - j ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Temporary Storage + Sessions + Versions: How?
Chris and Joachim - Maybe this is too simplistic of an approach ... but why not have the session data manager automatically create a new, session data container (temporary storage) for a given zope version. The session data manager can be optimized for the case of no version and then when and if needed create temporary storages on demand for a particular version. - j ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Patch acceptance. What about this one?
Ok .. here's the collector url: http://collector.zope.org/Zope/108 - j At Fri, 28 Dec 2001 00:14:21 -0500, Chris McDonough wrote: At the time, I hadn't received any feedback (however, I'm not blaming anyone). I also never posted this to the collector before. Should one of us post this? It would be appreciated, Joseph. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Patch acceptance. What about this one?
Adam - At Thu, 27 Dec 2001 21:42:10 -0500, Adam Manock wrote: It was written against 2.4.1 I run 2.4.3. It would have been really nice if it had been included in Zope 2.4.next.. Hmm, has anything else changed that I need to be aware of before applying the patch? FYI. I have been using this patch as is up and to including Zope 2.4.3 without any troubles. I can't help but wonder why something this simple and useful was not just included in Zope 2.4.next? It would have saved me and all the poor fools like me so much duplicated effort... At the time, I hadn't received any feedback (however, I'm not blaming anyone). I also never posted this to the collector before. Should one of us post this? Just to be safe ... You shouldn't use this entire patch unless your server is behind apache or a proxy server and best if protected by a firewall. It could open a potential security leak if you use the domains field for authentication and the zope server is not protected by apache. - joe n. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Patch acceptance. What about this one?
At Fri, 28 Dec 2001 00:14:21 -0500, Chris McDonough wrote: At the time, I hadn't received any feedback (however, I'm not blaming anyone). I also never posted this to the collector before. Should one of us post this? It would be appreciated, Joseph. ok ... I can post this afternoon. Just to be safe ... You shouldn't use this entire patch unless your server is behind apache or a proxy server and best if protected by a firewall. It could open a potential security leak if you use the domains field for authentication and the zope server is not protected by apache. Is the issue that the X-Forwarded-For header controls the domain setting? yes ... everyone should probably not use this patch right-out-of-the-box. - j ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: Zope 2.4 crashes -- possible fix identified, other solutions also suggested
Matt - If possible, I would prefer to use a source Python 2.1.2 release with a source zope 2.4.4 ? bugfix release (or create my own from the 2.4 cvs branch) once the fixes are complete. We do not want to put a 2.5 release in production at this time. Thanks for the update. regards, - j At Wed, 19 Dec 2001 09:25:08 -0500, Matthew T. Kromer wrote: The ExtensionClass.h patch has NOT been merged into the branches and trunk yet pending further review. just my 2 cents, but we have been using this in production for about 1 week or more without any troubles. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
Matt - Ok, I installed everything and the system is running fine (or no worse). However, we still faced one restart so far. I have included the debug information below. This looks similiar to the problem report on sourceforge: http://sourceforge.net/tracker/?func=detailatid=105470aid=471942group_id=5470 I posted a comment to see if they have any updates. One question ... does anyone every malloc a plain ClassExtension object? It seems that every CE-based object has their own struct typedef. If so, then I think yesterday's patch problaby won't do any harm but won't help either. The current running process is being monitored by truss so I will be able to get at least one more core dump (if we get one). I won't be able to get any more information until tomorrow. Any other ideas? Thanks for your help. - joe . (gdb) info threads 17 Thread 10 0xef5b9810 in _lwp_sema_wait () 16 Thread 9 0xef647cac in _swtch () 15 Thread 8 0xef5b9810 in _lwp_sema_wait () 14 Thread 7 (LWP 5) 0xcaeb50 in ?? () 13 Thread 6 0xef647cac in _swtch () 12 Thread 5 0xef5b9810 in _lwp_sema_wait () 11 Thread 4 0xef647cac in _swtch () 10 Thread 3 0xef647cac in _swtch () 9 Thread 2 (LWP 2) 0xef5b9958 in _signotifywait () 8 Thread 1 (LWP 6) 0xef5b7488 in _poll () 7 LWP8 0xef5b6a24 in door_restart () 6 LWP6 0xef5b7488 in _poll () 5 LWP5 0xcaeb50 in ?? () 4 LWP4 0xef5b9810 in _lwp_sema_wait () 3 LWP3 0xef5b9810 in _lwp_sema_wait () 2 LWP2 0xef5b9958 in _signotifywait () * 1 LWP1 0xef5b9810 in _lwp_sema_wait () (gdb) thread 14 [Switching to Thread 7 (LWP 5)] #0 0xcaeb50 in ?? () (gdb) where #0 0xcaeb50 in ?? () #1 0x516bc in collect (young=0x13dec8, old=0x13ded4) at ./Modules/gcmodule.c:379 #2 0x51984 in collect_generations () at ./Modules/gcmodule.c:484 #3 0x519fc in _PyGC_Insert (op=0xecf7d4) at ./Modules/gcmodule.c:507 #4 0x664ec in PyMethod_New (func=0x3f796c, self=0x11c0d44, class=0x3c7e5c) at Objects/classobject.c:1834 #5 0x63850 in instance_getattr2 (inst=0x11c0d44, name=0x3d5378) at Objects/classobject.c:642 #6 0x63750 in instance_getattr1 (inst=0x11c0d44, name=0x3d5378) at Objects/classobject.c:608 #7 0x63898 in instance_getattr (inst=0x11c0d44, name=0x3d5378) at Objects/classobject.c:656 #8 0x78330 in PyObject_GetAttr (v=0x11c0d44, name=0x3d5378) at Objects/object.c:1052 #9 0x895ec in builtin_hasattr (self=0x0, args=0x12ed944) at Python/bltinmodule.c:886 #10 0x35a44 in call_cfunction (func=0x1609b0, arg=0x12ed944, kw=0x0) at Python/ceval.c:2854 #11 0x33c5c in eval_code2 (co=0x3cbf80, globals=0x1, locals=0x0, args=0x2, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:1948 and so on ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
Matt - Well, your patch seems fine in our testing environment. Unfortunately, we do not see any restarts in the testing environment ... always in production. I had to rebuild our entire software base because we are using other products that use extensions class and they are not included under the main zope installation. It caused a bus error the first time (with only running wo_pcgi.py). As I mentioned in my prior e-mail, I modified the patch slightly to exactly match the struct in Python's object.h. Please review this patch. I will apply the patch in production tomorrow morning, 12/13, (Japan Standard Time or GMT+9) and monitor the system. If zope does not restart during the day, then I think you have fixed the problem. I'm using Zope 2.4.3 and Python 2.1.1 with pymalloc disabled on the solaris platform. thanks and regards, - joe n. p.s. I looked **briefly** at the Zope 2.5 source and this patch will not be compatible since there doesn't seem to be a standard among the different extension classes on whether to include or not include the COUNT_ALLOCS define. The cAccessControl class seems to be the exception. *** ExtensionClass.h.bakFri Nov 16 10:37:11 2001 --- ExtensionClass.hWed Dec 12 15:10:03 2001 *** *** 136,154 PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ! /* More standard operations (at end for binary compatibility) */ hashfunc tp_hash; ternaryfunc tp_call; reprfunc tp_str; getattrofunc tp_getattro; setattrofunc tp_setattro; ! /* Space for future expansion */ ! long tp_xxx3; ! long tp_xxx4; char *tp_doc; /* Documentation string */ #ifdef COUNT_ALLOCS /* these must be last */ int tp_alloc; --- 136,169 PySequenceMethods *tp_as_sequence; PyMappingMethods *tp_as_mapping; ! /* More standard operations (here for binary compatibility) */ hashfunc tp_hash; ternaryfunc tp_call; reprfunc tp_str; getattrofunc tp_getattro; setattrofunc tp_setattro; ! ! /* Functions to access object as input/output buffer */ ! PyBufferProcs *tp_as_buffer; ! ! /* Flags to define presence of optional/expanded features */ ! long tp_flags; char *tp_doc; /* Documentation string */ + /* call function for all accessible objects */ + traverseproc tp_traverse; + + /* delete references to contained objects */ + inquiry tp_clear; + + /* rich comparisons */ + richcmpfunc tp_richcompare; + + /* weak reference enabler */ + long tp_weaklistoffset; + #ifdef COUNT_ALLOCS /* these must be last */ int tp_alloc; *** *** 302,308 { PyExtensionClassCAPI-Export(D,N,T); } /* Convert a method list to a method chain. */ ! #define METHOD_CHAIN(DEF) { DEF, NULL } /* The following macro checks whether a type is an extension class: */ #define PyExtensionClass_Check(TYPE) \ --- 317,330 { PyExtensionClassCAPI-Export(D,N,T); } /* Convert a method list to a method chain. */ ! /* MTK -- make it pad the type structure out -- presumes only use is in ! ** type structure initialization ! */ ! #ifdef COUNT_ALLOCS ! #define METHOD_CHAIN(DEF) 0,0,0,0,0,0,0,0,{ DEF, NULL } ! #else ! #define METHOD_CHAIN(DEF) 0,0,0,0,{ DEF, NULL } ! #endif /* The following macro checks whether a type is an extension class: */ #define PyExtensionClass_Check(TYPE) \ *** *** 336,342 #define PURE_MIXIN_CLASS(NAME,DOC,METHODS) \ static PyExtensionClass NAME ## Type = { PyObject_HEAD_INIT(NULL) \ 0, # NAME, sizeof(PyPureMixinObject), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, \ ! 0, 0, 0, 0, 0, 0, 0, DOC, {METHODS, NULL}, \ EXTENSIONCLASS_BASICNEW_FLAG} /* The following macros provide limited access to extension-class --- 358,364 #define PURE_MIXIN_CLASS(NAME,DOC,METHODS) \ static PyExtensionClass NAME ## Type = { PyObject_HEAD_INIT(NULL) \ 0, # NAME, sizeof(PyPureMixinObject), 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, \ ! 0, 0, 0, 0, 0, 0, 0, DOC, METHOD_CHAIN(METHODS), \ EXTENSIONCLASS_BASICNEW_FLAG} /* The following macros provide limited access to extension-class ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
At Tue, 11 Dec 2001 10:42:46 -0500, Matthew T. Kromer wrote: #0 0xef5b9810 in _lwp_sema_wait () (gdb) info threads 19 Thread 10 0xef5b9810 in _lwp_sema_wait () 18 Thread 9 0xef5b9810 in _lwp_sema_wait () 17 Thread 8 0xef5b9810 in _lwp_sema_wait () 16 Thread 7 (LWP 8) subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 Aha! See? Matthew - I performed the operations that you recommended and here are the results (see below). The problem seems to be with the value of the tp_traverse field. I am not aware of any T type python object. I'm wondering if this is an extension class type (just a guess). I searched through all of the *.c files in zope, etc. but I as not able to find any type of name T. I also ran across a bug posting at sourceforge ... http://sourceforge.net/tracker/?func=detailatid=105470aid=471942group_id=5470 This bug report looks very similiar. - j #0 0xef5b9810 in _lwp_sema_wait () (gdb) info threads 19 Thread 10 0xef5b9810 in _lwp_sema_wait () 18 Thread 9 0xef5b9810 in _lwp_sema_wait () 17 Thread 8 0xef5b9810 in _lwp_sema_wait () 16 Thread 7 (LWP 8) subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 15 Thread 6 0xef647cac in _swtch () 14 Thread 5 0xef5b9810 in _lwp_sema_wait () 13 Thread 4 (LWP 0) 0xef647b7c in _swtch () 12 Thread 3 0xef647cac in _swtch () 11 Thread 2 (LWP 2) 0xef5b9958 in _signotifywait () 10 Thread 1 (LWP 6) 0xef5b7488 in _poll () 9 LWP9 0xef5b6a24 in door_restart () 8 LWP8 subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 7 LWP7 0xef5b9810 in _lwp_sema_wait () 6 LWP6 0xef5b7488 in _poll () 5 LWP5 0xef5b9814 in _lwp_sema_wait () 4 LWP4 0xef5b9810 in _lwp_sema_wait () 3 LWP3 0xef5b9810 in _lwp_sema_wait () 2 LWP2 0xef5b9958 in _signotifywait () * 1 LWP1 0xef5b9810 in _lwp_sema_wait () (gdb) thread 16 [Switching to Thread 7 (LWP 8)] #0 subtract_refs (containers=0x13dec8) at ./Modules/gcmodule.c:166 ./Modules/gcmodule.c:166: No such file or directory. (gdb) print *((PyObject *) gc)-ob_type $1 = {ob_refcnt = 18213696, ob_type = 0x2d70b0, ob_size = 0, tp_name = 0x1 T, tp_basicsize = 1328272, tp_itemsize = 4156348, tp_dealloc = 0x125865c, tp_print = 0x3c1b04, tp_getattr = 0, tp_setattr = 0, tp_compare = 0x29, tp_repr = 0x3adeb0, tp_as_number = 0xf66198, tp_as_sequence = 0xdf3fa0, tp_as_mapping = 0x0, tp_hash = 0x1, tp_call = 0x144490 PyMethod_Type, tp_str = 0x3f0a1c, tp_getattro = 0x125865c, tp_setattro = 0x3c1b04, tp_as_buffer = 0x0, tp_flags = 158561192, tp_doc = 0x29 , tp_traverse = 0x4c4f4144, tp_clear = 0xd908c0, tp_richcompare = 0x1151300, tp_weaklistoffset = 0} (gdb) print *((PyObject *) 0x2d70b0)-ob_type $2 = {ob_refcnt = 2977968, ob_type = 0xff5b80, ob_size = 0, tp_name = 0x1 T, tp_basicsize = 1328272, tp_itemsize = 4155228, tp_dealloc = 0x125865c, tp_print = 0x3c1b04, tp_getattr = 0, tp_setattr = 0, tp_compare = 0x29, tp_repr = 0, tp_as_number = 0x1212b48, tp_as_sequence = 0xbf8d30, tp_as_mapping = 0x, tp_hash = 0x1, tp_call = 0x144490 PyMethod_Type, tp_str = 0x4ab2cc, tp_getattro = 0x1089d5c, tp_setattro = 0x4ab30c, tp_as_buffer = 0x0, tp_flags = 0, tp_doc = 0x29 , tp_traverse = 0, tp_clear = 0x122d140, tp_richcompare = 0x11ccd70, tp_weaklistoffset = -1} gdb) x 0x4c4f4144 0x4c4f4144: Cannot access memory at address 0x4c4f4144. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] recipe for trapping SIGSEGV and SIGILL signals on solaris
Hello. We are facing zope restarts on the solaris 5.6 platform with zope 2.4.3 and python 2.1.1. I put together a script based some information on an old posting to the apache mailing list. The following shell/perl script allows one to get a core file from a dying zope child process and also allow the zope to restart without any side effects. The script #!/bin/sh PATH=$PATH:/usr/local/bin export PATH cd /tmp for PID in `ps -u zfs -f -o pid,comm,args | fgrep z2.py | cut -d' ' -f1` do export PID truss -f -l -t\!all -S SIGSEGV,SIGILL -p $PID 21 \ | perl -pe 'system(gcore $ENV{'PID'} sleep 5 kill -9 $ENV{'PID'}), exit($ENV{'PID'}) if /(SIGSEGV|SIGILL)/;' done Step 1: modify script to match your environment. Step 2: execute script Step 3: wait for core file to be dumped in /tmp. Step 4: analyze with gdb where $PID is the pid of the dumped process #bash gdb /path/to/bin/python /tmp/core.$PID #0 0xef5b9810 in _lwp_sema_wait () (gdb) where #0 0xef5b9810 in _lwp_sema_wait () #1 0xef647ea0 in _park () #2 0xef647b84 in _swtch () #3 0xef6468a4 in cond_wait () #4 0xef6467c8 in _ti_pthread_cond_wait () #5 0x50220 in PyThread_acquire_lock (lock=0xd9d878, waitflag=1) at Python/thread_pthread.h:313 #6 0x51f18 in lock_PyThread_acquire_lock (self=0xda39b8, args=0x0) at ./Modules/threadmodule.c:67 #7 0x35db4 in fast_cfunction (func=0xda39b8, pp_stack=0xed40f828, na=0) at Python/ceval.c:2994 #8 0x33ca0 in eval_code2 (co=0x267848, globals=0x51ec4, locals=0x0, args=0x0, argcount=0, kws=0x0, kwcount=0, defs=0x0, defcount=0, closure=0x0) at Python/ceval.c:1951 : : It seems that we are facing trouble due to the thread library on solaris (unless the truss command has introduced a side-effect). Anyone else facing similiar troubles? or maybe I should post this to a python mailing list. - joe ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] More signal 11 restarts....
Daniel - I have **not** tried this myself yet, but I plan to check this week. Please take a look at the following URL: http://www.humanfactor.com/cgi-bin/cgi-delegate/apache-ML/nh/1998/Oct/0130.html This mail is related to apache but the same analysis might apply to zope. I believe the truss command on the solaris platform is similiar to strace command on the linux platform. I'm facing similar problems on the solaris platform but the restart is occuring maybe 2-3 times per day. regards, - j At Mon, 26 Nov 2001 15:47:30 -0200, Daniel Duclos wrote: I have a zope that is dumping signal 11 every 40 minutes or so. I have tried recompile python 2.1.1 with-threads without-pymalloc, recompile Zope with it, recompile ZPAtterns, recompile and instal MYSQL for Python 0.9.1, upgraded to Zope 2.4.3, all this on a Debian Linux box. Nothing changed... still restarting... Anybody, please, has any ideia on this matter? Please, let me know if there's any relevant info that I forgot to mention abot my case! Thanks in advance!! -- daniel lobato duclos -- [EMAIL PROTECTED] -- http://www.hiperlogica.com.br - ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope ) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Re: core session tracking and zope 2.5 integration
Correction ... http://www.zope.org/Members/natsukashi/index.html http://www.zope.org/Members/natsukashi/index_html is not able to be found on www.zope.org. At Tue, 09 Oct 2001 12:35:08 +0900, http://www.zope.org/Members/natsukashi/Prototypes/CoreSessionUserFolder.tgz or link via zope.org home page http://www.zope.org/Members/natsukashi/ ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: core session tracking and zope 2.5 integration
Chris - Thanks for your response. I had already put together a prototype since my first mail to you on this subject and just completed integration with version 0.9. I have placed a tar bundle at the following url: http://www.zope.org/Members/natsukashi/Prototypes/CoreSessionUserFolder.tgz or link via zope.org home page http://www.zope.org/Members/natsukashi/ with a bundled example (stest_internal.zexp). Please make sure that you read both of the README.txt files. bash$ tar --exclude CVS -cvzf CoreSessionUserFolder.tgz CoreSessionUserFolder CoreSessionUserFolder/ CoreSessionUserFolder/__init__.py CoreSessionUserFolder/User.py CoreSessionUserFolder/images/ CoreSessionUserFolder/images/UserFolder_icon.gif CoreSessionUserFolder/help/ CoreSessionUserFolder/refresh.txt CoreSessionUserFolder/patches/ CoreSessionUserFolder/patches/README.txt CoreSessionUserFolder/patches/CMFCore-CookieCrumbler-p1.txt CoreSessionUserFolder/patches/CoreSessionTracking0-9-p1.txt CoreSessionUserFolder/patches/zope-2.4.1-absolute_url-p1.txt CoreSessionUserFolder/SessionIdManager.py CoreSessionUserFolder/import/ CoreSessionUserFolder/import/README.txt CoreSessionUserFolder/import/stest_internal.zexp It is far from perfect but just a proof of concept. This approach would also benefit from the encrypted user password support (I believe). Please take a look and provide some feedback. thanks, - joe At Sat, 6 Oct 2001 02:04:49 -0400, Chris McDonough wrote: I need to elaborate more on my thoughts but I thought it would be worthwhile to ask if something like this is already in the works before I spend too much investigating how to implement such a acl_user folder. No, nothing like this in the works. It'd be pretty neat to see something like it, at least as proof of concept... ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] RFC: Date property requiers valid date (no more)
Johan - I have done a similiar hotfix by using None - which I think is a better alternative than ''. If you want to see all of the source, download the following Product, http://www.zope.org/Members/natsukashi/Products/CMFPropertyCore, and look inside the file CMFPropertyCore/LinkPropertyManager.py 1) I created my own converter functions and simply call the real function if the value is not none. Here is an excerpt from a file called LinkPropertyManager.py: from OFS.PropertyManager import PropertyManager from ZPublisher import Converters class LinkPropertyManager(PropertyManager): def field2float(v): if not v: return None else: return Converters.field2float(v) def field2int(v): if not v: return None else: return Converters.field2int(v) def field2long(v): if not v: return None else: return Converters.field2long(v) def field2date(v): if not v: return None else: return Converters.field2date(v) def field2link(v): if hasattr(v,'read'): v=v.read() else: v=str(v) return v type_converters = Converters.type_converters type_converters.update({'float' : field2float , 'int' : field2int , 'long' : field2long , 'date' : field2date , 'link' : field2link }) PropertyManager.type_converters = LinkPropertyManager.type_converters 2) Since you are patching the type converter, you might as well patch the manage_propertiesForm. This can be done by hotfixing the OFS.PropertyManager.PropertyManager.manage_propertiesForm value with your own dtml file. Here is a diff between the CMFPropertyCore/dtml/properties.dtml file and zope's properties.dtml file. bash$ diff -c properties.dtml /opt/arseed/tfs-lib/zope/zope-2.4.1/lib/python/OFS/dtml/properties.dtml *** properties.dtml Thu Aug 9 10:41:41 2001 --- /opt/arseed/tfs-lib/zope/zope-2.4.1/lib/python/OFS/dtml/properties.dtml Wed Oct 3 07:49:38 2001 *** *** 58,74 dtml-if type == 'int' input type=text name=dtml-var id:dtml-var type size=35 value=dtml-if hasProperty(id)dtml-var !'%s' % (getProperty(id) or '') html_quote/dtml-if dtml-elif type == 'long' input type=text name=dtml-var id:dtml-var type size=35 value=dtml-if hasProperty(id)dtml-var !('%s' % (getProperty(id) or ''))[:-1] html_quote/dtml-if dtml-elif type in ('float', 'date') input type=text name=dtml-var id:dtml-var type size=35 !value=dtml-var getProperty(id) html_quote null= dtml-elif type=='string' input type=text name=dtml-var id:string size=35 !value=dtml-var getProperty(id) html_quote null= dtml-elif type=='boolean' input type=checkbox name=dtml-var id:boolean size=35 dtml-if getProperty(id)CHECKED/dtml-if --- 58,74 dtml-if type == 'int' input type=text name=dtml-var id:dtml-var type size=35 value=dtml-if hasProperty(id)dtml-var !'%s' % getProperty(id) html_quote/dtml-if dtml-elif type == 'long' input type=text name=dtml-var id:dtml-var type size=35 value=dtml-if hasProperty(id)dtml-var !('%s' % getProperty(id))[:-1] html_quote/dtml-if dtml-elif type in ('float', 'date') input type=text name=dtml-var id:dtml-var type size=35 !value=dtml-var getProperty(id) html_quote dtml-elif type=='string' input type=text name=dtml-var id:string size=35 !value=dtml-var getProperty(id) html_quote dtml-elif type=='boolean' input type=checkbox name=dtml-var id:boolean size=35 dtml-if getProperty(id)CHECKED/dtml-if *** *** 77,83 value=dtml-in getProperty(id)dtml-var sequence-item html_quote /dtml-in dtml-elif type=='text' textarea name=dtml-var id:text rows=6 cols=35dtml-var !getProperty(id) html_quote null=/textarea dtml-elif type=='lines' textarea name=dtml-var id:lines rows=6 cols=35dtml-in getProperty(id)dtml-var sequence-item html_quotedtml-if --- 77,83 value=dtml-in getProperty(id)dtml-var sequence-item html_quote /dtml-in dtml-elif type=='text' textarea name=dtml-var id:text rows=6 cols=35dtml-var !getProperty(id) html_quote/textarea dtml-elif type=='lines' textarea name=dtml-var id:lines rows=6 cols=35dtml-in getProperty(id)dtml-var sequence-item html_quotedtml-if *** *** 91,97 dtml-in getProperty(select_variable) option dtml-if _['sequence-item']==getProperty(id)SELECTED/dtml-if ! dtml-var sequence-item html_quote null= /option /dtml-in /select /div --- 91,97 dtml-in getProperty(select_variable) option dtml-if _['sequence-item']==getProperty(id)SELECTED/dtml-if ! dtml-var sequence-item
[Zope-dev] apache ProxyPass and REMOTE_ADDR -- any further discussion or
consensus? User-Agent: Wanderlust/2.5.8 (Smooth Criminal) SEMI/1.14.3 (Ushinoya) FLIM/1.14.2 (Yagi-Nishiguchi) APEL/10.3 MULE XEmacs/21.4 (patch 1) (Copyleft) (i386-debian-linux) Reply-To: [EMAIL PROTECTED] MIME-Version: 1.0 (generated by SEMI 1.14.3 - Ushinoya) Content-Type: text/plain; charset=US-ASCII Hello. I have put together a patch (see below) which adds the necessary support for performing user authentication based on domain (and logging) if your zope server is hiding behind apache+mod_proxy+mod_proxy_add_forward. I noticed a posting to zope-dev early this year regarding apache ProxyPass and SiteAccess http://aspn.activestate.com/ASPN/Mail/Message/zope-Dev/479449 Has there been any further discussion or consensus on this issue? regards, - joe n. *** Zope-2.4.1-src/ZServer/HTTPServer.pyWed Aug 8 22:04:32 2001 --- zope-2.4.1/ZServer/HTTPServer.pyTue Sep 25 12:01:55 2001 *** *** 294,299 --- 294,315 if value and not env_has(key): env[key]=value env.update(self.env_override) + + # set REMOTE_ADDR_X and REMOTE_HOST_X + if env_has('HTTP_X_FORWARDED_FOR'): + # only fetch last addr -- appended by mod_proxy_add_forward + remote_addr_x = strip(split(env['HTTP_X_FORWARDED_FOR'], ,)[-1]) + if remote_addr_x != '': + env['REMOTE_ADDR_X']=remote_addr_x + # If we're using a resolving logger, try to get the + # remote host from the resolver's cache. + if hasattr(server.logger, 'resolver'): + dns_cache=server.logger.resolver.cache + if dns_cache.has_key(env['REMOTE_ADDR_X']): + remote_host_x=dns_cache[env['REMOTE_ADDR_X']][2] + if remote_host_x is not None: + env['REMOTE_HOST_X']=remote_host_x + return env def continue_request(self, sin, request): *** Zope-2.4.1-src/ZServer/medusa/http_server.pyTue Jul 3 04:45:22 2001 --- zope-2.4.1/ZServer/medusa/http_server.pyTue Sep 25 12:29:08 2001 *** *** 284,291 else: name = t[0] self.channel.server.logger.log ( ! self.channel.addr[0], ' - %s [%s] %s %d %d %s %s\n' % ( name, self.log_date_string (time.time()), --- 284,295 else: name = t[0] + channel_addr=self.get_header('X-Forwarded-For') + if channel_addr: channel_addr = string.strip(string.split(channel_addr, +,)[-1]) + if not channel_addr: channel_addr = self.channel.addr[0] + self.channel.server.logger.log ( ! channel_addr, ' - %s [%s] %s %d %d %s %s\n' % ( name, self.log_date_string (time.time()), *** Zope-2.4.1-src/lib/python/AccessControl/User.py Sat Aug 4 22:49:26 2001 --- zope-2.4.1/lib/python/AccessControl/User.py Tue Sep 25 12:00:54 2001 *** *** 1039,1048 if len(spec) == 1 and spec[0] == '*': return 1 ! if request.has_key('REMOTE_HOST'): host=request['REMOTE_HOST'] ! if request.has_key('REMOTE_ADDR'): addr=request['REMOTE_ADDR'] if not host and not addr: --- 1039,1052 if len(spec) == 1 and spec[0] == '*': return 1 ! if request.has_key('REMOTE_HOST_X'): ! host=request['REMOTE_HOST_X'] ! elif request.has_key('REMOTE_HOST'): host=request['REMOTE_HOST'] ! if request.has_key('REMOTE_ADDR_X'): ! addr=request['REMOTE_ADDR_X'] ! elif request.has_key('REMOTE_ADDR'): addr=request['REMOTE_ADDR'] if not host and not addr: ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] core session tracking and zope 2.5 integration
Chris - I'm not sure of your plans for integrating core session tracking with zope 2.5. This may be obvious to you but I just realized today how I would really like to use core session tracking. I haven't yet investigated the feasability of an implementation. However, it would be nice to have 3 out-of-the-box choices for acl_users folders: - the current acl_users folder - a new acl_users folder with core session tracking support (ram-based storage) - a new acl_users folder with core session tracking support (possibly mounted zeo client storage) The core session tracking based acl_users folders would hide all of the details of installing and setting up the current core session tracking product. By combining this with something like the cookie crumbler, password-less users authenticated only by session key (user name equal to the session key) could be automatically created and expired when the session expires. The session data could also simply hang off of the authenticated user object and default roles (or no roles at all) could be assigned at the user's creation time. The session based acl_users folder would behave the same as the current acl_users folder for users whose expiration time is NONE or less than zero. In essence, it would be really helpful to make the core session tracking product as easy to use (and install) as the REQUEST object. I need to elaborate more on my thoughts but I thought it would be worthwhile to ask if something like this is already in the works before I spend too much investigating how to implement such a acl_user folder. thanks, - joe n. -- [EMAIL PROTECTED] +81-3-3823-5757 2-10-7 Tabata, Kita-ku, Tokyo 114-0014, Japan 〒114-0014 東京都北区田端2丁目10-7 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] RestrictedDTML w/isPrincipiaFolderish: error with CMF 1.1 and Zope 2.4.0
Hello. I'm getting the following error message (see below) while accessing the contents of a folder (http://localhost/cmf/Portal/FolderD/folder_contents) underneath a CMF site instance. I was testing Python 2.1, and CMF 1.1/Zope 2.4.0 (latest cvs checked out versions). I do not have any problem while using CMF 1.1 (latest cvs version), Zope 2.3.3, and Python 1.52. The starting ZODB database is identical in both cases. However, I'm not using the ZDebug product with Zope 2.4.0. I'm just not sure if the issue is due to: - CMF - isPrincipiaFolderish being used in the folder_contents skin or - Zope 2.4.0 - RestrictedDTML bug ? or - Zope 2.3.3 security mechanism bug ? or - bug in my acl setup I feel the following priviledges should be sufficient ... Access contents information List folder contents View to access isPrincipiaFolderish. Any pointers ??? - joe n. TD WIDTH=90% H2Site Error/H2 PAn error was encountered while publishing this resource. /P PSTRONGUnauthorized/STRONG/P You are not authorized to access emisPrincipiaFolderish/em. !-- Traceback (innermost last): File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/ZPublisher/Publish.py, line 223, in publish_module File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/ZPublisher/Publish.py, line 187, in publish File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/ZPublisher/Publish.py, line 171, in publish File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/ZPublisher/mapply.py, line 160, in mapply (Object: RestrictedDTML) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/ZPublisher/Publish.py, line 112, in call_object (Object: RestrictedDTML) File /opt/arseed/tfs-lib/zope/zope-2.4.0/lib/python/Products/CMFCore/FSDTMLMethod.py, line 182, in __call__ (Object: RestrictedDTML) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/DocumentTemplate/DT_String.py, line 544, in __call__ (Object: RestrictedDTML) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/DocumentTemplate/DT_Let.py, line 148, in render (Object: filterString=REQUEST.get( 'folderfilter', '' ) filter=decodeFolderFilter( filterString )) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/DocumentTemplate/DT_In.py, line 661, in renderwb (Object: listFolderContents( filter=filter )) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/DocumentTemplate/DT_Let.py, line 146, in render (Object: obj=_.getitem('sequence-item', 0 ) folderish=isPrincipiaFolderish portalish=_.hasattr( obj, 'isPortalContent' ) and obj.isPortalContent methodID=folderish and '/folder_contents' or ( portalish and '/view' or '' ) getIcon=_.hasattr(obj, 'getIcon') and obj.getIcon() icon=getIcon or _.getattr(obj, 'icon', '')) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/AccessControl/DTML.py, line 101, in guarded_getattr (Object: RestrictedDTML) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/AccessControl/ZopeGuards.py, line 120, in guarded_getattr (Object: DynamicType) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/AccessControl/ZopeGuards.py, line 103, in aq_validate (Object: DynamicType) File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/AccessControl/SecurityManager.py, line 149, in validate File /export/arseed/tfs-lib/zope/zope-2.4.0/lib/python/AccessControl/ZopeSecurityPolicy.py, line 229, in validate Unauthorized: (see above) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] SiteAccess: virtual host monster and setServerURL issue
Evan and Zope Dev - I'm using the virtual host monster with apache proxy/rewrite mechanism. I faced a little bit of trouble dealing with zope's setServerURL method. I'm using a configuration such as this on the apache side: RewriteRule ^/(.*)$ \ http://localhost:1080/VirtualHostBase/http/%{HTTP_HOST}/vtfs/%{SERVER_NAME}/VirtualHostRoot/$1 [P,L] * HTTP_HOST varies depending on how the client reaches the apache server. * SERVER_NAME (or really the name of the virtual host) is a constant defined in the apache virtual host. The correct virtual URL is not setup properly because the setServerURL method is always picking up the port #1080 (via oldhost,oldport) from the mod_proxy/mod_rewrite request environment even if HTTP_HOST does not contain a port number. I patched this issue by simply commenting out the lines below. I would like to keep using the HTTP_HOST variable mechanism, because the absolute_url() generated doesn't directly depend on the proper dns settings of the web client. For example, this allows apache to serve the content properly even if ssh is being used to forward http/https connections. Any recommendations?? - joe n. def setServerURL(self, protocol=None, hostname=None, port=None): """ Set the parts of generated URLs. """ other = self.other server_url = other.get('SERVER_URL', '') if protocol is None and hostname is None and port is None: return server_url oldprotocol, oldhost = splittype(server_url) oldhostname, oldport = splitport(oldhost[2:]) if protocol is None: protocol = oldprotocol if hostname is None: hostname = oldhostname #if port is None: port = oldport #if (port is None or default_port[protocol] == port): if (port is None): host = hostname else: host = hostname + ':' + port server_url = other['SERVER_URL'] = '%s://%s' % (protocol, host) self._resetURLS() return server_url -- ___A_R_S_e_e_D__I_n_c_.__ Joseph Norton C.T.O. (Chief Technology Officer) Tel:03-5431-5240 (Fax:5241) Visit us! http://www.arseed.co.jp/garden/
[Zope-dev] Re: SiteAccess: virtual host monster and setServerURL issue
Evan - I agree with your statement, but the system is behaving differently. The generated URLs should not have :80 in them and they do not ... rather they have :1080 included in the generated URLs although I have specifed something as follows: mod_rewrite result: proxy:http://localhost:1080/VirtualHostBase/http/www.foo.com/vtfs/www.foo.com/vdmn/prd/vstg/VirtualHostRoot/ [OK] and zope/vhm is generating URLs of the form: http://www.foo.com:1080/ I believe the setServerURL method is using the port number from the environment setup from the mod_proxy/mod_rewrite request. I'm currently using 1080 for zope (z2.py -w 1080). - j p.s. One more related question have you tried using mod_proxy/mod_rewrite with zope's webdav source port (-W) and virtual host monster? At Tue, 10 Apr 2001 10:57:15 -0400, Evan Simpson wrote: From: "Joseph Wayne Norton" [EMAIL PROTECTED] The correct virtual URL is not setup properly because the setServerURL method is always picking up the port #1080 (via oldhost,oldport) from the mod_proxy/mod_rewrite request environment even if HTTP_HOST does not contain a port number. This is deliberate. Omitting the port number means "leave it alone". If you want port 80 (the http default), you need to specify it. Generated URLs will not have ":80" in them, since it *is* the default port. Cheers, Evan @ digicool 4-am ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] python methods zope - security hole ?
Evan - I believe I have found a situation while using python methods that exposes a security hole. A user that can create python methods can grant the Manager role to himself by simply writing and then calling the manage_users method. This issue might also apply to other scenarios. I created a simple test case to check the behavior ... - create a user without any roles in a acl_users folder (see below) - create a python method such as the one below that grants root access to the user (see below) - create a dtml method that calls the python method this method should have view and access content information granted (see below) to the user created in the first step. In my case, I'm simply using the anonymous role ... but it could be another role setup for developers, content editors, etc. - execute the dtml method (see below) I also created a similiar test case for dtml methods and the dtml method behaved as I expected preventing the user from obtaining the manage_users role because they do not have the role in the first place. I just want to confirm if this is a known issue or a new issue (or feature or mistake in my zope setup). I'm currently running with zope 2.2.5 and python methods 0-1-7. If this is indeed an issue, it might be worthwhile to simply document this issue on the PythonMethods product page. thanks, - joe n. ### # Here is the output of this behavior ... UserName: dummy_non_manager Roles: () Has 'View' ?: 0 Has 'Manage users' ?: 0 user 'dummy_non_manager' roles '['Manager']' user 'dummy_manager' roles '['Manager']' UserName: dummy_non_manager Roles: ('Manager',) Has 'View' ?: 1 Has 'Manage users' ?: 1 ### # Here is the python method args: self,REQUEST body: ret = '' users = {'dummy_manager' : ['Manager'],'dummy_non_manager' : ['Manager']} for u in users.keys(): roles = users.get(u) REQUEST.set('name', u) REQUEST.set('roles', roles) self.acl_users.manage_users('Change',REQUEST) ret = ret + ('user \'%s\' roles \'%s\' br\n' % (u, roles)) return ret ### # Here is the dtml method: dtml-var standard_html_header UserName: dtml-var "REQUEST['AUTHENTICATED_USER'].getUserName()"br Roles:dtml-var "REQUEST['AUTHENTICATED_USER'].getRoles()"br Has 'View' ?: dtml-var "REQUEST['AUTHENTICATED_USER'].has_permission('View',REQUEST['AUTHENTICATED_USER'])"br Has 'Manage users' ?: dtml-var "REQUEST['AUTHENTICATED_USER'].has_permission('Manage users',REQUEST['AUTHENTICATED_USER'])"br dtml-var "acl_users_roles_python_method(REQUEST)" UserName: dtml-var "REQUEST['AUTHENTICATED_USER'].getUserName()"br Roles:dtml-var "REQUEST['AUTHENTICATED_USER'].getRoles()"br Has 'View' ?: dtml-var "REQUEST['AUTHENTICATED_USER'].has_permission('View',REQUEST['AUTHENTICATED_USER'])"br Has 'Manage users' ?: dtml-var "REQUEST['AUTHENTICATED_USER'].has_permission('Manage users',REQUEST['AUTHENTICATED_USER'])"br dtml-var standard_html_footer -- -- Joseph Norton [EMAIL PROTECTED] +81-3-3822-6936 2-10-7 Tabata, Kita-ku, Tokyo 114-0014, Japan (〒114-0014 東京都北区田端2丁目10-7) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )
[Zope-dev] problems with webdav MOVE and COPY
Hello. I tried to use some of the webdav functions in zope but ran into some trouble with the MOVE and COPY operations. One issue seems to be that the method "getSecurityManager" is not imported into the file zope-2.2/lib/python/webdav/Resource.py. def dav__validate(self, object, methodname, REQUEST): msg='strongYou are not authorized to access this resource./strong' method=None if hasattr(object, methodname): method=getattr(object, methodname) else: try:method=object.aq_acquire(methodname) except: method=None if method is not None: -- try: return getSecurityManager().validateValue(method) except: pass raise 'Unauthorized', msg Secondly, the method try: parent._verifyObjectPaste(self) is failing even though the user is authenticated as a manager. For example, try: parent._checkId(name, allow_dup=1) except: raise 'Forbidden', sys.exc_info()[1] -- try: parent._verifyObjectPaste(self) except: raise 'Forbidden', sys.exc_info()[1] Lastly, where are properties that can be set and put using webdav. Are these object attributes? I was expecting to see this properties show up in the "Properties" tab for a folder, for a dtml document etc. but they did not show up. Any pointers? thanks, - joe n. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://lists.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://lists.zope.org/mailman/listinfo/zope-announce http://lists.zope.org/mailman/listinfo/zope )