Re: global variables and reparsing question (low priority ;)
On Tue, May 23, 2000 at 10:05:01AM +0200, Marc Lehmann wrote: > On Tue, May 23, 2000 at 12:56:28AM -0500, Autarch <[EMAIL PROTECTED]> wrote: > > On Tue, 23 May 2000, Marc Lehmann wrote: > > > > > stable (mod_perl really is very unstable for large applications). Apart > > > > Wow, I wish you'd warned me before I did several large applications using > > mod_perl. Fortunately, they haven't experienced any mod_perl related > > Well, add XML::Parser, some large and twisted perl modules (like > XML::XSLT), do a lot of things dynamically (recompiling etc..) and apache > segfaults quite a lot, at random times, and even with some weird effects > (adding a comment makes httpd segfault, removing it and it works again). > Fortunately, once you found a source form where httpd starts up, mod_perl > seems to run fine for a long time. Did you follow the thread about XML::Parser's bug WRT apache embedding? You need to change the configure arguments to apache if you want this to work... Dan /----\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Perl as DSO...whats wrong with it, specifically?
On Tue, Jul 25, 2000 at 01:14:32PM -0700, Michael Nachbaur wrote: > I'm working with IBM (sorry! Its not my choice!) to try to get > mod_perl compiled into their custom version of Apache, and their > techs want to know why I can't run mod_perl as a DSO. So, I need to > know, whats specifically wrong with it? Like, what are the > repurcussions of it? This is going to be run under a lot of load as > well. > > Thanks for your feedback in advance. Well, it works now in most cases, if you use the current version. It's not necessarily a good idea if you're dealing with x86 (it has to be built PIC, and thus increases register pressure). But other than the slight PIC slowdown it hasn't caused problems in a few months. Dan /----\ /----\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Bug#74416: apache-perl: Apache::MyConfig should provide path to `apxs' (/usr/bin/apxs) in %Setup
[Please keep the CC: to bugs.debian.org] On Mon, Oct 09, 2000 at 10:59:29PM +0200, [EMAIL PROTECTED] wrote: > Package: apache-perl > Version: 1.3.9-13.1-1.21.2309-1 > Severity: normal > > I've tried to create a package `libapache-request-perl' > (Module Apache::Request). (Such a package doesn't yet exist > neither in potato nor in woody.) > > I have currently installed apache-perl and apache-dev (beside others). > When I try to build the package, gcc can't find `httpd.h' (and other > header files). Makefile.PL asks the path to include files from Apache::MyConfig. > But without `APXS' set this module can't provide all needed directories. The problem is that apache-perl is the statically linked binary, and thus does not use apxs in its own build process. modperl folks: should APXS get set anyway? Is Apache::Request wrong? > A few months ago I could successfully build libapache-request-perl since > then I had installed `apache' (together with `libapache-mod-perl') instead > of `apache-perl'. Apache::MyConfig from `libapache-mod-perl' provides > the path to `apxs'. > > Following the contents from the two versions of Apache/MyConfig.pm: > > libapache-mod-perl 1.24-1 > = > > # > # Configuartion for mod_perl and Apache::... > # > package Apache::MyConfig ; > > %Setup = ( >'Apache_Src' => '', >'SSL_BASE' => '', >'APXS' => '/usr/bin/apxs', > ); > > 1; > > __END__ > > > apache-perl 1.3.9-13.1-1.21.2309-1 > = > > # > # Configuartion for mod_perl and Apache::... > # > package Apache::MyConfig ; > > %Setup = ( >'Apache_Src' => >'/usr/src/aperl/apache-perl-1.3.9-13.1-1.21.2309/apache-1.3.9/build-tree/apache_1.3.9/src', >'SSL_BASE' => '', >'APXS' => '', > ); > > 1; > > __END__ > > As you can see the value for `APXS' is empty in apache-perl's version. > > In case the lack of /usr/bin/apxs is intentional, I will make my package > Build-Depend on `apache' and `libapache-mod-perl'. Shouldn't be necessary... it should probably have just Build-Depends: libapache-mod-perl - note that apache-perl already provides both apache and libapache-mod-perl. Also don't forget apache-dev. Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Shroud+ Perl obfuscator....
On Sat, Dec 21, 2002 at 12:53:34AM -0500, kyle dawkins wrote: > Are you for real? > > Or is this some lame attempt at sarcasm? > > Andrzej posts to the list, SHARING some code he's written in case some > people actually might use it, and he gets bitchslapped with some > holier-than-thou rhetoric? Puh-leaze. Take your total bullshit > somewhere else because you're about as constructive as > Richard-f**king-Stallman. > > You may not know this, but people actually use perl for things other > than one-liners. Commercial projects actually use perl, and oftentimes > these commercial projects are sensitive and copyrighted. And > occasionally, just occasionally, these projects actually want to have > some level of security, for numerous reasons that I'm sure Andrzej > could explain quite easily to us if he were asked. And if they do have something to protect, they should put their thinking caps on and realize that this sort of "security" is called obfuscation for a reason: it does not accomplish anything except to make the results hard to read. If you're giving away or selling the perl source, obfuscating it doesn't have any significant effect. -- Daniel Jacobowitz MontaVista Software Debian GNU/Linux Developer
A year of accumulated patches
Hi, I'm your neighborhood Debian package maintainer for mod_perl... I made the mistake of getting off this list a while back, and I've accumulated a bit of a collection of patches since then. Most in the last few days. So I figured now that I was actively working on mod_perl again, I would submit them back where they belong. First of all, in CVS I see this: > fix $ENV{PATH} corruption, thanks to help from Chip Turner, > Oleg Bartunov and Tomasz Przygoda I'm still seeing some $ENV{PATH} corruption built as a DSO, though. I'll look in to it later tonight. The first really important one is a fix for the crashes I've been seeing in boot_DBI. I can't find the exact message in the archive now, but anyone who reported that a problem was fixed by setting PERL_STARTUP_DONE_CHECK=1 in the environment probably wants this patch instead. The problem was that a library (in this case libpg.so) called putenv() after perl was initialized but before perl's my_setenv got called. Thus environ[] never got copied to malloc()'d memory, and trying to overwrite $ENV{TZ} caused a bogus free(). That's environment-mod_perl.c. A pair of small perl patches, from Roderick Schertler <[EMAIL PROTECTED]>: - Apache.pm: send_cgi_header was modifying the string in $_. Fixed in local-underscore-Apache.pm. - There was a precedence problem in StatINC.pm. Fixed in precedence-StatINC.pm. And a trivial fix for mod_perl.c - one of the status messages had "...ok" instead of "...". Fixed in message-cleanup-mod_perl.c. Makefile.PL would not handle PERL_DEBUG=1 if USE_DSO, because the code was stuck in a loop over available apache source trees. I just moved it outside of the loop. Fixed in PERL_DEBUG-Makefile.PL. And now the doozy. This one is not like the others; it is a stopgap, EVIL hack and should not be included in the source. It's included only for observation purposes. I worked around the ongoing bug involving the DSO not liking being reloaded. From what I could tell the former hack, mp_dso_unload, was somehow never getting called; if it had gotten called, it would have caused a segfault at dlclose(0), which I found out the hard way. The patch 'hideous-hack-mod_perl.c' actually walks the cleanup tree and removes all module unloads - removing our own did not suffice to fix the problem, oddly. I can think of a dozen better ways to do this cleanly, involving small apache patches; I am also fully aware that this is a symptom, not a problem, that I am fixing. But it -does- suffice to make the DSO work. For now. Be warned, the patch might reverses the meaning of PERL_DSO_UNLOAD. I couldn't quite tell what it was supposed to accomplish before as I could not even get that code to trigger. I saw the message about the cyclic .init sections... I don't think that is truly the problem, although I'm sure it's a problem. I'm using libperl.a instead of libperl.so on this platform, and thus there is no .init section. Right? Certainly none showed up in the linker debug logs, although this is the linux dynamic loader, which does not do cyclic dependency checking far as I can tell, or at least can't print a message about it. LD_PRELOAD'ing the module DOES work to prevent the bug, however. I can't see why, and I can't get a good debugging dump. I'll look at it more later. Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/ precedence-StatINC.pm PERL_DEBUG-Makefile.PL --- libapache-mod-perl-1.21.orig/src/modules/perl/mod_perl.c +++ libapache-mod-perl-1.21/src/modules/perl/mod_perl.c @@ -659,6 +655,16 @@ } MP_TRACE_g(fprintf(stderr, "ok\n")); +/* Force the environment to be copied out of its original location + above argv[]. This fixes a crash caused when a module called putenv() + before any Perl modified the environment - environ would change to a + new value, and the check in my_setenv() to duplicate the environment + would fail, and then setting some environment value which had a previous + value would cause perl to try to free() something from the original env. + This crashed free(). */ +my_setenv("MODPERL_BROKEN_ENV", "0"); +my_setenv("MODPERL_BROKEN_ENV", NULL); + { dTHR; TAINT_NOT; /* At this time all is safe */ local-underscore-Apache.pm --- libapache-mod-perl-1.21.orig/src/modules/perl/mod_perl.c +++ libapache-mod-perl-1.21/src/modules/perl/mod_perl.c @@ -607,7 +603,7 @@ } MP_TRACE_g(fprintf(stderr, "ok\n")); -MP_TR
Re: A year of accumulated patches
On Sat, Jan 15, 2000 at 12:55:47AM +, Tim Bunce wrote: > On Fri, Jan 14, 2000 at 07:28:22PM -0500, Daniel Jacobowitz wrote: > > > > The first really important one is a fix for the crashes I've been > > seeing in boot_DBI. I can't find the exact message in the archive now, > > but anyone who reported that a problem was fixed by setting > > PERL_STARTUP_DONE_CHECK=1 in the environment probably wants this patch > > instead. The problem was that a library (in this case libpg.so) called > > putenv() after perl was initialized but before perl's my_setenv got > > called. Thus environ[] never got copied to malloc()'d memory, and > > trying to overwrite $ENV{TZ} caused a bogus free(). That's > > environment-mod_perl.c. > > I've never looked at the mod_perl source code, but does it call > perl_parse() with a NULL for the env arg? If not it should. Yes, it does. Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Apache::ASP: crash when placed in startup.pl
This sounds like the same corruption problem that I talked about in my last message - could you try exporting LD_PRELOAD=/path/to/modperl.so? (I'm guessing from context that you're using a DSO - apologies for the non-sequitor if I'm wrong, I just got back on the list). On Fri, Jan 14, 2000 at 05:57:49PM -0800, Dmitry Beransky wrote: > > >I don't know about this, are you sure you copied over your > >new modperl httpd to /usr/local/apache after the build, > >and did a full stop / start, this is a common step overlooked > >by the best of us. > > Yes, I'm using the new build. I've compiled perl with debugging turned on > and now gdb shows exactly where the problem occurs: line 267 of SDBM_File.c > > 262 #ifdef __cplusplus > 263 extern "C" > 264 #endif > 265 XS(boot_SDBM_File) > 266 { > 267 dXSARGS; > 268 char* file = __FILE__; > 269 > 270 XS_VERSION_BOOTCHECK ; > > I guess at this point I got to go to a bookstore, get a book on GDB and get > my hands dirty with some debugging :). > > Thanks > Dmitry > > > > >This might be yet another RedHat problem ... have you tried > > > >rebuilding your perl / mod_perl from scratch. Out of curiosity > > > >really, because I think you should still rebuild stuff, > > > >does SDBM_File load fine, if not being preloaded ? > > > > > > Well, I just finished recompiling perl and mod_perl, but the problem is > > > still there. Is there any way to verify that mod_perl is using the new > > perl? > > > > Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: A year of accumulated patches
On Fri, Jan 14, 2000 at 07:28:22PM -0500, Daniel Jacobowitz wrote: > First of all, in CVS I see this: > > fix $ENV{PATH} corruption, thanks to help from Chip Turner, > > Oleg Bartunov and Tomasz Przygoda > > I'm still seeing some $ENV{PATH} corruption built as a DSO, though. > I'll look in to it later tonight. Scratch that, it's gone now. Must have had a stale copy running. Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: A fix in sight: Apache::ASP: crash when placed in startup.pl
On Sat, Jan 15, 2000 at 08:58:35PM -0500, Daniel Jacobowitz wrote: > Thus my confusion. On Linux, with a static libperl.a, there are no > multiple copies of libperl.so floating around to link to. The symbols > only exist in one place. Thus there is no choice about what symbols > link to. Still, I'm going to go over my linker logs again and see if > something useful turns up on that front. Ah-HA! I believe I see the problem. In fact, I'm almost sure I do. >From the one useful ltrace log I was able to get: % grep '^dl' min.ltrace.log dlopen("/usr/lib/apache/1.3/mod_cgi.so", 258) = 0x080a4198 dlsym(0x080a4198, "cgi_module") = 0x40187540 dlopen("/usr/lib/apache/1.3/mod_mime.so", 258)= 0x080a4518 dlsym(0x080a4518, "mime_module") = 0x40017a00 dlopen("/usr/lib/apache/1.3/mod_perl.so", 258)= 0x080a4850 dlsym(0x080a4850, "perl_module") = 0x402b6bc0 dlopen("/usr/lib/perl5/5.005/i386-linux/"..., 1) = 0x08118e60 dlsym(0x08118e60, "boot_DBI") = 0x402e4834 dlclose(0x080a4850) = 0 dlclose(0x080a4518) = 0 dlclose(0x080a4198) = 0 dlopen("/usr/lib/apache/1.3/mod_cgi.so", 258) = 0x0817f958 dlsym(0x0817f958, "cgi_module") = 0x40187540 dlopen("/usr/lib/apache/1.3/mod_mime.so", 258)= 0x080afcb8 dlsym(0x080afcb8, "mime_module") = 0x4018aa00 dlopen("/usr/lib/apache/1.3/mod_perl.so", 258)= 0x080a4170 dlsym(0x080a4170, "perl_module") = 0x402b9bc0 dlopen("/usr/lib/perl5/5.005/i386-linux/"..., 1) = 0x08118e60 dlsym(0x08118e60, "boot_DBI") = 0x402e4834 Notice that DBI is never dlclose()'d. But mod_perl is, when apache unloads its modules. The linker is not clever enough to realize that DBI depends on symbols in mod_perl.so. When mod_perl is reloaded at a DIFFERENT address, DBI has no reason to be rebound; its symbols are already resolved! But now they are resolved to the wrong places. This looks to be precisely contrary to the way that the linker is supposed to work, but I'll get back to that later. Thus my question is, is there a way to tell perl to dlclose() all modules? From looking through the perl code and seeing a noticeable lack of dlclose() calls, I'd say no. DynaLoader doesn't seem to support unloading things. It does, however, store the loaded dlopen handles in @DynaLoader::dl_librefs, which we could get at in a cleanup. Woah! This worked! Can I get opinions on the attached patch? Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/ --- mod_perl.c.orig Sat Jan 15 22:08:27 2000 +++ mod_perl.c Sat Jan 15 22:13:01 2000 @@ -417,23 +417,30 @@ #if MODULE_MAGIC_NUMBER >= MMN_130 static void mp_dso_unload(void *data) { -module *modp; - -if(!PERL_DSO_UNLOAD) - return; - -if(strEQ(top_module->name, "mod_perl.c")) - return; - -for(modp = top_module; modp; modp = modp->next) { - if(modp->dynamic_load_handle) { - MP_TRACE_g(fprintf(stderr, - "mod_perl: cancel dlclose for %s\n", - modp->name)); - modp->dynamic_load_handle = NULL; + I32 i; + void *handle; + AV *librefs; + SV *handle_sv; + + librefs = perl_get_av("DynaLoader::dl_librefs", FALSE); + if (!librefs) { + MP_TRACE(fprintf(stderr, "Could not get @dl_librefs for +unloading.\n")); + return; } -} -} + + for(i = 0; i <= AvFILL(librefs); i++) { + handle_sv = *av_fetch(librefs, i, FALSE); + if(!handle_sv) { + MP_TRACE(fprintf(stderr, "Could not fetch $dl_librefs[%d]!\n", +i)); + continue; + } + handle = (void *)SvIV(handle_sv); + if(dlclose(handle) < 0) { + fprintf(stderr, "Could not close $dl_librefs[%d]: %s\n", i, +dlerror()); + continue; + } + } +} #endif static void mp_server_notstarting(void *data)
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 12:59:13PM +, Alan Burlison wrote: > Can someone please explain why Apache does all the dlclosing and > dlopening of shared files on startup and a restart? I can think of no > reson why this would ever be necessary - why on earth is it done? > > Alan Burlison > Probably the biggest reason for dlopen/dlclose on a restart is that the list of modules in the config file can change on a restart. The reason for the reload on startup has something to do with parsing the config file in the parent and child; it was never adequately explained to me. The trick would be getting it not to do this without busting up the module API, which I can actually think of a few ways to do, and in a way that the Apache Group didn't rigorously object to :) Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 02:40:59PM +0100, Gerald Richter wrote: > > Probably the biggest reason for dlopen/dlclose on a restart is that the > > list of modules in the config file can change on a restart. The reason > > for the reload on startup has something to do with parsing the config > > file in the parent and child; it was never adequately explained to me. > > > > The trick would be getting it not to do this without busting up the > > module API, which I can actually think of a few ways to do, and in a > > way that the Apache Group didn't rigorously object to :) > > > > Do you know, would it be possible to catch the dlclose inside mod_perl and > unload (dlclose) all external libaries that Perl has loaded so far? The > problem is, that these libraries will persist between such an dlclose/dlopen > and afterward don't get proberly initialised, so some of the code really > points to data from the old Perl interpreter, which causes strange > behaviour. > That is what my patch did. And that was the explanation I posted of the problem last week when we were debugging it. Dan /----\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 08:26:17PM +0100, Gerald Richter wrote: > Also I didn't tried it yet, your patch makes much sense too me. I will try > it out as soon as I get a litle free time. The next step is to port it NT, > because there isn't a dlopen/dlclose (of course there is one, but with a > different name) and on NT Apache only work with dynamic libraries. If it > works on Unix and NT it's should go straight to the CVS and a mod_perl 1.22 > ! Now if only we could get the memory leaks tracked down... ltrace logs of apache are simply too huge to deal with, but I'll try anyway to straighten them out and figure what is not getting freed. Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 08:03:42PM +, Alan Burlison wrote: > The current fix is to forcibly unload the perl XS modules during the > unload. However, on reflection I'm not at all sure this is the correct > thing to do. Although you can unload the .so component of a perl > module, you can't unload the .pm component, so just removing the .so > part as in the current workaround is suspect at least. Remember - mod_perl is being unloaded. Perl going away. At this point perl_destruct/perl_free have already been called, and thus the .pm components are effectively unloaded. > I think the correct fix is for the Apache core to avoid dlclosing > anything it has dlopened in the first place. If new modules have been > added to the config files, they should be dlopened, but any old ones > should *not* be dlclosed, EVEN IF THEY ARE NO LONGER IN THE CONFIG > FILE!!! > > I firmly believe this needs fixing in the Apache core, not by hacking > around it in mod_perl. > > Alan Burlison > Dan /----\ /----\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 10:19:04PM +0100, Gerald Richter wrote: > > > > > Yes it can. > > No it can't :-) > > > Its main job is to keep track and control the dependencies > > between libraries. It's just that sometimes thy don't do a particularly > > good job of it ;-) > > > > This works only if this dependencies are know at link time, but look at the > source of Dynloader. You can retrieve address of any (public)symbol inside a > library dynamicly at runtime. Now you have the entry address and can pass it > around. No linker will ever have a chance to know what you do in your > programm. As soon as you use such things (and Dynloader uses them), the > linker doesn't have chance! You're confusing the dynamic and static linkers. The dynamic linker is what he was referring to; it knows what libraries it resolves symbols to. Dan /----\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Why does Apache do this braindamaged dlclose/dlopen stuff?
On Tue, Jan 18, 2000 at 10:43:09PM +0100, Gerald Richter wrote: > > > > You're confusing the dynamic and static linkers. The dynamic linker is > > what he was referring to; it knows what libraries it resolves symbols > > to. > > > Yes, I know this difference and you will be right in most cases, but the > address that is returned, could be passed around to other libraries and the > linker can't know this. (the dynloader.so can retrieve an adresse of > embperl.so and pass it's address to modperl.so, which then pass the address > to whatever. How should the dynamic linker know, that whatever is calling > the symbol in embperl.so (and stores the address) Ah, I see what you mean. That's boundedly undefined - the linker certainly shouldn't be trying to catch that sort of case. But it should, IMO, catch bindings that it makes itself. Dan /--------\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: A fix in sight: Apache::ASP: crash when placed in startup.pl
On Tue, Jan 18, 2000 at 11:02:27PM -0800, Doug MacEachern wrote: > On Sat, 15 Jan 2000, Daniel Jacobowitz wrote: > ... > > Notice that DBI is never dlclose()'d. But mod_perl is, when apache > > unloads its modules. The linker is not clever enough to realize that > > DBI depends on symbols in mod_perl.so. When mod_perl is reloaded at a > > DIFFERENT address, DBI has no reason to be rebound; its symbols are > > already resolved! > > wow, *nice* catch!! Daniel, I can't thank you and Alan enough for your > efforts here. it's such a thorny problem to debug, the closest I came was > trying to prevent the dlclose of modperl's libperl.so, but had no idea why > that bandaid prevented the bleeding. I hadn't heard of ltrace, what a > spectacular looking tool. > I'd really like to get the workaround in 1.22, even if Apache and/or > DynaLoader is the "right" place, we don't have any control over the > release schedules of either and modperl-1.22 is already long overdue. > > perl_destruct/perl_free are not called at restart, only during child_exit. > it looks like that might need to change to finish the dso puzzle. I > experienced some problems there, but that was a few years ago, looks > like it's time to revisit. *blink* Oh, you're right, of course. They weren't getting called at all. I think that's a problem, and I have a good idea of how to go about fixing it - I'll get back to you on that in a few hours. Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: OS Independend Patch for XS Extentions unload
On Wed, Jan 19, 2000 at 06:32:54AM +0100, Gerald Richter wrote: > Hi, > > I have tested the patch to unload all XS libraries, when libperl is > unloaded, Daniel sended a few days ago on Unix and on NT and it works! > Really great!! > > Here is a sligthly modified version, so it works also on NT (and on others > OS's). It uses the Apache function ap_os_dso_unload instead of dlclose. > > The patch is against the actual CVS of mod_perl. Well, one problem. I deleted the old code instead of #if'ing it for a reason - it does NOT work. Apache will try to dlclose() the modules anyway, and dlclose(0) causes a segfault, on Linux at least. Other than that, looks just fine. Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Results of calling perl_shutdown in mp_dso_unload
(with appropriate changes to unload DSOs after shutting down perl, of course). I still get memory leakage - that's not terribly surprising - but it is much less. It's on the order of about 24K/restart and is probably the fault of some module - my guts would be DBD::Pg. I'll play around with that later. It's small enough to show that the idea is workable. Restarts are also a bit slower, unsurprisingly - there's a lot more to do. Then we have the fact that this currently removes PerlRestartHandler functionality and goes far beyond PerlFreshRestart. Those are a big issue - those will HAVE to work differently depending on whether or not mod_perl is a DSO. If not, they can continue to work as before, but if it is they will never be invoked. I'll clean up the patch and fire it off later this weekend, if Doug doesn't get a chance to look at it before then. Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: [ID 20000118.004] DynaLoader doesn't dlclose XS code on interpreter
On Wed, Jan 26, 2000 at 04:06:29PM +, Alan Burlison wrote: > Gurusamy Sarathy wrote: > > > Scheduling dlclose()s using call_atexit() ought to work. I'm going > > to let the interested parties test that and send me a patch. > > Is anyone up for this? I'm in the US this week, so I'm not able to > contribute much (plus I'd probably just break something ;-) Gurusamay, > do you have a timescale for when you need this? I'll try to do it this weekend, or else if I get bored during LinuxWorld Expo :) Dan /--------\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Results of calling perl_shutdown in mp_dso_unload
On Wed, Jan 26, 2000 at 08:13:41PM -0800, Doug MacEachern wrote: > right, that's one of the reasons restarts are a noop for mod_perl by > default (not including dso-magic) Makes sense. Here's a question - an apache patch should be able to mark the module as not-to-unload (although since it should be not-to-unload-unless-removed-from-config it would be a bit more complicated than that - still not impossible). But is that worthwhile or should we just adjust to being unloaded if built as DSO? > > Then we have the fact that this currently removes PerlRestartHandler > > functionality and goes far beyond PerlFreshRestart. Those are a big issue - > > those will HAVE to work differently depending on whether or not mod_perl is > > a DSO. If not, they can continue to work as before, but if it is they will > > never be invoked. > > > > I'll clean up the patch and fire it off later this weekend, if Doug doesn't > > get a chance to look at it before then. > > Doug is slowly crawling back from the death flu :-/ > I'm think about releasing 1.22 as it stands in the cvs tree, since it's so > long overdue, and bang out the dso issues from there. as much as I want > to cure the dso problems, I don't want to introduce new problems the > currently stable static configuration. Understood - good idea. Which reminds me - the CVS instructions linked from perl.apache.org no longer work. Is there a public CVS archive still? Where did it move? Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Bug#61231: mod_perl segfaults child-processes in combination with XML::Parser::Expat
[mod_perl people - any comment? Please keep the Cc: list to the Debian bug tracking system.] On Tue, Mar 28, 2000 at 12:50:45PM +0200, [EMAIL PROTECTED] wrote: > Package: libapache-mod-perl > Version: 1.21.2309-1 > Severity: Important > > When using the XML::Parser::Expat under mod_perl, this causes segmentation > faults (quite random?) in the child-processes of httpd. > The error message in the error.log of apache is: > "[notice] child pid 28177 exit signal Segmentation fault (11)" > > This can be repreduced with the following script, > run under mod_perl, just wait a while, and monitor the error.log (takes a > while, multiple request will speed up): It's very random, but I can reproduce this. However, I can also reproduce it with apache-perl (statically linked). It seems extremely unlikely to me that this is the DSO code's fault, therefore. Here's a backtrace: (gdb) bt #0 0x400f8b59 in free () from /lib/libc.so.6 #1 0x400f89ed in free () from /lib/libc.so.6 #2 0x4026eb7a in Perl_safefree () from /usr/lib/apache/1.3/mod_perl.so #3 0x4027fcb9 in Perl_sv_setsv () from /usr/lib/apache/1.3/mod_perl.so #4 0x402782dc in Perl_pp_sassign () from /usr/lib/apache/1.3/mod_perl.so #5 0x402a8440 in Perl_runops_standard () from /usr/lib/apache/1.3/mod_perl.so #6 0x4024c0a9 in perl_call_sv () from /usr/lib/apache/1.3/mod_perl.so #7 0x4023201e in perl_call_handler () from /usr/lib/apache/1.3/mod_perl.so #8 0x402318cc in perl_run_stacked_handlers () from /usr/lib/apache/1.3/mod_perl.so #9 0x402302cf in perl_handler () from /usr/lib/apache/1.3/mod_perl.so #10 0x8053e64 in ap_invoke_handler (r=0x826e23c) at http_config.c:508 #11 0x80625ec in process_request_internal (r=0x826e23c) at http_request.c:1214 #12 0x8062648 in ap_process_request (r=0x826e23c) at http_request.c:1230 #13 0x805c439 in child_main (child_num_arg=0) at http_main.c:4122 #14 0x805c5cc in make_child (s=0x8099c24, slot=0, now=954874778) at http_main.c:4235 #15 0x805c6e9 in startup_children (number_to_start=5) at http_main.c:4317 #16 0x805cb9b in standalone_main (argc=2, argv=0xbdc4) at http_main.c:4605 #17 0x805d24d in main (argc=2, argv=0xbdc4) at http_main.c:4933 Very interesting. I'll look at this further. Dan /--------\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Bug#61231: mod_perl segfaults child-processes in combination with XML::Parser::Expat
ule /usr/lib/apache/1.3/mod_cgi.so > LoadModule userdir_module /usr/lib/apache/1.3/mod_userdir.so > LoadModule alias_module /usr/lib/apache/1.3/mod_alias.so > LoadModule rewrite_module /usr/lib/apache/1.3/mod_rewrite.so > LoadModule access_module /usr/lib/apache/1.3/mod_access.so > LoadModule auth_module /usr/lib/apache/1.3/mod_auth_ssl.so > LoadModule expires_module /usr/lib/apache/1.3/mod_expires.so > LoadModule unique_id_module /usr/lib/apache/1.3/mod_unique_id.so > LoadModule setenvif_module /usr/lib/apache/1.3/mod_setenvif.so > LoadModule perl_module /usr/lib/apache/1.3/mod_perl.so > > access.conf: > > AddHandler perl-script .pl > PerlHandler Apache::Registry > PerlSendHeader On > > > > > Notice: > - preloading perl-scripts or perl-modules doesn't matter. > - both Apachr::Registry as Apache::PerlRun (without perlsetvar > perlrunonce!) both cause the problem. > - both XML::Parser::Expat and XML::Parser::ExpatNB cause the problem. > - only few httpd-children will cause bug faster > - maxrequestperchild greather 1 > - when using more (greater/complexer program) xml-parser-expat objects, > segfaults occur fatser and more often. > - test-program above runs clean under warnings, strict and taintcheck > > > Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Bug#61231: mod_perl segfaults child-processes in combination with XML::Parser::Expat
reassign 61231 libxml-parser-perl thanks And sure enough, the man is right. This is not mod_perl's problem. When I disable RULE_EXPAT, apache ceases crashing. On Tue, Apr 04, 2000 at 02:49:36PM -0700, Doug MacEachern wrote: > On Tue, 4 Apr 2000, Daniel Jacobowitz wrote: > > > [mod_perl people - any comment? Please keep the Cc: list to the Debian > > bug tracking system.] > > > > When using the XML::Parser::Expat under mod_perl, this causes segmentation > > > faults (quite random?) in the child-processes of httpd. > > > The error message in the error.log of apache is: > > > "[notice] child pid 28177 exit signal Segmentation fault (11)" > > yeah, try configuring Apache with: > RULE_EXPAT=no > > otherwise, the symbols in XML::Parser clash with apache's. i've suggested > several times that the XML::Parser author should consider making those > symbols static or use a different prefix, but no idea if that's happened > or not. > > Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Trouble with DSO/APXS.
On Thu, Apr 20, 2000 at 11:56:00AM -0400, Robert B. Easter wrote: > Will mod_perl work as a DSO using USE_APXS=1? > > I've tried it very simply as: > perl Makefile.PL USE_APXS=1 > make > make install > > It compiles ok and Apache 1.3.12 will startup identifying itself as > mod_perl/1.22_01-dev mod_ssl 2.6.2 openSSL 0.9.5a (sometimes I Load php4 > too but it doesn't matter either way). When its up, I can load regular html > files and images and php scripts that access a PostgreSQL db. But when I try > to access a pl script or directory, I get no data returned and: Try the current version. Dan /\ /--------\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/
Re: Challenging things to do: SIGSEGV catcher and backtrace extractor
On Fri, Apr 12, 2002 at 02:09:44AM +0800, Stas Bekman wrote: > Doug MacEachern wrote: > >On Fri, 12 Apr 2002, Stas Bekman wrote: > > > > > >>If you read the rest of the post I mention it (without telling the name > >>:). The problem with this module is that it's useful only after you have > >>the core file. which is not good, because (as I've already explained): > > > > > >it's important to mention Devel::CoreStack, as it is a good starting > >point. > > true, as it has the config for a few known debuggers, but otherwise it > just invokes the debugger and pipes the 'bt' or equivalent command to it > and grabs the output. > > >>1. Many users have problems getting the core file dumped > > > > > >then there'd be no way to automate generating a stacktrace anyhow. > > You can get a backtrace if you run the process under debugger without > dumping a core file. No special setup required. I was thinking to attach > the debugger on SIGSEGV event. Is it too late? I see certain gnome apps > failing and they ask you if you want to get the stack, without me doing > anything at all. That's what I want for modperl. You say it's not possible? Sure, that's possible from the SEGV handler. -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer
Re: Memory leak on reload when the 'Pg' driver is preloaded
On Wed, Oct 16, 2002 at 02:01:33PM -0500, Keith G. Murphy wrote: > Ged Haywood wrote: > >Hi there, > > > >On Wed, 16 Oct 2002, Keith G. Murphy wrote: > > > > > >>do you mean that the problems with the loadable module overall are > >>so well-known that no one in his right mind should ever use it? > > > > > >It's not as bad as that. Significant improvements have been made in > >the reliability of mod_perl as DSO and nowadays there is much less > >discussion about it on this list. > > Are you sure it's not because 'most everyone has silently given up on it? > > >There are still one or two dusty > >corners but in general thesedays I'd say try it. > > I had not expected Debian stable to be one of the dusty corners. But I > did find, upon investigating the bug reports, that there were *very* old > reports about memory leaks, etc., with libapache-mod-perl. > > My own bug report is now 47 days old, without apparent followup. That's because I'm having an attack of real life. I haven't had time to look at my Debian bug reports in a month or so now. I don't know why the DSO is leaking in a case when the static module isn't, but it should be fixable; I just need to find five or six hours to sit down with it. -- Daniel Jacobowitz MontaVista Software Debian GNU/Linux Developer
[phil@fifi.org: Bug#85328: New method $req->allowed() for libapache-mod-perl]
[please maintain the CC's] I just received the following bug report on the Debian bug tracking system. The patch seems to apply to 1.25; is it a good idea? Dan /\ /\ | Daniel Jacobowitz|__|SCS Class of 2002 | | Debian GNU/Linux Developer__Carnegie Mellon University | | [EMAIL PROTECTED] | | [EMAIL PROTECTED] | \/ \/ Package: libapache-mod-perl Version: 1.21.2309-1 Severity: wishlist Tags: patch Here is enclosed a patch to libapache-mod-perl to expose a missing feature of the Apache API. This feature is needed to answer correctly to HTTP 1.1 requests. The patch is pretty much trivial and self-explanatory. Although it was done on the libapache-mod-perl from potato, it applies (with some offset) to the woody package. Please send upstream too... Phil. diff -ru ../x/libapache-mod-perl-1.21.2309/Apache/Apache.pm ./Apache/Apache.pm --- ../x/libapache-mod-perl-1.21.2309/Apache/Apache.pm Mon Mar 6 18:52:06 2000 +++ ./Apache/Apache.pm Thu Feb 8 18:04:16 2001 @@ -271,6 +271,18 @@ Returns true if the current request is the first internal request, returns false if the request is a sub-request or internal redirect. +=item $r->allowed($bitmask) + +Get or set the allowed methods bitmask. This allowed bitmask should be +set whenever a 405 (method not allowed) or 501 (method not implemented) +answer is returned. The bit corresponding to the method number should be +et. + + unless ($r->method_number == M_GET) { + $r->allowed($r->allowed | (1<allowed); + +OUTPUT: +RETVAL + time_t request_time(r) Apache r
[PATCH] [phil@fifi.org: Bug#86964: One more method (& patch) for libapache-mod-perl]
Here's another reasonable-looking patch from Philippe Troin; it exposes child_num() in Apache::Connection. -- Daniel Jacobowitz Debian GNU/Linux Developer Monta Vista Software Debian Security Team "I am croutons!" Package: libapache-mod-perl Version: 1.25-2 Severity: wishlist Included is more more patchlet to libapache-mod-perl. It adds the child_num() method to the Apache::Connection class. Why ? - The Apache C API has it, why not the perl API ? - You can do some cool stuff with it... I've used it as an index to a pseudo-scoreboard which forbids the same IP address to download the same file with two or more connection (that is bounce the download managers which were saturating my Apache server with multiple non-overlapping requests to the same file "to speed-up downloads"). It's a trivial patch. Please forward upstream. Phil. diff -ruN libapache-mod-perl-1.25.maint/Apache/Apache.pm libapache-mod-perl-1.25/Apache/Apache.pm --- libapache-mod-perl-1.25.maint/Apache/Apache.pm Thu Feb 22 01:01:58 2001 +++ libapache-mod-perl-1.25/Apache/Apache.pm Thu Feb 22 01:02:13 2001 @@ -567,6 +567,10 @@ This can be used to detect client disconnect without doing any I/O, e.g. using IO::Select. +=item $c->child_num + +Returns which child number (or "slot") handles the connection. + =back =back diff -ruN libapache-mod-perl-1.25.maint/src/modules/perl/Connection.xs libapache-mod-perl-1.25/src/modules/perl/Connection.xs --- libapache-mod-perl-1.25.maint/src/modules/perl/Connection.xs Fri Sep 29 07:41:30 2000 +++ libapache-mod-perl-1.25/src/modules/perl/Connection.xs Thu Feb 22 01:02:13 2001 @@ -157,6 +157,16 @@ OUTPUT: RETVAL +int +child_num(conn) +Apache::Connection conn + +CODE: +RETVAL = conn->child_num; + +OUTPUT: +RETVAL + char * auth_type(conn, ...) Apache::Connection conn
Re: [PATCH] [phil@fifi.org: Bug#86964: One more method (& patch) for libapache-mod-perl]
On Tue, May 01, 2001 at 11:15:30AM -0700, Doug MacEachern wrote: > On Sun, 4 Mar 2001, Daniel Jacobowitz wrote: > > > Here's another reasonable-looking patch from Philippe Troin; it exposes > > child_num() in Apache::Connection. > > i hesitate adding this to Connection.xs at this point since it is 1.3 > specific. but i would be happy to include it as an new method > Apache::Scoreboard. Eh? mod_perl 1.25 doesn't have a Scoreboard.xs... Are you talking about 2.0? -- Daniel Jacobowitz Debian GNU/Linux Developer Monta Vista Software Debian Security Team
Re: [PATCH] [phil@fifi.org: Bug#86964: One more method (& patch) for libapache-mod-perl]
On Thu, May 03, 2001 at 11:10:59AM +0800, Stas Bekman wrote: > On Wed, 2 May 2001, Daniel Jacobowitz wrote: > > > On Tue, May 01, 2001 at 11:15:30AM -0700, Doug MacEachern wrote: > > > On Sun, 4 Mar 2001, Daniel Jacobowitz wrote: > > > > > > > Here's another reasonable-looking patch from Philippe Troin; it exposes > > > > child_num() in Apache::Connection. > > > > > > i hesitate adding this to Connection.xs at this point since it is 1.3 > > > specific. but i would be happy to include it as an new method > > > Apache::Scoreboard. > > > > Eh? mod_perl 1.25 doesn't have a Scoreboard.xs... Are you talking > > about 2.0? > > it's on CPAN: Apache::Scoreboard Oh, duh. Thanks. -- Daniel Jacobowitz Debian GNU/Linux Developer Monta Vista Software Debian Security Team
Re: DSO Issues
On Wed, Nov 28, 2001 at 10:34:00AM -0500, Vivek Khera wrote: > >>>>> "SR" == Stephen Reppucci <[EMAIL PROTECTED]> writes: > > SR> If we're collecting a list of things that don't work in a DSO > SR> build, add perl subs (via ). > > SR> At least, they didn't work as of January of this year. > > Doubtful that they ever will, either, since that introduces a > dependency on mod_perl into mod_include. Both have to be linked > together (ie, statically) for that to work. Not really, you can do it with just module DT_NEEDED depends. It does mean you have to build them together and load in the right order. -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer
Re: $ENV{'REMOTE_USER'} getting clobbered?
On Wed, Nov 28, 2001 at 04:31:07PM -0800, Kenny Smith wrote: > Hello, > > I'm using mod_perl as a replacement for CGI. We are getting strange behavior > where $ENV{'REMOTE_USER'} isn't being set correctly. > > The script is protected by an .htaccess type of authentication, so when I > log in as 'kenny', $ENV{'REMOTE_USER'} should be equal to 'kenny'. However, > every once in awhile, $ENV{'REMOTE_USER'} will be equal to someone else's > login name, even though I'm still logged in as 'kenny' when I execute the > script. Does that make sense? > > This system was working fine for 2 months or more, but recently the number > of users and the load on the machine has increased, and we have started > seeing the problem. > > My theory is that %ENV is shared between requests and user 'bob' is > executing the script in the middle of my execution and $ENV{'REMOTE_USER'} > is getting overwritten in my execution with 'bob'. I believe the script is > taking longer to execute due to the load time which increases the window of > opportunity for REMOTE_USER to be clobbered. > > Software involved: > > apache 1.3.12 > mod_perl 1.24 > solaris 5.7 > perl 5.005_03 I believe that we fixed an environment corruption bug (at least for the DSO case, possibly static) sometime right after 1.24 was released. -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer
Re: Case of the Vanishing Symbol Tables
On Tue, Dec 11, 2001 at 04:15:49PM -0600, Stephen Clouse wrote: > With the PerlModule/%INC problem recently being rehashed, here's another one > involving PerlModule vs. use that will really bake your noodle. > > Attached is a full mod_perl trace, where I hacked into perl_require_module a > dump of one of my vanishing namespaces on each module load. So you can > definitely see it's loading up properly. All is well until you reach the end, > where it starts reloading everything, and the namespace is now completely gone. > Exporter predictably throws an error at this point (since @EXPORT_OK is now > undefined) and the server fails to start. OTOH, one wonders how Exporter got > the call, since @ISA is gone as well. > > I posted this once before and got blown off -- a pox on to those who tell me to > check @INC or what not. There is nothing wrong with my code or my setup -- the > only change I have to make to get things working is to `use` all the PerlModule > modules in startup.pl first, before they're called with PerlModule. And don't > tell me to just use `use` either -- they all load up Apache config directives, > so PerlModule has to be called at some point so Apache recognizes the > directives. But if things were working as advertised, PerlModule would be all I > need. > > The two obvious questions: > > 1) Where the fsck did everything go? > 2) Why does this only emanate when stuff is loaded up via PerlModule? I mean, >look at perl_require_module -- all it does is `eval "require $foo"`. Hard to >go wrong there. I'm willing to bet that this is the Known Nasty having to do with how Apache re-reloads modules. Are you using mod_perl as a DSO? If so, have you tried it statically? -- Daniel Jacobowitz Carnegie Mellon University MontaVista Software Debian GNU/Linux Developer