[mp2] [BUG] Segfault with Test::More - duping STDOUT?
(I have read the bug reporting guides but unfortunately cannot follow all the
instructions because I am missing Apache::TestReportPerl and do not have the
necessary permissions to install it, nor to rebuild perl with debugging turned
on. Hopefully I have provided enough information.)
I have discovered that simply "use"ing Test::More in a mod_perl script causes
my apache to segfault. I originally reported this to the author of Test::More
(see https://rt.cpan.org/Public/Bug/Display.html?id=69687) but now believe the
problem to have wider scope. By progressively removing code from my original
example I have reduced it to the trivial example that follows.
To REPRODUCE the problem, you need a Broken.pm module like this:
package Broken;
my $Testout;
create(); sub create { # Insert "{#" at the start of this line and the
segfault goes away
open( $Testout, ">&STDOUT" ) or die "Can't dup STDOUT: $!";
}
and a script like this:
#!/usr/bin/perl -w
use Broken;
print "Content-Type: text/html\r\n\r\nhello ";
open TEST, "|/bin/cat";
close TEST;
print "world\n";
Holding down F5 to generate reloads soon causes a segfault (my server is
configured for five mod_perl processes).
Note that if the code to dup STDOUT is not in a sub then the fault seems to go
away.
Apache, mod_perl and perl version info appended.
I have found a similar(ish) report at
http://http://stackoverflow.com/questions/4413411/ but no resolution. Note that
removing from my example the code that opens the pipe to cat, solves the
problem - this is the similarity here (and I realise that's likely a red
herring).
Merlyn Kline
Apache/2.2.15 (Unix) DAV/2 mod_ssl/2.2.15 OpenSSL/1.0.0-fips mod_perl/2.0.4
Perl/v5.10.1 configured -- resuming normal operations
Summary of my perl5 (revision 5 version 10 subversion 1) configuration:
Platform:
osname=linux, osvers=2.6.18-194.26.1.el5, archname=x86_64-linux-thread-multi
uname='linux x86-003.build.bos.redhat.com 2.6.18-194.26.1.el5 #1 smp fri
oct 29 14:21:16 edt 2010 x86_64 x86_64 x86_64 gnulinux '
config_args='-des -Doptimize=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic
-DDEBUGGING=-g -Dversion=5.10.1 -Dmyhostname=localhost
-Dperladmin=root@localhost -Dcc=gcc -Dcf_by=Red Hat, Inc. -Dprefix=/usr
-Dvendorprefix=/usr -Dsiteprefix=/usr/local -Dsitelib=/usr/local/share/perl5
-Dsitearch=/usr/local/lib64/perl5 -Dprivlib=/usr/share/perl5
-Darchlib=/usr/lib64/perl5 -Dvendorlib=/usr/share/perl5/vendor_perl
-Dvendorarch=/usr/lib64/perl5/vendor_perl -Dinc_version_list=5.10.0
-Darchname=x86_64-linux-thread-multi -Dlibpth=/usr/local/lib64 /lib64
/usr/lib64 -Duseshrplib -Dusethreads -Duseithreads -Duselargefiles -Dd_dosuid
-Dd_semctl_semun -Di_db -Ui_ndbm -Di_gdbm -Di_shadow -Di_syslog -Dman3ext=3pm
-Duseperlio -Dinstallusrbinperl=n -Ubincompat5005 -Uversiononly
-Dpager=/usr/bin/less -isr -Dd_gethostent_r_proto -Ud_endhostent_r_proto
-Ud_sethostent_r_proto -Ud_endprotoent_r_proto -Ud_setprotoent_r_proto
-Ud_endservent_r_proto -Ud_setservent_r_proto -Dscriptdir=/usr/bin'
hint=recommended, useposix=true, d_sigaction=define
useithreads=define, usemultiplicity=define
useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef
use64bitint=define, use64bitall=define, uselongdouble=undef
usemymalloc=n, bincompat5005=undef
Compiler:
cc='gcc', ccflags ='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include -D_LARGEFILE_SOURCE
-D_FILE_OFFSET_BITS=64',
optimize='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic',
cppflags='-D_REENTRANT -D_GNU_SOURCE -fno-strict-aliasing -pipe
-fstack-protector -I/usr/local/include'
ccversion='', gccversion='4.4.4 20100726 (Red Hat 4.4.4-13)',
gccosandvers=''
intsize=4, longsize=8, ptrsize=8, doublesize=8, byteorder=12345678
d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
ivtype='long', ivsize=8, nvtype='double', nvsize=8, Off_t='off_t',
lseeksize=8
alignbytes=8, prototype=define
Linker and Libraries:
ld='gcc', ldflags =' -fstack-protector'
libpth=/usr/local/lib64 /lib64 /usr/lib64
libs=-lresolv -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lpthread -lc
perllibs=-lresolv -lnsl -ldl -lm -lcrypt -lutil -lpthread -lc
libc=, so=so, useshrplib=true, libperl=libperl.so
gnulibc_version='2.12'
Dynamic Linking:
dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='-Wl,-E
-Wl,-rpath,/usr/lib64/perl5/CORE'
cccdlflags='-fPIC', lddlflags='-shared -O2 -g -pipe -Wall
-Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector
--param=ssp-buffer-size=4 -m64 -mtune=generic'
Characteristics of this binary (from libperl):
Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV
PERL_IMPL
Re: Is it me or is mod_perl extremely dangerous?
(cc'ing mod_perl list) On Tue, Dec 6, 2011 at 5:35 AM, Desilets, Alain wrote: > Thx to Fred, Andre and Jon for answering. BTW: I hope I didn't offend anyone. > What I really meant was: No offense taken at all. Welcome to mod_perl! > > "Is it me or is mod_perl *much more slippery than standard CGI*?" > > I know that there are lots of folks out there using mod_perl, so I assume it > can be used in a safe way. I am just trying to figure out where the slippery > patches are, and how easy it will be to slip on them for me, my team, my > external collaborators, and even writers of CPAN modules. > > The last point is important. Fred wrote: > > " Most developers find it useful to use CPAN modules which are generally high > quality." > > That's what I do also. In fact, I don't think I have ever used a third party > module that wasn't from CPAN, and they are, as you say, generally high > quality. > > However, even those high quality CPAN modules can fall flat on their face > when they are used in contexts that they weren't designed for. For example, I > recently discovered that several CPAN modules cannot be used safely in a > multithreaded or forked environment. Some of them fail "nicely" (i.e. they > KNOW they aren't supposed to be used in this kind of environment and tell you > at run time), but others just crash the process upon spawning a new fork or > thread, and leave you with no indication as to what caused this. I am > concerned that similar issues might arise when using certain CPAN modules in > a mod_perl context. > > --- > Question: Is this something that has been known to happen (CPAN modules that > don't work well in mod_perl), and if so, how common is it? > --- > > Coming back to where the slippery patches are. Initially, it looked to me > that the main thing to watch out for was global variables. By this, I mean > variables that can be seen from anywhere in the code. I was OK with that, > because I never use global variables, and I have taught everyone in my team > to stay away from them. Also, I don't see global variables being used in CPAN > modules. > > But after doing a couple experiments, I now realize that package variables > are also unsafe in a mod_perl environment. Although I try to avoid them, I > don't consider package variables to be "sloppy". Even though they are public > and can be seen outside of the package, they can only be seen by those files > which actually load the package. So, I am a bit more uncomfortable with that > particular slippery patch, because I know that I and some of my team members > use package variables occasionally. I also see lots of CPAN modules that use > them as well. > > One thing that I think will greatly mitigate the risk for us is that we use > Test Driven Development, and have PerlUnit tests for every class in our > application. This forces us to write our classes that don't assume they will > be used in a single CGI script call context. That's because the tests > process instantiated and uses a the class several times in a number of tests > and scenarios. But I think this will only help to a point. It's one thing to > have a process that runs dozens of tests on a class, but it's a completely > different thing to have a process which uses that class to execute thousands > of requests per day, and which stays up and running for months at a time. > > I guess the only way to find out whether mod_perl works for us is to try it, > while keeping the door for going back to standard CGI in case our app is too > brittle for it. > > Time to buy a mod_perl cookbook I guess ;-). > > Again, thx for your useful answers. > > Alain > > -Original Message- > From: Fred Moyer [mailto:f...@redhotpenguin.com] > Sent: Monday, December 05, 2011 4:45 PM > To: Desilets, Alain > Cc: mod_perl list > Subject: Re: Is it me or is mod_perl extremely dangerous? > > On Mon, Dec 5, 2011 at 1:06 PM, Desilets, Alain > wrote: >> I'm a complete newbie to mod_perl, and after reading the following >> documentation: >> >> http://perl.apache.org/docs/1.0/guide/porting.html >> >> I am scared witless by the fact that many variables don't get reinitialized >> between calls to the CGI scripts. >> >> Particularly scary is the example provided on that page, where the >> authentication status is stored in a global variable that doesn't get >> reinitialized. In that example, if Joe logs into the system, and Jane then >> runs the script, she can get access to the system also without every logging >> in, because Joe's authentication status is still there. YIKES! > > If you read through the entire example, you will see the point of the > example is to show what can happen from bad programming. > > "Because of sloppy programming, a global variable was not reset at the > beginning of the program and voila, you can easily peek into someone > else's email! Here is an example of sloppy code" > >> But I'm not convinced, because package variables are not reini
Re: Is it me or is mod_perl extremely dangerous?
> On Tue, Dec 6, 2011 at 5:35 AM, Desilets, Alain
> wrote:
>> However, even those high quality CPAN modules can fall flat on their face
>> when they are used in contexts that they weren't designed for. For example,
>> I recently discovered that several CPAN modules cannot be used safely in a
>> multithreaded or forked environment.
If you're running on a unix environment, you probably want to run
multi-process, not multithreaded. Most modules that work in CGI will
work here, but you need to be aware of issues with opening files and
sockets before the parent process forks. This is described in detail
in the docs. Is there a specific module that you need that isn't
working for you?
>> Coming back to where the slippery patches are. Initially, it looked to me
>> that the main thing to watch out for was global variables. By this, I mean
>> variables that can be seen from anywhere in the code. I was OK with that,
>> because I never use global variables, and I have taught everyone in my team
>> to stay away from them. Also, I don't see global variables being used in
>> CPAN modules.
>>
>> But after doing a couple experiments, I now realize that package variables
>> are also unsafe in a mod_perl environment.
Well, there's no real distinction between "globals" and "package
variables" in perl. I think that by globals you mean package
variables that are in the current package so they can be referenced
without a package prefix.
Package variables are indeed persistent in mod_perl and similar web
environments. You should only use them for things that you want to
persist between requests. Everything else goes in lexically-scoped
("my") variables.
- Perrin
Re: Is it me or is mod_perl extremely dangerous?
(cc'ing the mod_perl list on this one also, please remember to reply all :) On Tue, Dec 6, 2011 at 11:28 AM, Dr James A Smith wrote: > On 06/12/2011 19:09, Fred Moyer wrote: >> (cc'ing mod_perl list) >> >> On Tue, Dec 6, 2011 at 5:35 AM, Desilets, Alain >> wrote: >>> Thx to Fred, Andre and Jon for answering. BTW: I hope I didn't offend >>> anyone. What I really meant was: >> >> No offense taken at all. Welcome to mod_perl! >> >>> >>> "Is it me or is mod_perl *much more slippery than standard CGI*?" >>> >>> I know that there are lots of folks out there using mod_perl, so I assume >>> it can be used in a safe way. I am just trying to figure out where the >>> slippery patches are, and how easy it will be to slip on them for me, my >>> team, my external collaborators, and even writers of CPAN modules. >>> >>> The last point is important. Fred wrote: >>> >>> " Most developers find it useful to use CPAN modules which are generally >>> high quality." >>> >>> That's what I do also. In fact, I don't think I have ever used a third >>> party module that wasn't from CPAN, and they are, as you say, generally >>> high quality. >>> >>> However, even those high quality CPAN modules can fall flat on their face >>> when they are used in contexts that they weren't designed for. For example, >>> I recently discovered that several CPAN modules cannot be used safely in a >>> multithreaded or forked environment. Some of them fail "nicely" (i.e. they >>> KNOW they aren't supposed to be used in this kind of environment and tell >>> you at run time), but others just crash the process upon spawning a new >>> fork or thread, and leave you with no indication as to what caused this. I >>> am concerned that similar issues might arise when using certain CPAN >>> modules in a mod_perl context. >>> >>> --- >>> Question: Is this something that has been known to happen (CPAN modules >>> that don't work well in mod_perl), and if so, how common is it? >>> --- >>> >>> Coming back to where the slippery patches are. Initially, it looked to me >>> that the main thing to watch out for was global variables. By this, I mean >>> variables that can be seen from anywhere in the code. I was OK with that, >>> because I never use global variables, and I have taught everyone in my team >>> to stay away from them. Also, I don't see global variables being used in >>> CPAN modules. > > Perl::Critic is your friend in number of places - there are some subtle > gotchas that people don't realise that are more subtle than the ones > you mentioned - and perlcritic will point a lot of these out > > In a mod_perl registry script, what is wrong with the following: > > my $user = $self->user_from_cookie( $user_cookie ) if $user_cookie; > > ?
RE: Is it me or is mod_perl extremely dangerous?
> > On Tue, Dec 6, 2011 at 5:35 AM, Desilets, Alain
> > wrote:
> >> However, even those high quality CPAN modules can fall flat on their face
> >> when > they are used in contexts that they weren't designed for. For
> >> example, I recently > discovered that several CPAN modules cannot be used
> >> safely in a multithreaded or
> forked environment.
>
> If you're running on a unix environment, you probably want to run
> multi-process, not multithreaded. Most modules that work in CGI will
> work here, but you need to be aware of issues with opening files and
> sockets before the parent process forks. This is described in detail
> in the docs. Is there a specific module that you need that isn't
> working for you?
I'm using Windows. In my experience, modules that don't work well with
multithread, also don't work well with fork, and vice-versa.
The one module that I know for sure does not work in a forked or multi-threaded
environment is KinoSearch. Because of that, we had to move to the Lucy module
instead.
Last summer, I had also concluded (possibly falsely) that there were other
modules in my application which were not thread-safe. But I could not identify
which ones, because when I ran the app in multi-thread fashion, all I got was a
one line error message about not being able to free address such and such. I
assumed that this was due to some perl module that was accessing an external
DLL, but I can't be sure. I had googled the error message, and I seem to recall
that it had been linked to problems with non-thread safe modules. Can't
remember the details.
> Package variables are indeed persistent in mod_perl and similar web
> environments. You should only use them for things that you want to
> persist between requests. Everything else goes in lexically-scoped
> ("my") variables.
Not sure what you mean by "lexically scoped".
But declaring a variable as "my" is not sufficient to prevent it from being
persistent. You can declare a package variable with "my" and it will persist.
Alain
Re: Is it me or is mod_perl extremely dangerous?
On Tue, Dec 6, 2011 at 3:11 PM, Desilets, Alain
wrote:
> I'm using Windows. In my experience, modules that don't work well with
> multithread, also don't work well with fork, and vice-versa.
I wouldn't expect that. In a multi-process situation, files and
sockets are not shared unless you opened them before forking. DBI is
a classic example of something that works fine in multi-process
situations but may fail in multithreaded ones.
> The one module that I know for sure does not work in a forked or
> multi-threaded environment is KinoSearch.
I don't use KinoSearch so I don't know why it doesn't work, but I'd
suggest that when something seems to be not working it's worth
checking with the developer or mailing list for the module and
checking on this list if anyone is using it.
> Not sure what you mean by "lexically scoped".
You probably know all about them but haven't heard that term before.
Try http://perl.plover.com/FAQs/Namespaces.html for some explanations
and terminology.
> But declaring a variable as "my" is not sufficient to prevent it from being
> persistent. You can declare a package variable with "my" and it will persist.
That's true! I assume you mean this:
my $foo = 7;
sub print_foo {
print $foo;
}
That's called a closure. Don't do that unless you want persistence.
Using locally declared variables in subs and passing in arguments
instead of grabbing data directly from variables in a larger scope is
good programming practice, and it's the way to avoid unwanted variable
persistence in mod_perl and FastCGI.
- Perrin
RE: Is it me or is mod_perl extremely dangerous?
> > But declaring a variable as "my" is not sufficient to prevent it from being
> > persistent. You can declare a package variable with "my" and it will
> > persist.
>
> That's true! I assume you mean this:
>
> my $foo = 7;
>
> sub print_foo {
> print $foo;
> }
>
> That's called a closure. Don't do that unless you want persistence.
>
> Using locally declared variables in subs and passing in arguments
> instead of grabbing data directly from variables in a larger scope is
> good programming practice, and it's the way to avoid unwanted variable
> persistence in mod_perl and FastCGI.
No, I meant more things like this:
---
package MyClass;
my $class_level_attribute;
sub new {
my ($class) = @_;
$self->{instance_level_attribute} = undef;
bless $self, $class;
return $self;
}
The idea here is to emulate the concept of "static" (in Java or C++ parlance)
class level attributes, i.e., attributes which are tied to the the class itself
as opposed to instances of it.
In the above example, every instance of MyClass can have its own value of
instance_level_attribute. But there is only one shared value of
$class_level_attribute.
This is something that I use sparingly (for example, to track how many
instances of a particular class are generated) and AFAIK, it's not considered
"bad manners". But it seems like it could get me in trouble in the context of
mod_perl.
Note that the above not exactly the same as a static attribute in Java, in the
sense that the instance does not actually "inherit" the attribute. In other
words, you can't access $class_level_attribute by looking at
$self->{class_level_attribute}. But it's the closest thing I have found in Perl
for emulating Java style static class attributes.
Alain
Re: Is it me or is mod_perl extremely dangerous?
On Tue, Dec 6, 2011 at 3:55 PM, Desilets, Alain
wrote:
> No, I meant more things like this:
>
> ---
>
> package MyClass;
>
> my $class_level_attribute;
>
> sub new {
> my ($class) = @_;
> $self->{instance_level_attribute} = undef;
> bless $self, $class;
> return $self;
> }
Well, in this case, $lcass_level_attribute is technically not
persistent, because you didn't use it inside your sub. If you had
though, it would be persistent, just like my example. Both would be
closures.
> This is something that I use sparingly (for example, to track how many
> instances of a particular class are generated) and AFAIK, it's not considered
> "bad manners". But it seems like it could get me in trouble in the context of
> mod_perl.
It's fine to do that, but persistent is the desired effect there
because it's class-level data, so it shouldn't be unexpected.
> Note that the above not exactly the same as a static attribute in Java, in
> the sense that the instance does not actually "inherit" the attribute. In
> other words, you can't access $class_level_attribute by looking at
> $self->{class_level_attribute}. But it's the closest thing I have found in
> Perl for emulating Java style static class attributes.
If you really want that, you should look at Moose on CPAN.
- Perrin
