I don't have any RHEL5.x servers at my disposal but, based on what I know about perl and RHEL4, from your strace output it looks like your perl installation may be questionable or at fault here. I recommend verifying your perl-5.8.8 RPM. Don't be surprised if files show up as "missing". When you install updated perl modules, the install may choose to uninstall the old versions. If there are "missing" files from the RPM, check in "vendor_perl" and "site_perl" to make sure they're installed elsewhere.
I have experienced similar segfaults from Apache-2.0 when a server is short on memory because I have set vm.overcommit_memory=2 and vm.overcommit_ratio=100 on my RHEL4 servers. This essentially puts a limit on how much memory the kernel can claim to have and thereby forces malloc() to return NULL if the server doesn't have enough free memory to allocate. It seems that Apache-2.0 doesn't know how to deal with the inability to allocate memory; it's possible that this bug is still present in Apache-2.2. Have you attempted to allow Apache to leave coredumps? In Apache-2.0 this was done by setting the "CoreDumpDirectory" to some directory with write permissions for the appropriate User in the httpd.conf and also setting the "DAEMON_COREFILE_LIMIT" environment variable to "unlimited" in the /etc/sysconfig/httpd file. I assume it's not too different with Apache-2.2 and RHEL5. If you can get Apache to dump core, you can backtrace with gdb. I'm sorry I cannot be of more assistance, but I hope this helps somewhat. --phess -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Aleksey Tsalolikhin Sent: Friday, January 21, 2011 4:26 PM To: rt-users Subject: Re: [rt-users] apache httpd segfaults On Fri, Jan 21, 2011 at 4:12 PM, Aleksey Tsalolikhin <[email protected]> wrote: > My RT 3.8.8 has lost its prettiness. It's text-based now, no colors > or graphics, only text. It does work, but it isn't pretty. No > buttons, only links. > > The only errors I can find are in the Apache httpd log, and it seems > each time I click on a link within RT, I get a message like this in my > httpd error log: > > [Fri Jan 21 16:05:06 2011] [notice] child pid 32708 exit signal > Segmentation fault (11) > > I use postgres 8.4 on the backend. > > Would appreciate any suggestions for troubleshooting this. This is on CentOS 5.5 and Apache httpd 2.2 strace of the apache httpd process shows the following: open("/usr/lib/perl5/5.8.8/PerlIO.pm", O_RDONLY) = 60 ioctl(60, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff0aee6850) = -1 ENOTTY (Inappropriate ioctl for device) lseek(60, 0, SEEK_CUR) = 0 read(60, "package PerlIO;\n\nour $VERSION = "..., 4096) = 4096 lseek(60, 382, SEEK_SET) = 382 lseek(60, 0, SEEK_CUR) = 382 close(60) = 0 stat("/opt/rt3/bin/../local/lib/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/opt/rt3/bin/../local/lib/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/opt/rt3/bin/../lib/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/opt/rt3/bin/../lib/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/PerlIO/scal ar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/PerlIO/scal ar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/site_perl/5.8.8/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/site_perl/5.8.8/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/site_perl/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/site_perl/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/PerlIO/sc alar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/PerlIO/sc alar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/vendor_perl/5.8.8/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/vendor_perl/5.8.8/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/vendor_perl/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib/perl5/vendor_perl/PerlIO/scalar.pm", 0x7fff0aee6ac0) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/PerlIO/scalar.pmc", 0x7fff0aee6c00) = -1 ENOENT (No such file or directory) stat("/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/PerlIO/scalar.pm", {st_mode=S_IFREG|0644, st_size=979, ...}) = 0 open("/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/PerlIO/scalar.pm", O_RDONLY) = 60 ioctl(60, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff0aee6850) = -1 ENOTTY (Inappropriate ioctl for device) lseek(60, 0, SEEK_CUR) = 0 read(60, "package PerlIO::scalar;\nour $VER"..., 4096) = 979 lseek(60, 108, SEEK_SET) = 108 lseek(60, 0, SEEK_CUR) = 108 close(60) = 0 stat("/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/PerlIO/scalar/sc alar.so", {st_mode=S_IFREG|0755, st_size=11024, ...}) = 0 stat("/usr/lib64/perl5/5.8.8/x86_64-linux-thread-multi/auto/PerlIO/scalar/sc alar.bs", 0x2b8fb08059a0) = -1 ENOENT (No such file or directory) --- SIGSEGV (Segmentation fault) @ 0 (0) --- chdir("/etc/httpd") = 0 rt_sigaction(SIGSEGV, {SIG_DFL, [], SA_RESTORER|SA_INTERRUPT, 0x2b8f9056bb10}, {SIG_DFL, [], SA_RESTORER|SA_RESETHAND, 0x2b8f9056bb10}, 8) = 0 kill(8729, SIGSEGV) = 0 rt_sigreturn(0x2219) = 0 --- SIGSEGV (Segmentation fault) @ 0 (0) --- my httpd config for rt has not changed, I still have <Location /NoAuth/images> SetHandler default </Location>
