[R] Segmentation fault/buffer overflow with fix() in Fedora Core 5 from Extras repository

2006-10-19 Thread Paul Johnson 
The Fedora Extras update of R found its way onto my systems today and
I noted that fix() and edit() no longer work. There is a program crash
that closes up R, but it does not leave a core file.   I've tested by
turning off SELinux, it had no effect.

Do you see it too?  What do you think?  It happens on both systems
I've tested. As far as I know, both of these systems are up-to-date.

I restarted with R -d gdb to try to get a backtrace, but gdb says
the debugging symbols have been removed and I don't see the
debuginfo package on the Extras archive.  I'm attaching the gdb info
later, but  I don't think it helps much without line numbers..

I think my next step will be to re-build R on these systems and see if
the problem disappears. Right? If it still crashes, I'll make sure I
have debugging symbols and give you a full backtrace.  If it does not
crash, I'll let you know as well


Here's the session that crashes


 library(car)
 data(Chile)
 edit(Chile)
*** buffer overflow detected ***: /usr/lib/R/bin/exec/R terminated
=== Backtrace: =
/lib/libc.so.6(__chk_fail+0x29)[0xa8079d]
/lib/libc.so.6[0xa8195d]
/usr/lib/R/modules//R_X11.so[0x7c094a]
/usr/lib/R/modules//R_X11.so[0x7c20dd]
/usr/lib/R/modules//R_X11.so[0x7c3428]
/usr/lib/R/modules//R_X11.so(RX11_dataentry+0xa25)[0x7c4b15]
/usr/lib/R/lib/libR.so[0x2bf4c5]
/usr/lib/R/lib/libR.so[0x1dfd26]
/usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
/usr/lib/R/lib/libR.so[0x1b4d28]
/usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
/usr/lib/R/lib/libR.so[0x1b1887]
/usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
/usr/lib/R/lib/libR.so(Rf_applyClosure+0x2a7)[0x1b2f67]
/usr/lib/R/lib/libR.so[0x1e146f]
/usr/lib/R/lib/libR.so(Rf_usemethod+0x609)[0x1e28d9]
/usr/lib/R/lib/libR.so[0x1e30ae]
/usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
/usr/lib/R/lib/libR.so(Rf_applyClosure+0x2a7)[0x1b2f67]
/usr/lib/R/lib/libR.so(Rf_eval+0x2f4)[0x1b07e4]
/usr/lib/R/lib/libR.so(Rf_ReplIteration+0x311)[0x1d01b1]
/usr/lib/R/lib/libR.so[0x1d03c1]
/usr/lib/R/lib/libR.so(run_Rmainloop+0x60)[0x1d0710]
/usr/lib/R/lib/libR.so(Rf_mainloop+0x1c)[0x1d073c]
/usr/lib/R/bin/exec/R(main+0x46)[0x8048696]
/lib/libc.so.6(__libc_start_main+0xc6)[0x9c41fe]
/usr/lib/R/bin/exec/R[0x8048591]
=== Memory map: 
0011-00329000 r-xp  08:05 553625 /usr/lib/R/lib/libR.so
00329000-00336000 rwxp 00219000 08:05 553625 /usr/lib/R/lib/libR.so
00336000-003cd000 rwxp 00336000 00:00 0
003cd000-003d5000 r-xp  08:05 683486 /lib/libnss_files-2.4.90.so
003d5000-003d6000 r-xp 7000 08:05 683486 /lib/libnss_files-2.4.90.so
003d7000-003f5000 r-xp  08:05 1045723
/usr/lib/R/library/grDevices/libs/grDevices.so
003f5000-003f6000 rwxp 0001d000 08:05 1045723
/usr/lib/R/library/grDevices/libs/grDevices.so
003f6000-003fc000 r-xp  08:05 1046746
/usr/lib/R/library/methods/libs/methods.so
003fc000-003fd000 rwxp 5000 08:05 1046746
/usr/lib/R/library/methods/libs/methods.so
003fd000-0040 r-xp  08:05 1050384
/usr/lib/R/library/tools/libs/tools.so
0040-00401000 rwxp 2000 08:05 1050384
/usr/lib/R/library/tools/libs/tools.so
00413000-0043d000 r-xp  08:05 553410 /usr/lib/R/lib/libRblas.so
0043d000-0043e000 rwxp 00029000 08:05 553410 /usr/lib/R/lib/libRblas.so
0043e000-004b9000 r-xp  08:05 2868184/usr/lib/libgfortran.so.1.0.0
004b9000-004ba000 rwxp 0007b000 08:05 2868184/usr/lib/libgfortran.so.1.0.0
004ba000-0050b000 r-xp  08:05 1049782
/usr/lib/R/library/stats/libs/stats.so
0050b000-0050d000 rwxp 0005 08:05 1049782
/usr/lib/R/library/stats/libs/stats.so
0051-00511000 r-xp 0051 00:00 0  [vdso]
00511000-0060a000 r-xp  08:05 2868912/usr/lib/libX11.so.6.2.0
0060a000-0060e000 rwxp 000f9000 08:05 2868912/usr/lib/libX11.so.6.2.0
00664000-0067b000 r-xp  08:05 683622 /lib/libpcre.so.0.0.1
0067b000-00692000 rwxp 00017000 08:05 683622 /lib/libpcre.so.0.0.1
007bb000-007d4000 r-xp  08:05 1050764/usr/lib/R/modules/R_X11.so
007d4000-007d5000 rwxp 00018000 08:05 1050764/usr/lib/R/modules/R_X11.so
007d5000-007e1000 rwxp 007d5000 00:00 0
00896000-008eb000 r-xp  08:05 2876525/usr/lib/libXt.so.6.0.0
008eb000-008ef000 rwxp 00054000 08:05 2876525/usr/lib/libXt.so.6.0.0
0099-009a7000 r-xp  08:05 683431 /lib/ld-2.4.90.so
009a7000-009a8000 r-xp 00017000 08:05 683431 /lib/ld-2.4.90.so
009a8000-009a9000 rwxp 00018000 08:05 683431 /lib/ld-2.4.90.so
009ab000-00acf000 r-xp  08:05 683432 /lib/libc-2.4.90.so
00acf000-00ad1000 r-xp 00124000 08:05 683432 /lib/libc-2.4.90.so
00ad1000-00ad2000 rwxp 00126000 08:05 683432 /lib/libc-2.4.90.so
00ad2000-00ad5000 rwxp 00ad2000 00:00 0
00ad7000-00afc000 r-xp  08:05 683433 /lib/libm-2.4.90.so
00afc000-00afd000 r-xp 00024000 08:05 683433 /lib/libm-2.4.90.so
00afd000-00afe000 rwxp 00025000 08:05 683433 /lib/libm-2.4.90.so
00b0-00b02000 r-xp  08:05 683435

Re: [R] Segmentation fault/buffer overflow with fix() in Fedora Core 5 from Extras repository

2006-10-19 Thread Prof Brian Ripley 
Is this in a UTF-8 locale?  If so, this is covered by Ei-ji Nakama's 
posting to both R-help and R-devel yesterday: see

https://stat.ethz.ch/pipermail/r-devel/2006-October/039792.html

You have three choices:

1) Use a single-byte locale.
2) Compile with the standard CFLAGS and not the extra flags used by FC.
3) Use R-patched, which has this fixed.

As my dept still sets Linux boxes up in en_GB and not en_GB.utf8, I am 
using workaround 1 and so took a while to work out what the problem might 
be.

What is happening is that FC sets CFLAGS to something other than the R 
default.  This enables extra checks on buffer overflow and stack-smashing, 
but unfortunately removes the flag -std=gnu99 that is needed to allow C99 
features to be used.  Those extra checks are triggered by a few places in 
the MBCS code that Mr Nakama contributed, and some of those were patched 
prior to the release of 2.4.0.  AFAIK the problems are not new but the 
detection has got more efficient.

It is very helpful to include a concise description of your environment. 
You only mentioned the OS in the subject line, never the architecture, 
exact version of R (let alone the exact RPM) nor the locale. 
sessionInfo() provides such information in a compact form.


On Thu, 19 Oct 2006, Paul Johnson wrote:

 The Fedora Extras update of R found its way onto my systems today and
 I noted that fix() and edit() no longer work. There is a program crash
 that closes up R, but it does not leave a core file.   I've tested by
 turning off SELinux, it had no effect.

 Do you see it too?  What do you think?  It happens on both systems
 I've tested. As far as I know, both of these systems are up-to-date.

 I restarted with R -d gdb to try to get a backtrace, but gdb says
 the debugging symbols have been removed and I don't see the
 debuginfo package on the Extras archive.  I'm attaching the gdb info
 later, but  I don't think it helps much without line numbers..

 I think my next step will be to re-build R on these systems and see if
 the problem disappears. Right? If it still crashes, I'll make sure I
 have debugging symbols and give you a full backtrace.  If it does not
 crash, I'll let you know as well


 Here's the session that crashes


 library(car)
 data(Chile)
 edit(Chile)
 *** buffer overflow detected ***: /usr/lib/R/bin/exec/R terminated
 === Backtrace: =
 /lib/libc.so.6(__chk_fail+0x29)[0xa8079d]
 /lib/libc.so.6[0xa8195d]
 /usr/lib/R/modules//R_X11.so[0x7c094a]
 /usr/lib/R/modules//R_X11.so[0x7c20dd]
 /usr/lib/R/modules//R_X11.so[0x7c3428]
 /usr/lib/R/modules//R_X11.so(RX11_dataentry+0xa25)[0x7c4b15]
 /usr/lib/R/lib/libR.so[0x2bf4c5]
 /usr/lib/R/lib/libR.so[0x1dfd26]
 /usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
 /usr/lib/R/lib/libR.so[0x1b4d28]
 /usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
 /usr/lib/R/lib/libR.so[0x1b1887]
 /usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
 /usr/lib/R/lib/libR.so(Rf_applyClosure+0x2a7)[0x1b2f67]
 /usr/lib/R/lib/libR.so[0x1e146f]
 /usr/lib/R/lib/libR.so(Rf_usemethod+0x609)[0x1e28d9]
 /usr/lib/R/lib/libR.so[0x1e30ae]
 /usr/lib/R/lib/libR.so(Rf_eval+0x483)[0x1b0973]
 /usr/lib/R/lib/libR.so(Rf_applyClosure+0x2a7)[0x1b2f67]
 /usr/lib/R/lib/libR.so(Rf_eval+0x2f4)[0x1b07e4]
 /usr/lib/R/lib/libR.so(Rf_ReplIteration+0x311)[0x1d01b1]
 /usr/lib/R/lib/libR.so[0x1d03c1]
 /usr/lib/R/lib/libR.so(run_Rmainloop+0x60)[0x1d0710]
 /usr/lib/R/lib/libR.so(Rf_mainloop+0x1c)[0x1d073c]
 /usr/lib/R/bin/exec/R(main+0x46)[0x8048696]
 /lib/libc.so.6(__libc_start_main+0xc6)[0x9c41fe]
 /usr/lib/R/bin/exec/R[0x8048591]
 === Memory map: 
 0011-00329000 r-xp  08:05 553625 /usr/lib/R/lib/libR.so
 00329000-00336000 rwxp 00219000 08:05 553625 /usr/lib/R/lib/libR.so
 00336000-003cd000 rwxp 00336000 00:00 0
 003cd000-003d5000 r-xp  08:05 683486 /lib/libnss_files-2.4.90.so
 003d5000-003d6000 r-xp 7000 08:05 683486 /lib/libnss_files-2.4.90.so
 003d7000-003f5000 r-xp  08:05 1045723
 /usr/lib/R/library/grDevices/libs/grDevices.so
 003f5000-003f6000 rwxp 0001d000 08:05 1045723
 /usr/lib/R/library/grDevices/libs/grDevices.so
 003f6000-003fc000 r-xp  08:05 1046746
 /usr/lib/R/library/methods/libs/methods.so
 003fc000-003fd000 rwxp 5000 08:05 1046746
 /usr/lib/R/library/methods/libs/methods.so
 003fd000-0040 r-xp  08:05 1050384
 /usr/lib/R/library/tools/libs/tools.so
 0040-00401000 rwxp 2000 08:05 1050384
 /usr/lib/R/library/tools/libs/tools.so
 00413000-0043d000 r-xp  08:05 553410 /usr/lib/R/lib/libRblas.so
 0043d000-0043e000 rwxp 00029000 08:05 553410 /usr/lib/R/lib/libRblas.so
 0043e000-004b9000 r-xp  08:05 2868184/usr/lib/libgfortran.so.1.0.0
 004b9000-004ba000 rwxp 0007b000 08:05 2868184/usr/lib/libgfortran.so.1.0.0
 004ba000-0050b000 r-xp  08:05 1049782
 /usr/lib/R/library/stats/libs/stats.so
 0050b000-0050d000 rwxp 0005 08:05 1049782
 /usr/lib/R/library/stats/libs/stats.so
 0051-00511000