Re: [R] Spare some CPU cycles for testing lme?
Tamas K Papp [EMAIL PROTECTED] writes: On Mon, Sep 13, 2004 at 08:40:15AM -0400, Frank Samuelson wrote: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) Hi, It ran smoothly on my installation. version _ platform powerpc-apple-darwin6.8 Ditto (SuSE 9.1) platform x86_64-unknown-linux-gnu arch x86_64 os linux-gnu system x86_64, linux-gnu status Under development (unstable) major2 minor0.0 year 2004 month09 day 13 language R -- O__ Peter Dalgaard Blegdamsvej 3 c/ /'_ --- Dept. of Biostatistics 2200 Cph. N (*) \(*) -- University of Copenhagen Denmark Ph: (+45) 35327918 ~~ - ([EMAIL PROTECTED]) FAX: (+45) 35327907 __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Spare some CPU cycles for testing lme?
Hello, Frank Samuelson schrieb: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) I also tested your code by using R-1.91 under Debian Sid. After hundreds of iterations it ended up with the already noticed segmentation fault HTH Thomas __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Spare some CPU cycles for testing lme?
As others have said, this needs tools not CPU cycles: gctorture or valgrind. Valgrind found (after a few seconds and on the first pass) ==23057== Invalid read of size 4 ==23057==at 0x3CF4E645: ssc_symbolic_permute (Mutils.c:373) ==23057==by 0x3CF5BF75: ssclme_create (ssclme.c:168) ==23057==by 0x80AF5E8: do_dotcall (/users/ripley/R/svn/R-devel/src/main/dotcode.c:640) ==23057==by 0x80CFA84: Rf_eval (/users/ripley/R/svn/R-devel/src/main/eval.c:399) ==23057== Address 0x3C7F3BD0 is 4 bytes before a block of size 524 alloc'd ==23057==at 0x3C01CB56: calloc (in /opt/local/lib/valgrind/vgpreload_memcheck.so) ==23057==by 0x80F9EBE: R_chk_calloc (/users/ripley/R/svn/R-devel/src/main/memory.c:2151) ==23057==by 0x3CF4E515: ssc_symbolic_permute (Mutils.c:352) ==23057==by 0x3CF5BF75: ssclme_create (ssclme.c:168) ==23057== ==23057== Use of uninitialised value of size 8 ==23057==at 0x80C0137: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:160) ==23057==by 0x80BFA85: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:101) ==23057==by 0x80BFEB3: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:154) ==23057==by 0x816ED96: do_subset2_dflt (/users/ripley/R/svn/R-devel/src/main/subset.c:816) ==23057== ==23057== Conditional jump or move depends on uninitialised value(s) ==23057==at 0x3CF62229: ssclme_fitted (ssclme.c:1587) ==23057==by 0x80AF646: do_dotcall (/users/ripley/R/svn/R-devel/src/main/dotcode.c:646) ==23057==by 0x80CFA84: Rf_eval (/users/ripley/R/svn/R-devel/src/main/eval.c:399) ==23057==by 0x80D1D80: do_set (/users/ripley/R/svn/R-devel/src/main/eval.c:1280) which is pretty definitive evidence of a problem (possibly 2) in the code. I strongly recommend valgrind (http://valgrind.kde.org/) if you are using x86 Linux. It has found quite a few errors in R and in certain packages recently. The only thing to watch is that optimized BLASes will probably crash it. On Mon, 13 Sep 2004, Frank Samuelson wrote: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) The above code simply reruns the example from the lme help page a large number of times and returns a bunch of 0's, so you'll need to have the lme4 and Matrix packages installed. It might take a while to complete, but you can always nice it and let it run. I'm attempting to bootstrap lme() from the lme4 package, but it causes a segfault after a couple hundred iterations. This happens on my Linux x86 RedHat 7.3, 8.0, 9.0, FC1 systems w/ 1.9.1 and devel 2.0.0 (not all possible combinations actually tested.) I've communicated w/ Douglas Bates about this and he doesn't appear to have the problem. -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Spare some CPU cycles for testing lme?
Prof Brian D Ripley wrote: As others have said, this needs tools not CPU cycles: gctorture or valgrind. Valgrind found (after a few seconds and on the first pass) ==23057== Invalid read of size 4 ==23057==at 0x3CF4E645: ssc_symbolic_permute (Mutils.c:373) ==23057==by 0x3CF5BF75: ssclme_create (ssclme.c:168) ==23057==by 0x80AF5E8: do_dotcall (/users/ripley/R/svn/R-devel/src/main/dotcode.c:640) ==23057==by 0x80CFA84: Rf_eval (/users/ripley/R/svn/R-devel/src/main/eval.c:399) ==23057== Address 0x3C7F3BD0 is 4 bytes before a block of size 524 alloc'd ==23057==at 0x3C01CB56: calloc (in /opt/local/lib/valgrind/vgpreload_memcheck.so) ==23057==by 0x80F9EBE: R_chk_calloc (/users/ripley/R/svn/R-devel/src/main/memory.c:2151) ==23057==by 0x3CF4E515: ssc_symbolic_permute (Mutils.c:352) ==23057==by 0x3CF5BF75: ssclme_create (ssclme.c:168) ==23057== ==23057== Use of uninitialised value of size 8 ==23057==at 0x80C0137: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:160) ==23057==by 0x80BFA85: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:101) ==23057==by 0x80BFEB3: Rf_duplicate (/users/ripley/R/svn/R-devel/src/main/duplicate.c:154) ==23057==by 0x816ED96: do_subset2_dflt (/users/ripley/R/svn/R-devel/src/main/subset.c:816) ==23057== ==23057== Conditional jump or move depends on uninitialised value(s) ==23057==at 0x3CF62229: ssclme_fitted (ssclme.c:1587) ==23057==by 0x80AF646: do_dotcall (/users/ripley/R/svn/R-devel/src/main/dotcode.c:646) ==23057==by 0x80CFA84: Rf_eval (/users/ripley/R/svn/R-devel/src/main/eval.c:399) ==23057==by 0x80D1D80: do_set (/users/ripley/R/svn/R-devel/src/main/eval.c:1280) which is pretty definitive evidence of a problem (possibly 2) in the code. I strongly recommend valgrind (http://valgrind.kde.org/) if you are using x86 Linux. It has found quite a few errors in R and in certain packages recently. The only thing to watch is that optimized BLASes will probably crash it. You're right. I had (at least) two thinko's in that code. The problems show up in the lme4 package but the errors in the C code are in the Matrix package. I will upload a repaired version of the Matrix package. __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Spare some CPU cycles for testing lme?
On Sep 13, 2004, at 8:47 PM, Tamas K Papp wrote: It ran smoothly on my installation. version _ platform powerpc-apple-darwin6.8 arch powerpc os darwin6.8 system powerpc, darwin6.8 status major1 minor9.1 year 2004 month06 day 21 language R Typical lines from top (if it helps anything; around 45-50k iterations): 1225 R.bin 90.7% 2:55:12 162 1164 71.7M+ 13.7M 66.6M+ 228M+ 1225 R.bin 78.6% 3:18:27 162 1606 81.3M+ 13.7M 75.5M+ 234M+ I can confirm this. same installation __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
Re: [R] Spare some CPU cycles for testing lme?
On Mon, 2004-09-13 at 07:40, Frank Samuelson wrote: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) The above code simply reruns the example from the lme help page a large number of times and returns a bunch of 0's, so you'll need to have the lme4 and Matrix packages installed. It might take a while to complete, but you can always nice it and let it run. I'm attempting to bootstrap lme() from the lme4 package, but it causes a segfault after a couple hundred iterations. This happens on my Linux x86 RedHat 7.3, 8.0, 9.0, FC1 systems w/ 1.9.1 and devel 2.0.0 (not all possible combinations actually tested.) I've communicated w/ Douglas Bates about this and he doesn't appear to have the problem. Thanks for any help. -Frank Replicated here on FC 2 running: Version 1.9.1 Patched (2004-09-07) The backtrace from gdb follows. It would probably make sense to move this thread to r-devel. As you can see, it got through 10,546 runs before segfaulting. cc: to Doug. HTH, Marc Schwartz dump-sapply( 1:5, function(i) { + fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, +random = ~ IQ.ver.cen | schoolNR); + cat( ,i,\r) + 0 + }) 10546 Program received signal SIGSEGV, Segmentation fault. 0x00982242 in _int_free () from /lib/tls/libc.so.6 (gdb) bt #0 0x00982242 in _int_free () from /lib/tls/libc.so.6 #1 0x0098373b in free () from /lib/tls/libc.so.6 #2 0x080c4003 in ReleaseLargeFreeVectors () at memory.c:695 #3 0x080c5999 in RunGenCollect (size_needed=1144) at memory.c:1282 #4 0x080c7084 in R_gc_internal (size_needed=1144) at memory.c:1933 #5 0x080c6b67 in Rf_allocVector (type=16, length=2287) at memory.c:1788 #6 0x0809b6b1 in Rf_duplicate (s=0xbed5d40) at duplicate.c:150 #7 0x0809b335 in Rf_duplicate (s=0xb64a3bc) at duplicate.c:100 #8 0x0809b517 in Rf_duplicate (s=0xbed15a8) at duplicate.c:142 #9 0x0809b777 in Rf_duplicate (s=0xb3f6048) at duplicate.c:137 #10 0x080a796d in EnsureLocal (symbol=0x9719f38, rho=0xb64c2b8) at eval.c:765 #11 0x080a8691 in evalseq (expr=0x9719f38, rho=0xb64c2b8, forcelocal=1, tmploc=0xb64a340) at eval.c:1135 #12 0x080a8847 in applydefine (call=0x97f9818, op=0x9705264, args=0x97f9834, rho=0xb64c2b8) at eval.c:1199 #13 0x080a7055 in Rf_eval (e=0x97f9818, rho=0xb64c2b8) at eval.c:375 #14 0x080a8436 in do_begin (call=0x97f97fc, op=0x9705168, args=0x97f97e0, rho=0xb64c2b8) at eval.c:1046 #15 0x080a7055 in Rf_eval (e=0x97f97fc, rho=0xb64c2b8) at eval.c:375 #16 0x080a7055 in Rf_eval (e=0x97f9bdc, rho=0xb64c2b8) at eval.c:375 #17 0x080a8436 in do_begin (call=0x97f4ac8, op=0x9705168, args=0x97f9bc0, rho=0xb64c2b8) at eval.c:1046 #18 0x080a7055 in Rf_eval (e=0x97f4ac8, rho=0xb64c2b8) at eval.c:375 #19 0x080a7301 in Rf_applyClosure (call=0xb64c130, op=0x97f9a8c, arglist=0xb64c050, rho=0xb65989c, suppliedenv=0xb64c0f8) at eval.c:566 #20 0x080cd2fb in applyMethod (call=0xb64c130, op=0x97f9a8c, args=0xb64c050, rho=0xb65989c, newrho=0xb64c0f8) at objects.c:119 #21 0x080cd985 in Rf_usemethod (generic=0x81a1f2b [, obj=0xb3ebbc0, call=0x9a668f0, args=0xb64c050, rho=0xb65989c, callrho=0xb65989c, defrho=0x96f6338, ans=0xfef58728) at objects.c:326 #22 0x080a994d in Rf_DispatchOrEval (call=0x9a668f0, op=0x9705050, generic=0x81a1f2b [, args=0xb64bffc, rho=0xb65989c, ans=0xfef58728, dropmissing=0, argsevald=1) at eval.c:1719 #23 0x0811d5aa in do_subset (call=0x9a668f0, op=0x9705050, args=0x9a66944, rho=0xb65989c) at subset.c:504 #24 0x080a7055 in Rf_eval (e=0x9a668f0, rho=0xb65989c) at eval.c:375 #25 0x080a8bc3 in do_set (call=0x9a66880, op=0x9705264, args=0x9a668b8, rho=0xb65989c) at eval.c:1271 #26 0x080a7055 in Rf_eval (e=0x9a66880, rho=0xb65989c) at eval.c:375 ---Type return to continue, or q return to quit--- #27 0x080a8436 in do_begin (call=0x9a67184, op=0x9705168, args=0x9a66864, rho=0xb65989c) at eval.c:1046 #28 0x080a7055 in Rf_eval (e=0x9a67184, rho=0xb65989c) at eval.c:375 #29 0x080a7055 in Rf_eval (e=0x9a66c44, rho=0xb65989c) at eval.c:375 #30 0x080a8436 in do_begin (call=0x9a66b10, op=0x9705168, args=0x9a66c28, rho=0xb65989c) at eval.c:1046 #31 0x080a7055 in Rf_eval (e=0x9a66b10, rho=0xb65989c) at eval.c:375 #32 0x080a7301 in Rf_applyClosure (call=0xb659730, op=0x9a67900, arglist=0xb65a558, rho=0xb65966c, suppliedenv=0xb6596f8) at eval.c:566 #33 0x080cd2fb in applyMethod (call=0xb659730, op=0x9a67900, args=0xb65a558, rho=0xb65966c, newrho=0xb6596f8) at objects.c:119 #34 0x080cd985 in Rf_usemethod (generic=0x9b1f300 model.matrix, obj=0xb415a64, call=0x9a6783c, args=0x96f6338, rho=0xb65966c, callrho=0xb415bb4, defrho=0xa038100,
Re: [R] Spare some CPU cycles for testing lme?
Marc Schwartz wrote: On Mon, 2004-09-13 at 07:40, Frank Samuelson wrote: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) The above code simply reruns the example from the lme help page a large number of times and returns a bunch of 0's, so you'll need to have the lme4 and Matrix packages installed. It might take a while to complete, but you can always nice it and let it run. I'm attempting to bootstrap lme() from the lme4 package, but it causes a segfault after a couple hundred iterations. This happens on my Linux x86 RedHat 7.3, 8.0, 9.0, FC1 systems w/ 1.9.1 and devel 2.0.0 (not all possible combinations actually tested.) I've communicated w/ Douglas Bates about this and he doesn't appear to have the problem. Thanks for any help. -Frank Replicated here on FC 2 running: Version 1.9.1 Patched (2004-09-07) The backtrace from gdb follows. It would probably make sense to move this thread to r-devel. As you can see, it got through 10,546 runs before segfaulting. cc: to Doug. HTH, Marc Schwartz dump-sapply( 1:5, function(i) { + fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, +random = ~ IQ.ver.cen | schoolNR); + cat( ,i,\r) + 0 + }) 10546 Program received signal SIGSEGV, Segmentation fault. 0x00982242 in _int_free () from /lib/tls/libc.so.6 (gdb) bt #0 0x00982242 in _int_free () from /lib/tls/libc.so.6 #1 0x0098373b in free () from /lib/tls/libc.so.6 #2 0x080c4003 in ReleaseLargeFreeVectors () at memory.c:695 #3 0x080c5999 in RunGenCollect (size_needed=1144) at memory.c:1282 #4 0x080c7084 in R_gc_internal (size_needed=1144) at memory.c:1933 #5 0x080c6b67 in Rf_allocVector (type=16, length=2287) at memory.c:1788 #6 0x0809b6b1 in Rf_duplicate (s=0xbed5d40) at duplicate.c:150 #7 0x0809b335 in Rf_duplicate (s=0xb64a3bc) at duplicate.c:100 #8 0x0809b517 in Rf_duplicate (s=0xbed15a8) at duplicate.c:142 #9 0x0809b777 in Rf_duplicate (s=0xb3f6048) at duplicate.c:137 #10 0x080a796d in EnsureLocal (symbol=0x9719f38, rho=0xb64c2b8) at eval.c:765 #11 0x080a8691 in evalseq (expr=0x9719f38, rho=0xb64c2b8, forcelocal=1, tmploc=0xb64a340) at eval.c:1135 #12 0x080a8847 in applydefine (call=0x97f9818, op=0x9705264, args=0x97f9834, rho=0xb64c2b8) at eval.c:1199 #13 0x080a7055 in Rf_eval (e=0x97f9818, rho=0xb64c2b8) at eval.c:375 #14 0x080a8436 in do_begin (call=0x97f97fc, op=0x9705168, args=0x97f97e0, rho=0xb64c2b8) at eval.c:1046 #15 0x080a7055 in Rf_eval (e=0x97f97fc, rho=0xb64c2b8) at eval.c:375 #16 0x080a7055 in Rf_eval (e=0x97f9bdc, rho=0xb64c2b8) at eval.c:375 #17 0x080a8436 in do_begin (call=0x97f4ac8, op=0x9705168, args=0x97f9bc0, rho=0xb64c2b8) at eval.c:1046 #18 0x080a7055 in Rf_eval (e=0x97f4ac8, rho=0xb64c2b8) at eval.c:375 #19 0x080a7301 in Rf_applyClosure (call=0xb64c130, op=0x97f9a8c, arglist=0xb64c050, rho=0xb65989c, suppliedenv=0xb64c0f8) at eval.c:566 #20 0x080cd2fb in applyMethod (call=0xb64c130, op=0x97f9a8c, args=0xb64c050, rho=0xb65989c, newrho=0xb64c0f8) at objects.c:119 #21 0x080cd985 in Rf_usemethod (generic=0x81a1f2b [, obj=0xb3ebbc0, call=0x9a668f0, args=0xb64c050, rho=0xb65989c, callrho=0xb65989c, defrho=0x96f6338, ans=0xfef58728) at objects.c:326 #22 0x080a994d in Rf_DispatchOrEval (call=0x9a668f0, op=0x9705050, generic=0x81a1f2b [, args=0xb64bffc, rho=0xb65989c, ans=0xfef58728, dropmissing=0, argsevald=1) at eval.c:1719 #23 0x0811d5aa in do_subset (call=0x9a668f0, op=0x9705050, args=0x9a66944, rho=0xb65989c) at subset.c:504 #24 0x080a7055 in Rf_eval (e=0x9a668f0, rho=0xb65989c) at eval.c:375 #25 0x080a8bc3 in do_set (call=0x9a66880, op=0x9705264, args=0x9a668b8, rho=0xb65989c) at eval.c:1271 #26 0x080a7055 in Rf_eval (e=0x9a66880, rho=0xb65989c) at eval.c:375 ---Type return to continue, or q return to quit--- #27 0x080a8436 in do_begin (call=0x9a67184, op=0x9705168, args=0x9a66864, rho=0xb65989c) at eval.c:1046 #28 0x080a7055 in Rf_eval (e=0x9a67184, rho=0xb65989c) at eval.c:375 #29 0x080a7055 in Rf_eval (e=0x9a66c44, rho=0xb65989c) at eval.c:375 #30 0x080a8436 in do_begin (call=0x9a66b10, op=0x9705168, args=0x9a66c28, rho=0xb65989c) at eval.c:1046 #31 0x080a7055 in Rf_eval (e=0x9a66b10, rho=0xb65989c) at eval.c:375 #32 0x080a7301 in Rf_applyClosure (call=0xb659730, op=0x9a67900, arglist=0xb65a558, rho=0xb65966c, suppliedenv=0xb6596f8) at eval.c:566 #33 0x080cd2fb in applyMethod (call=0xb659730, op=0x9a67900, args=0xb65a558, rho=0xb65966c, newrho=0xb6596f8) at objects.c:119 #34 0x080cd985 in Rf_usemethod (generic=0x9b1f300 model.matrix, obj=0xb415a64, call=0x9a6783c, args=0x96f6338, rho=0xb65966c, callrho=0xb415bb4, defrho=0xa038100, ans=0xfef59288) at objects.c:326 #35
Re: [R] Spare some CPU cycles for testing lme?
On Mon, Sep 13, 2004 at 08:40:15AM -0400, Frank Samuelson wrote: If anyone has a few extra CPU cycles to spare, I'd appreciate it if you could verify a problem that I have encountered. Run the code below and tell me if it crashes your R before completion. library(lme4) data(bdf) dump-sapply( 1:5, function(i) { fm - lme(langPOST ~ IQ.ver.cen + avg.IQ.ver.cen, data = bdf, random = ~ IQ.ver.cen | schoolNR); cat( ,i,\r) 0 }) Hi, It ran smoothly on my installation. version _ platform powerpc-apple-darwin6.8 arch powerpc os darwin6.8 system powerpc, darwin6.8 status major1 minor9.1 year 2004 month06 day 21 language R Typical lines from top (if it helps anything; around 45-50k iterations): 1225 R.bin 90.7% 2:55:12 162 1164 71.7M+ 13.7M 66.6M+ 228M+ 1225 R.bin 78.6% 3:18:27 162 1606 81.3M+ 13.7M 75.5M+ 234M+ Best, Tamas __ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html