Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Fantastic. Thanks a lot, Luke! I can close the thread just to say the finalit wasn't Matrix after all! ;-) On Fri, Apr 8, 2011 at 20:48, luke-tier...@uiowa.edu wrote: Fixed in R-devel by 55385 and R-2-113-branch by 55386. The problem was a call to install() in the C srouce code without protecting another value before the install call. This particular issue was very unlikely to cause a problem outside of a gctorture context, but in that context with the memory checking enabled by the strict barrier it will get caught. Quick notes in case someone else needs to track this sort of thing down: When gctorture is used in conjuction with the memory checks enabled by the strict barrier it is very likely that an error will be detected and signaled very close to where the problem in the C code actually is, which is why I tried an example using only regexpr when the error was being signaled there. Once a reproducable example is found, setting a breakpoint in memory.c:CHK on the line that signals the error gives a stack trace of the C calls involved, and in this case the culprit was pretty easy to find at that point. luke On Fri, 8 Apr 2011, Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: l...@stat.uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Given the configure settings this looks like its probably an issue in regexpr's internals rather than in Matrix per se. I can confirm that with gctorture() for (i in 1:100) regexpr(package:, package:lattice, fixed = TRUE) Error in regexpr(package:, package:lattice, fixed = TRUE) : unprotected object (0x26b30a8) encountered (was INTSXP) run in r55347 with --enable-strict-barrier. I'll see if I can spot anyting. luke On Fri, 8 Apr 2011, Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: l...@stat.uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu__ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
--- On Fri, 8/4/11, Martin Maechler maech...@stat.math.ethz.ch wrote: Fantastic. Thanks a lot, Luke! I can close the thread just to say the final it wasn't Matrix after all! ;-) I am glad it is all working out - I have spent way too much time on this, having gone through about 4 different packages to get to the bottom. finger-cross gctorture(TRUE) works - and continue to :-)... until next time. Cheers, Hin-Tak On Fri, Apr 8, 2011 at 20:48, luke-tier...@uiowa.edu wrote: Fixed in R-devel by 55385 and R-2-113-branch by 55386. The problem was a call to install() in the C srouce code without protecting another value before the install call. This particular issue was very unlikely to cause a problem outside of a gctorture context, but in that context with the memory checking enabled by the strict barrier it will get caught. Quick notes in case someone else needs to track this sort of thing down: When gctorture is used in conjuction with the memory checks enabled by the strict barrier it is very likely that an error will be detected and signaled very close to where the problem in the C code actually is, which is why I tried an example using only regexpr when the error was being signaled there. Once a reproducable example is found, setting a breakpoint in memory.c:CHK on the line that signals the error gives a stack trace of the C calls involved, and in this case the culprit was pretty easy to find at that point. luke On Fri, 8 Apr 2011, Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics and Fax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: l...@stat.uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Fixed in R-devel by 55385 and R-2-113-branch by 55386. The problem was a call to install() in the C srouce code without protecting another value before the install call. This particular issue was very unlikely to cause a problem outside of a gctorture context, but in that context with the memory checking enabled by the strict barrier it will get caught. Quick notes in case someone else needs to track this sort of thing down: When gctorture is used in conjuction with the memory checks enabled by the strict barrier it is very likely that an error will be detected and signaled very close to where the problem in the C code actually is, which is why I tried an example using only regexpr when the error was being signaled there. Once a reproducable example is found, setting a breakpoint in memory.c:CHK on the line that signals the error gives a stack trace of the C calls involved, and in this case the culprit was pretty easy to find at that point. luke On Fri, 8 Apr 2011, Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Luke Tierney Statistics and Actuarial Science Ralph E. Wareham Professor of Mathematical Sciences University of Iowa Phone: 319-335-3386 Department of Statistics andFax: 319-335-3017 Actuarial Science 241 Schaeffer Hall email: l...@stat.uiowa.edu Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu__ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
(Resending with fewer recipients...) On Apr 8, 2011, at 07:09 , Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: Thanks. I wasn't expecting things to be different, just making a point about checking the right object. In principle, we could be putting in unstable development code in the trunk as soon as the branch was made. wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages Aha... Does it happen without --enable-byte-compiled-packages? To quote NEWS: by default the compiler is not used in this release so bugs in the compiler are interesting, but not release-critical. --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - -- Peter Dalgaard Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
On Fri, 8 Apr 2011, peter dalgaard wrote: (Resending with fewer recipients...) On Apr 8, 2011, at 07:09 , Hin-Tak Leung wrote: --- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: Thanks. I wasn't expecting things to be different, just making a point about checking the right object. In principle, we could be putting in unstable development code in the trunk as soon as the branch was made. And despite claims to the contrary earlier in this thread, we certainly did. (One example was in regexpr, although not AFAICS in a code branch used in this issue.) Once 2.x.0 branches, the trunk becomes a playpen for ideas considered too radical/experimental for a release a month or two off. wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages Note that the first define does nothing and the second just repeats what --enable-memory-profiling does. Neither are documented in current R AFAICS. Aha... Does it happen without --enable-byte-compiled-packages? To quote NEWS: by default the compiler is not used in this release so bugs in the compiler are interesting, but not release-critical. But --enable-byte-compiled-packages does nothing, as configure --help says in 2.13.0 RC (and AFAICS it is documented nowhere else). You really can't asumme that things not described in the R-admin manual are actually relevant to R (or current R). autoconf puts things in configure that are boilerplate code we do not use (and I've tried to indicate which ones come from libtool or libintl). --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - -- Peter Dalgaard Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595__ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
--- On Fri, 8/4/11, peter dalgaard pda...@gmail.com wrote: On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. Just so that there is no doubt, here is the recipe with the latest rc tar ball, cutting-and-pasting from my command history: wget -m http://cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd /tmp tar -zxpvf ~/cran.r-project.org/src/base-prerelease/R-rc_2011-04-07_r55373.tar.gz cd R-rc/ export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ; ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 ; make cd src/library/ cd Recommended/ ../../../bin/R CMD check --use-gct Matrix_0.999375-49.tar.gz -- ... Running examples in ‘Matrix-Ex.R’ failed The error occurred in: R version 2.13.0 RC (2011-04-07 r55373) Copyright (C) 2011 The R Foundation for Statistical Computing ISBN 3-900051-07-0 Platform: x86_64-unknown-linux-gnu (64-bit) ... pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3be2ba8) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted - __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
On Apr 7, 2011, at 23:57 , Hin-Tak Leung wrote: Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. Please do. It's the branch that is supposed to stabilize during prerelease times. Also, please check the prerelease tarballs, errors in make dist are not caught when building from svn. -pd -- Peter Dalgaard Center for Statistics, Copenhagen Business School Solbjerg Plads 3, 2000 Frederiksberg, Denmark Phone: (+45)38153501 Email: pd@cbs.dk Priv: pda...@gmail.com __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. Matrix-for-R-2.13@2659 (0.999375-49) has the problem at a different place: pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x2fe5818) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' That aside - with R 2.13 only about a week to go, which version of Matrix will it ship? Either way, since this problem causes dependent packages to crash on load, I have made some local changes to load Matrix only on demand and not on library initialization, since only one routine needs it. I thought I'd add a warning as well, but if R crashes, the warning may not come out in time so there is little incentive of emitting a warning which may not be seen. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 7 Apr 2011 07:51:44 +0100 writes: HL Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. HL Matrix-for-R-2.13@2659 (0.999375-49) has the problem at a different place: pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') HL Loading required package: lattice HL Error in regexpr(package:, envName, fixed = TRUE) : HL unprotected object (0x2fe5818) encountered (was INTSXP) HL Error: package/namespace load failed for 'Matrix' HL That aside - with R 2.13 only about a week to go, which version of Matrix will HL it ship? Well, R-2.13.0 is in Release Candidate ('RC') stage, after having passed the alpha and beta stages. So, almost surely, the current CRAN version of Matrix, also part of all recent R-2.13.0 tarballs will be shipped along. If we can reproduce your problem, we will fix it, hopefully pretty soon, and release a version of Matrix, 0.9996875-0 close to the release time of R-2.13.0; but for stability reasons (of R-2.13.0 and all its testing on all platforms), it would not ship with 2.13.0 and only people who explicitly update.packages(...) [after R-2.13.0 installation] will get it. Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Martin Maechler maech...@stat.math.ethz.ch on Thu, 7 Apr 2011 12:24:32 +0200 writes: HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 7 Apr 2011 07:51:44 +0100 writes: HL Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. HL Matrix-for-R-2.13@2659 (0.999375-49) has the problem at a different place: Are you sure that you really took 0.999375-49 and not its predecessor ??? I haven't been able to reproduce the problem (on 32-bit Linux, R-2.13.0 RC, after running your code snippet below for a couple of hours, I got the prompt back, and things continued working..) and Doug Bates has had a strong suspicion that the error message below looks like the problem that he fixed just between *-48 and *-49. pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') HL Loading required package: lattice HL Error in regexpr(package:, envName, fixed = TRUE) : HL unprotected object (0x2fe5818) encountered (was INTSXP) HL Error: package/namespace load failed for 'Matrix' Can you give more exact info about the platform ? Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Martin Maechler wrote: Martin Maechler maech...@stat.math.ethz.ch on Thu, 7 Apr 2011 12:24:32 +0200 writes: HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 7 Apr 2011 07:51:44 +0100 writes: HL Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. HL Matrix-for-R-2.13@2659 (0.999375-49) has the problem at a different place: Are you sure that you really took 0.999375-49 and not its predecessor ??? The two errors look similiar but different (the previous one was about a REALSXP), and both are fairly reproducible. As much as I can be sure, I guess. I overwrote src/library/Recommended and library/Matrix to be sure. I haven't been able to reproduce the problem (on 32-bit Linux, R-2.13.0 RC, after running your code snippet below for a couple of hours, I got the prompt back, and things continued working..) and Doug Bates has had a strong suspicion that the error message below looks like the problem that he fixed just between *-48 and *-49. pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') HL Loading required package: lattice HL Error in regexpr(package:, envName, fixed = TRUE) : HL unprotected object (0x2fe5818) encountered (was INTSXP) HL Error: package/namespace load failed for 'Matrix' Can you give more exact info about the platform ? fedora 14 x86_64. export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ./configure --enable-memory-profiling --enable-strict-barrier \ --enable-byte-compiled-packages --with-valgrind-instrumentation=2 Matrix-for-R-2.13@2659 was the one I was playing with. I am currently testing Matrix trunk@2666, since you seems to have put something relevant in r2661. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 07 Apr 2011 19:31:35 +0100 writes: HL Martin Maechler wrote: Martin Maechler maech...@stat.math.ethz.ch on Thu, 7 Apr 2011 12:24:32 +0200 writes: HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 7 Apr 2011 07:51:44 +0100 writes: HL Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. HL Matrix-for-R-2.13@2659 (0.999375-49) has the problem at HL a different place: Are you sure that you really took 0.999375-49 and not its predecessor ??? HL The two errors look similiar but different (the previous HL one was about a REALSXP), and both are fairly HL reproducible. well, in my experience such problems typically don't show exactly at the same place, and so the exact instance at which an unprotected object is GC'ed also changes, and hence I guesed it could be a REALSXP once and an INTSXP next time. HL As much as I can be sure, I guess. I overwrote HL src/library/Recommended and library/Matrix to be sure. ??? But the prerelease version of R-2.13.0 *contains* already Matrix_0.999375-49 the one you claim has the bug. [[and you still haven't told use the exact R version you were using]] I haven't been able to reproduce the problem (on 32-bit Linux, R-2.13.0 RC, after running your code snippet below for a couple of hours, I got the prompt back, and things continued working..) and Doug Bates has had a strong suspicion that the error message below looks like the problem that he fixed just between *-48 and *-49. pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') HL Loading required package: lattice Error in HL regexpr(package:, envName, fixed = TRUE) : unprotected HL object (0x2fe5818) encountered (was INTSXP) Error: HL package/namespace load failed for 'Matrix' Can you give more exact info about the platform ? HL fedora 14 x86_64. export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ./configure --enable-memory-profiling --enable-strict-barrier \ --enable-byte-compiled-packages --with-valgrind-instrumentation=2 aah, ok. Well of course that's not an R version I just have at my disposal ... ... but I'm building it now (for R-2.13.0 RC, i.e. the R-2-13-branch in R's subversion tree, revision 55350), and start your test - with R's own (=CRAN's) Matrix before finishing for today. HL Matrix-for-R-2.13@2659 was the one I was playing with. HL I am currently testing Matrix trunk@2666, since you HL seems to have put something relevant in r2661. That was really something else, also an important clean up step, but entirely unrelated to protection and garbage collection... and ... that version will definitely not make it into R-2.13.0 since it has too many changes compared with the one R-2.13.0 RC has been shipping with. Martin __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
--- On Thu, 7/4/11, Martin Maechler maech...@stat.math.ethz.ch wrote: ??? But the prerelease version of R-2.13.0 *contains* already Matrix_0.999375-49 the one you claim has the bug. [[and you still haven't told use the exact R version you were using]] Oh, I am tracking both R and Matrix via git-svn and retrieves all revisions to all branches daily (or at least, regularly). I.e. R svn head. 2.13.0 only forked off recently and most of the trunk-2.13.0rc differences are so far mostly documentation-related. I could switch to track R 2.13.x branch if you insist. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Hin-Tak Leung wrote: Martin Maechler wrote: Martin Maechler maech...@stat.math.ethz.ch on Thu, 7 Apr 2011 12:24:32 +0200 writes: HL == Hin-Tak Leung ht...@users.sourceforge.net on Thu, 7 Apr 2011 07:51:44 +0100 writes: HL Douglas Bates wrote: I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. HL Matrix-for-R-2.13@2659 (0.999375-49) has the problem at a different place: Are you sure that you really took 0.999375-49 and not its predecessor ??? The two errors look similiar but different (the previous one was about a REALSXP), and both are fairly reproducible. As much as I can be sure, I guess. I overwrote src/library/Recommended and library/Matrix to be sure. I haven't been able to reproduce the problem (on 32-bit Linux, R-2.13.0 RC, after running your code snippet below for a couple of hours, I got the prompt back, and things continued working..) and Doug Bates has had a strong suspicion that the error message below looks like the problem that he fixed just between *-48 and *-49. pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) options(error=dump.frames) library('Matrix') HL Loading required package: lattice HL Error in regexpr(package:, envName, fixed = TRUE) : HL unprotected object (0x2fe5818) encountered (was INTSXP) HL Error: package/namespace load failed for 'Matrix' Can you give more exact info about the platform ? fedora 14 x86_64. export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' ./configure --enable-memory-profiling --enable-strict-barrier \ --enable-byte-compiled-packages --with-valgrind-instrumentation=2 Matrix-for-R-2.13@2659 was the one I was playing with. I am currently testing Matrix trunk@2666, since you seems to have put something relevant in r2661. finished testing trunk r2666 (0.9996875-0). It takes about 3.5 hours (with gctorture(TRUE)) to get to: pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error in regexpr(package:, envName, fixed = TRUE) : unprotected object (0x3a37878) encountered (was INTSXP) Error: package/namespace load failed for 'Matrix' Execution halted __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
On Tue, Mar 29, 2011 at 2:17 PM, Douglas Bates ba...@stat.wisc.edu wrote: On Tue, Mar 29, 2011 at 5:34 AM, Hin-Tak Leung ht...@users.sourceforge.net wrote: Martin Maechler wrote: Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). interesting. In the mean time, I *did* run --- for several hours! --- your code example below, and it did *not* segfault for me (64-bit, Linux Fedora 13). Martin 64-bit fedora 14. For building R svn (and checking soon-to-be-released R packages, rather than daily R-related work), I also have these, and indeed have --enable-strict-barrier: export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' \ ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 sessionInfo() R version 2.14.0 Under development (unstable) (--) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_GB.UTF-8 LC_COLLATE=en_GB.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_GB.UTF-8 [7] LC_PAPER=en_GB.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base Thanks for the information. I can replicate the problem on a Red Hat EL 5 64-bit system and will start to debug now. I isolated the problem and tested then committed a fix. I am going to ask Martin to upload the new release as I have gotten out of sync with some of his recent changes and he will, I hope, reconcile the branches and trunk. If you need the fixed version immediately, say for testing, the changes are in the file Matrix/src/chm_common.c You can visit the SVN archive for the project, https://r-forge.r-project.org/scm/?group_id=61, click on the link to Browse Subversion Repository and go to pkg/Matrix/src/chm_common.c I made the mistake that I chastised others for making, performing an operation on the result of a call to install and another value that may have required allocation of memory. The first time install is called on a particular string it allocates and SEXPREC, which may cause a garbage collection. Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). On Sat, Mar 26, 2011 at 5:21 PM, Hin-Tak Leung hintak_le...@yahoo.co.uk wrote: Current core/Recommended Matrix package (0.999375-48) has been segfaulting against R 2.13-alpha/2.14-trunk for the last week or so (since R-2.13 was branched, when I started trying) when run with R CMD check --use-gct: -- pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error : .onLoad failed in loadNamespace() for 'Matrix', details: call: fun(...) error: unprotected object (0x2768b18) encountered (was REALSXP) Error: package/namespace load failed for 'Matrix' Execution halted --- I traced to this because R CMD check --use-gct snpStats (both 1.1.13 and 1.1.12) segfaults with the same message, and before that, the snpMatrix 1.15.8.4 which includes some of David's newly written ld() ( which depends on Matrix.) If the Matrix package segfaults, David's new ld() isn't useable. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Martin Maechler wrote: Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). interesting. In the mean time, I *did* run --- for several hours! --- your code example below, and it did *not* segfault for me (64-bit, Linux Fedora 13). Martin 64-bit fedora 14. For building R svn (and checking soon-to-be-released R packages, rather than daily R-related work), I also have these, and indeed have --enable-strict-barrier: export DEFS='-DUSE_TYPE_CHECKING_STRICT -DR_MEMORY_PROFILING' \ ./configure --enable-memory-profiling --enable-strict-barrier --enable-byte-compiled-packages --with-valgrind-instrumentation=2 sessionInfo() R version 2.14.0 Under development (unstable) (--) Platform: x86_64-unknown-linux-gnu (64-bit) locale: [1] LC_CTYPE=en_GB.UTF-8 LC_NUMERIC=C [3] LC_TIME=en_GB.UTF-8LC_COLLATE=en_GB.UTF-8 [5] LC_MONETARY=C LC_MESSAGES=en_GB.UTF-8 [7] LC_PAPER=en_GB.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C [11] LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). On Sat, Mar 26, 2011 at 5:21 PM, Hin-Tak Leung hintak_le...@yahoo.co.uk wrote: Current core/Recommended Matrix package (0.999375-48) has been segfaulting against R 2.13-alpha/2.14-trunk for the last week or so (since R-2.13 was branched, when I started trying) when run with R CMD check --use-gct: -- pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error : .onLoad failed in loadNamespace() for 'Matrix', details: call: fun(...) error: unprotected object (0x2768b18) encountered (was REALSXP) Error: package/namespace load failed for 'Matrix' Execution halted --- I traced to this because R CMD check --use-gct snpStats (both 1.1.13 and 1.1.12) segfaults with the same message, and before that, the snpMatrix 1.15.8.4 which includes some of David's newly written ld() ( which depends on Matrix.) If the Matrix package segfaults, David's new ld() isn't useable. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] core Matrix package segfaulted on R CMD check --use-gct
Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). interesting. In the mean time, I *did* run --- for several hours! --- your code example below, and it did *not* segfault for me (64-bit, Linux Fedora 13). Martin Douglas Bates ba...@stat.wisc.edu on Mon, 28 Mar 2011 09:24:39 -0500 writes: Can you provide the output from sessionInfo() so we can know the platform? Also, did you configure R with --enable-strict-barrier or set the C compilation flag -DTESTING_WRITE_BARRIER? I think that run-time error message can only be thrown under those circumstances (not that it isn't an error, it's just not checked for in other circumstances). On Sat, Mar 26, 2011 at 5:21 PM, Hin-Tak Leung hintak_le...@yahoo.co.uk wrote: Current core/Recommended Matrix package (0.999375-48) has been segfaulting against R 2.13-alpha/2.14-trunk for the last week or so (since R-2.13 was branched, when I started trying) when run with R CMD check --use-gct: -- pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error : .onLoad failed in loadNamespace() for 'Matrix', details: call: fun(...) error: unprotected object (0x2768b18) encountered (was REALSXP) Error: package/namespace load failed for 'Matrix' Execution halted --- I traced to this because R CMD check --use-gct snpStats (both 1.1.13 and 1.1.12) segfaults with the same message, and before that, the snpMatrix 1.15.8.4 which includes some of David's newly written ld() ( which depends on Matrix.) If the Matrix package segfaults, David's new ld() isn't useable. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel
[Rd] core Matrix package segfaulted on R CMD check --use-gct
Current core/Recommended Matrix package (0.999375-48) has been segfaulting against R 2.13-alpha/2.14-trunk for the last week or so (since R-2.13 was branched, when I started trying) when run with R CMD check --use-gct: -- pkgname - Matrix source(file.path(R.home(share), R, examples-header.R)) gctorture(TRUE) options(warn = 1) library('Matrix') Loading required package: lattice Error : .onLoad failed in loadNamespace() for 'Matrix', details: call: fun(...) error: unprotected object (0x2768b18) encountered (was REALSXP) Error: package/namespace load failed for 'Matrix' Execution halted --- I traced to this because R CMD check --use-gct snpStats (both 1.1.13 and 1.1.12) segfaults with the same message, and before that, the snpMatrix 1.15.8.4 which includes some of David's newly written ld() ( which depends on Matrix.) If the Matrix package segfaults, David's new ld() isn't useable. __ R-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-devel