[Rd] as.environment.list provides inconsistent results under torture
Hello,
Using R-devel (rev 53950), I get inconsistent results with
as.environment( VECSXP ) when gctorture is on.
Consider:
a - list( aa = rnorm, bb = runif )
gctorture(TRUE)
as.environment( a )
The last line sometimes produces the correct environment, but sometimes
I get errors. Here are the three situations:
# good
as.environment( a )
environment: 0x100b1c978
# not good
as.environment( a )
Erreur dans length(x) : 'x' est manquant
# not good either
as.environment( a )
Erreur dans list(NULL, list(aa = function (n, mean = 0, sd = 1) :
correspondance partielle de chaînes de caractères incorrecte
Is it because the call made by lang4 is not protected while evaluated in
this line :
case VECSXP: {
/* implement as.environment.list() {isObject(.) is false for a list} */
return(eval(lang4(install(list2env), arg,
/*envir = */R_NilValue, /* parent = */R_EmptyEnv),
rho));
}
(BTW, this was detected in a long Rcpp-devel thread. See
http://comments.gmane.org/gmane.comp.lang.r.rcpp/1336)
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] [Rcpp-devel] Loading a package using Rcpp Modules results in memory corruption
On Tue, Jan 11, 2011 at 2:41 PM, Romain Francois
rom...@r-enthusiasts.comwrote:
Le 11/01/11 19:57, Romain Francois a écrit :
Le 11/01/11 19:46, Douglas Bates a écrit :
On Tue, Jan 11, 2011 at 12:27 PM, Dominick
Samperidjsamp...@gmail.com wrote:
On Tue, Jan 11, 2011 at 1:20 PM, Romain
Francoisrom...@r-enthusiasts.com
wrote:
Le 11/01/11 19:12, Davor Cubranic a écrit :
I think an important point from Doug has been lost in the subsequent
20-odd messages of flamebombing:
I do not see this as compatible with the C++ Design principle we use
whereby
protection / unprotection occurs relative to the end of scope -- and
not
after every single assignment or allocation.
In other words, gctorture() may well be a fine test for the C API
and
its
PROTECT and UNPROTECT at every step but possibly not quite as
much for
Rcpp.
I don't think so. In the situation that Dominick is describing the C
API is being used (calls to Rf_install, Rf_lang1, Rf_eval, ...) and
you have to play by the rules of the C API. Essentially every time
that you call a function in the C API that can cause a memory
allocation you are open yourself to the possibility of having the
garbage collector run and getting unprotected SEXPs blown away. And,
naturally, Rf_eval can cause memory allocation.
I understood Dominick to be saying that in the code related to
Modules
and the evaluator and all that which we have been trying to debug
there are such cases of the use of the C API that are unsafe.
Can anyone comment whether this is correct?
Davor
Yep. Doug's analysis is right. Rcpp is implemented with the C R API,
and
apparently there were a few places where we were not careful enough.
Most
notably in calls to Rf_lcons and Rf_cons. This has been partially
dealt with
today.
Just for the record, Doug is summarizing my analysis, based on several
examples that I posted to this thread,
and that I am pleased to see have inspired some remedial action.
Sorry for not responding earlier. I'm in the middle of teaching a
short course.
Dominick is correct that it was his analysis that picked up the
failures to PROTECT/UNPROTECT in cases that were causing at least some
of the problems with loading Modules. Credit where credit is due.
As Romain has indicated, some of the problems have been fixed but
apparently problems still remain. Debugging memory protection
problems is often very difficult.
It is. Not sure what is my next step here. Remaining problems seem not
directly related to Rcpp, but rather associated with lazy loading. See:
This now by not be related to Rcpp. See the thread on R-devel:
http://comments.gmane.org/gmane.comp.lang.r.devel/26504
Please whoever is interested in fixing this, send your remarks and patches
to the R-devel mailing list.
This looks like the same problem that appeared in Rcpp, an unprotected SEXP
in
R/src/main/envir.c, in the function do_as_environment(), case VECSXP of
the
switch. Here is modified code that seems to fix the problem, at least under
Linux:
case VECSXP: {
Rprintf(VECSXP as.environment\n);
/* implement as.environment.list() {isObject(.) is false for a list} */
SEXP sp = PROTECT(lang4(install(list2env), arg,
/*envir = */R_NilValue, /* parent = */R_EmptyEnv));
SEXP res = eval(sp, rho);
UNPROTECT(1);
return res;
}
Dominick
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release
___
Rcpp-devel mailing list
rcpp-de...@lists.r-forge.r-project.org
https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
[[alternative HTML version deleted]]
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.environment.list provides inconsistent results under torture
Interesting, I'd argue that the bug is in eval() not protecting its arguments
since the usual convention is for functions to protect its arguments...
On Jan 11, 2011, at 2:33 PM, Romain Francois wrote:
Hello,
Using R-devel (rev 53950), I get inconsistent results with as.environment(
VECSXP ) when gctorture is on.
Consider:
a - list( aa = rnorm, bb = runif )
gctorture(TRUE)
as.environment( a )
The last line sometimes produces the correct environment, but sometimes I get
errors. Here are the three situations:
# good
as.environment( a )
environment: 0x100b1c978
# not good
as.environment( a )
Erreur dans length(x) : 'x' est manquant
# not good either
as.environment( a )
Erreur dans list(NULL, list(aa = function (n, mean = 0, sd = 1) :
correspondance partielle de chaînes de caractères incorrecte
Is it because the call made by lang4 is not protected while evaluated in this
line :
case VECSXP: {
/* implement as.environment.list() {isObject(.) is false for a list} */
return(eval(lang4(install(list2env), arg,
/*envir = */R_NilValue, /* parent = */R_EmptyEnv),
rho));
}
(BTW, this was detected in a long Rcpp-devel thread. See
http://comments.gmane.org/gmane.comp.lang.r.rcpp/1336)
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.environment.list provides inconsistent results under torture
No. Lots of internal functions expect their callers to protect their
arguments, for efficiency reasons. eval is called very often and
almost always with argument that are protected because they are in the
evaluation engine, so it would be wasteful and potentially very costly
if eval protected its arguments every time it is called. (I don't
tknow what the cost would be to do so in the current implementation
but it could be prohibitive if we moved to some different approaches,
so for now we whould continue to expect callers of eval to make sure
the argumetns are protected.)
Best,
luke
On Tue, 11 Jan 2011, Simon Urbanek wrote:
Interesting, I'd argue that the bug is in eval() not protecting its arguments
since the usual convention is for functions to protect its arguments...
On Jan 11, 2011, at 2:33 PM, Romain Francois wrote:
Hello,
Using R-devel (rev 53950), I get inconsistent results with as.environment(
VECSXP ) when gctorture is on.
Consider:
a - list( aa = rnorm, bb = runif )
gctorture(TRUE)
as.environment( a )
The last line sometimes produces the correct environment, but sometimes I get
errors. Here are the three situations:
# good
as.environment( a )
environment: 0x100b1c978
# not good
as.environment( a )
Erreur dans length(x) : 'x' est manquant
# not good either
as.environment( a )
Erreur dans list(NULL, list(aa = function (n, mean = 0, sd = 1) :
correspondance partielle de chaînes de caractères incorrecte
Is it because the call made by lang4 is not protected while evaluated in this
line :
case VECSXP: {
/* implement as.environment.list() {isObject(.) is false for a list} */
return(eval(lang4(install(list2env), arg,
/*envir = */R_NilValue, /* parent = */R_EmptyEnv),
rho));
}
(BTW, this was detected in a long Rcpp-devel thread. See
http://comments.gmane.org/gmane.comp.lang.r.rcpp/1336)
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: l...@stat.uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Re: [Rd] as.environment.list provides inconsistent results under torture
On Jan 11, 2011, at 6:55 PM, luke-tier...@uiowa.edu luke-tier...@uiowa.edu
wrote:
No. Lots of internal functions expect their callers to protect their
arguments, for efficiency reasons. eval is called very often and almost
always with argument that are protected because they are in the evaluation
engine, so it would be wasteful and potentially very costly if eval protected
its arguments every time it is called. (I don't tknow what the cost would be
to do so in the current implementation but it could be prohibitive if we
moved to some different approaches, so for now we whould continue to expect
callers of eval to make sure the argumetns are protected.)
Fair enough. It would be nice if this was explicitly documented since eval() is
part of the API and I see several packages on CRAN using eval(LCONS(..),..) and
eval(listX(...),...) - and I don't blame them (partly because one of them is
mine ;)). Unfortunately all the examples in R-ext use implicitly protected
arguments (as function arguments or parts of larger already protected
constructs) so it's not obvious from that, either.
Thanks,
Simon
On Tue, 11 Jan 2011, Simon Urbanek wrote:
Interesting, I'd argue that the bug is in eval() not protecting its
arguments since the usual convention is for functions to protect its
arguments...
On Jan 11, 2011, at 2:33 PM, Romain Francois wrote:
Hello,
Using R-devel (rev 53950), I get inconsistent results with as.environment(
VECSXP ) when gctorture is on.
Consider:
a - list( aa = rnorm, bb = runif )
gctorture(TRUE)
as.environment( a )
The last line sometimes produces the correct environment, but sometimes I
get errors. Here are the three situations:
# good
as.environment( a )
environment: 0x100b1c978
# not good
as.environment( a )
Erreur dans length(x) : 'x' est manquant
# not good either
as.environment( a )
Erreur dans list(NULL, list(aa = function (n, mean = 0, sd = 1) :
correspondance partielle de chaînes de caractères incorrecte
Is it because the call made by lang4 is not protected while evaluated in
this line :
case VECSXP: {
/* implement as.environment.list() {isObject(.) is false for a list} */
return(eval(lang4(install(list2env), arg,
/*envir = */R_NilValue, /* parent = */R_EmptyEnv),
rho));
}
(BTW, this was detected in a long Rcpp-devel thread. See
http://comments.gmane.org/gmane.comp.lang.r.rcpp/1336)
Romain
--
Romain Francois
Professional R Enthusiast
+33(0) 6 28 91 30 30
http://romainfrancois.blog.free.fr
|- http://bit.ly/fT2rZM : highlight 0.2-5
|- http://bit.ly/gpCSpH : Evolution of Rcpp code size
`- http://bit.ly/hovakS : RcppGSL initial release
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
--
Luke Tierney
Statistics and Actuarial Science
Ralph E. Wareham Professor of Mathematical Sciences
University of Iowa Phone: 319-335-3386
Department of Statistics andFax: 319-335-3017
Actuarial Science
241 Schaeffer Hall email: l...@stat.uiowa.edu
Iowa City, IA 52242 WWW: http://www.stat.uiowa.edu
__
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
