Hello R-devel,
Here's a simplification of the problem that currently plagues lme4:
x <- local({
# y[[1]] is large enough to have its own mmap() allocation
y <- list(rep(1, 1e7))
# y[[1]] is also large enough (>=64 elements) so structure() will
# create a wrapper instead of duplicating
structure(y[[1]], foo = 'bar')
# y[[1]] now has a reference count of >=2
})
# now x has the only reference to the original vector
writeLines('
#include <Rinternals.h>
static double run_arbitrary_R_code(SEXP x) {
const double * xd = REAL(x); // oops, forgot REAL_RO
// since the underlying vector is MAYBE_SHARED, the
// wrapper in x duplicated the underlying vector and
// returned a different data pointer
Rprintf("REAL(x) = %p\\n", xd);
double sum = 0;
for (R_xlen_t i = 0; i < XLENGTH(x); ++i) sum += xd[i];
R_gc();
// the original vector is now freed, memory unmapped
return sum;
}
SEXP foo(SEXP x) {
// cache a read-only pointer to the data of x
const double *xd = REAL_RO(x);
Rprintf("REAL_RO(x) = %p\\n", xd);
double ret1 = run_arbitrary_R_code(x);
// xd is now invalid
double ret2 = 0;
for (R_xlen_t i = 0; i < XLENGTH(x); ++i) ret2 += xd[i]; // <-- crash
return ScalarLogical(ret1 == ret2);
}
', 'foo.c')
stopifnot(tools::Rcmd('SHLIB foo.c') == 0)
dyn.load(paste0('foo', .Platform$dynlib.ext))
.Call('foo', x)
# REAL_RO(x) = 0x7f11527b4040
# REAL(x) = 0x7f114db68040
#
# *** caught segfault ***
# address 0x7f11527b4040, cause 'memory not mapped'
In the original lme4 code the data pointers are cached inside Eigen
objects courtesy of RcppEigen (shared with the R session, ensuring that
the R objects underlying the data pointers stay protected), and the role
of run_arbitrary_R_code() is played by a %*% operation (which uses
REAL() to read a vector).
Is there a nice way to solve this problem of a protected object's data
pointer suddenly becoming invalid? (Where is the best place to solve
it? lme4, Rcpp, R itself?) Would it be safe for lme4 to cache writable
data pointers (e.g. REAL() instead of DATAPTR_RO()), assuming that the
former will stay the same as long as the object exists? Or does it need
an extensive rewrite to always refresh cached data pointers?
--
Best regards,
Ivan
______________________________________________
[email protected] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
