In this email to the R-help list:
https://stat.ethz.ch/pipermail/r-help/2017-June/447474.html
and in this question on Stackoverflow:
https://stackoverflow.com/questions/44486643/nleqslv-memory-use-in-r
Andrew Leach has raised a question about the memory usage of my package nleqslv.
In a model with a loop within a function he has experienced continuously
increasing memory usage
by package nleqslv leading to an out of memory condition.
I have been able to reproduce the continuously increasing memory usage with the
following small example.
<code>
library(nleqslv,lib.loc="../nleqslv.Rcheck")
library(pryr)
dslnex <- function(x) {
y <- numeric(2)
y[1] <- x[1]^2 + x[2]^2 - 2
y[2] <- exp(x[1]-1) + x[2]^3 - 2
y
}
xstart <- c(x1=1.5,x2=2)
nsims <- 10
for(test_iter in seq_len(nsims)){
z <- nleqslv(xstart,dslnex,jacobian=NULL)
print(paste("nleqslv iteration",test_iter,"and memory used is",mem_used()))
}
memory.profile()
gc()
print(paste("At end memory used is", mem_used()))
</code>
The final output is
<output>
[1] "nleqslv iteration 1 and memory used is 28921288"
[1] "nleqslv iteration 2 and memory used is 29133256"
[1] "nleqslv iteration 3 and memory used is 29132992"
[1] "nleqslv iteration 4 and memory used is 29134712"
[1] "nleqslv iteration 5 and memory used is 29136432"
[1] "nleqslv iteration 6 and memory used is 29138152"
[1] "nleqslv iteration 7 and memory used is 29139872"
[1] "nleqslv iteration 8 and memory used is 29141592"
[1] "nleqslv iteration 9 and memory used is 29143312"
[1] "nleqslv iteration 10 and memory used is 29145032"
>
> memory.profile()
NULL symbol pairlist closure environment promise
1 8554 194726 4230 1065 6534
language special builtin char logical integer
51374 45 671 9786 8030 37258
double complex character ... any list
2645 1 53795 0 0 18487
expression bytecode externalptr weakref raw S4
1 14662 2233 592 593 1049
> gc()
used (Mb) gc trigger (Mb) max used (Mb)
Ncells 416340 22.3 750400 40.1 581181 31.1
Vcells 726127 5.6 1308461 10.0 1191521 9.1
> print(paste("At end memory used is", mem_used()))
[1] "At end memory used is 29126472"
</output>
Indeed memory used increases in each pass of the for loop.
I have added these two lines at the top of the code after the library(pryr)
invocation.
library(compiler)
oldJIT <- enableJIT(0)
This resolves the issue in the sense that memory used remains constant after
the first iteration.
Output in the for loop is now:
<output>
[1] "nleqslv iteration 1 and memory used is 24487784"
[1] "nleqslv iteration 2 and memory used is 24495816"
[1] "nleqslv iteration 3 and memory used is 24495816"
[1] "nleqslv iteration 4 and memory used is 24495816"
[1] "nleqslv iteration 5 and memory used is 24495816"
[1] "nleqslv iteration 6 and memory used is 24495816"
[1] "nleqslv iteration 7 and memory used is 24495816"
[1] "nleqslv iteration 8 and memory used is 24495816"
[1] "nleqslv iteration 9 and memory used is 24495816"
[1] "nleqslv iteration 10 and memory used is 24495816"
</output>
My questions are
- is this a bug(let) in the JIT compiler?
- if it isn't what would need to be changed in nleqslv.R in the package source?
(I haven't a clue)
regards
Berend Hasselman
My sessionInfo:
R version 3.4.0 (2017-04-21)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.5
Matrix products: default
BLAS:
/Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRblas.0.dylib
LAPACK:
/Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_IE.UTF-8/en_IE.UTF-8/en_IE.UTF-8/C/en_IE.UTF-8/en_IE.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.0
______________________________________________
R-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
