I think I've found a solution for this. I've submitted a pull request here https://github.com/RcppCore/RcppArmadillo/pull/88. It basically now allows the users to pass -DARMA_64BIT_WORD and compile with it (so ARMA_32BIT_WORD is not defined but it is still the default behavior of RcppArmadillo). Will work on tests on this repo https://github.com/gvegayon/arma64bit.
Best, George G. Vega Yon +1 (626) 381 8171 http://www.its.caltech.edu/~gvegayon/ On Thu, Apr 28, 2016 at 6:07 PM, George Vega Yon <g.vega...@gmail.com> wrote: > Hey! > > Since it seems to be the case that we don't know much about what happens, > I'm making an experiment which consists on an R package to stress > RcppArmadillo here: https://github.com/gvegayon/arma64bit. Following > Dirk's advice, I've included the #define ARMA_64BIT_WORD before #include > <RcppArmadillo.h> (as seen here > https://github.com/gvegayon/arma64bit/blob/c10b5d4a80b93088da730e45ee8c583feeff2ab6/src/misc.cpp#L2-L3) > and included CXX_STD = CXX11 in Makevars. After compiling the package and > asking for a matrix of size 1e5 x 1e5 I get an error, here (so it doesn't > seem to be working): > > > x <- sp_runif(1e5,1e5,.01) > > error: SpMat::init(): requested size is too large > Error: SpMat::init(): requested size is too large > > My guess is that #define ARMA_32BIT_WORD is overriding #define > ARMA_64BIT_WORD. I'm an R user that jumped to Rcpp a while ago so I still > don't fully understand how things work under the hood with R + Rcpp. Any > clues? > > Best, > > sessionInfo() > > R version 3.2.5 (2016-04-14) > Platform: x86_64-pc-linux-gnu (64-bit) > Running under: Ubuntu 14.04.4 LTS > > locale: > [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C > LC_TIME=en_US.UTF-8 > [4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_US.UTF-8 > LC_MESSAGES=en_US.UTF-8 > [7] LC_PAPER=en_US.UTF-8 LC_NAME=C LC_ADDRESS=C > > [10] LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 > LC_IDENTIFICATION=C > > attached base packages: > [1] stats graphics grDevices utils datasets methods base > > other attached packages: > [1] arma64bit_0.1.0.9000 Matrix_1.2-5 > > loaded via a namespace (and not attached): > [1] Rcpp_0.12.4 lattice_0.20-33 digest_0.6.9 withr_1.0.1 > grid_3.2.5 > [6] covr_2.0.1 git2r_0.14.0 magrittr_1.5 lazyeval_0.1.10 > rstudioapi_0.5 > [11] whisker_0.3-2 rex_1.1.1 rmarkdown_0.9.5 devtools_1.11.1 > tools_3.2.5 > [16] yaml_2.1.13 htmltools_0.3.5 memoise_1.0.0 > > George G. Vega Yon > +1 (626) 381 8171 > http://www.its.caltech.edu/~gvegayon/ > > On Thu, Mar 31, 2016 at 12:37 PM, Dirk Eddelbuettel <e...@debian.org> > wrote: > >> >> On 29 March 2016 at 15:55, George Vega Yon wrote: >> | Hi there, >> | >> | A couple of days ago I included this comment on a Stackoverflow >> question about >> | "too large" matrices in RcppArmadillo >> | I wonder if we can use this in a package. After trying to compile the >> package >> | with and without PKG_CXXFLAGS=-DARMA_64BIT_WORD I'm still not able to >> create >> | sparse matrices bigger than 65,535 x 65,535. I've realized that the >> | RcppArmadilloConfig.h (version 0.6.600.4.0) file has ARMA_32BIT_WORD 1 >> so it >> | actually prevents using ARMA_64BIT_WORD (I think?). Any way I can solve >> this, >> | or I just have to deal with R (3.2.4 Revised (2016-03-16 r70336)) not >> been able >> | to handle 64INT? >> | >> | http://stackoverflow.com/questions/16159174/ >> | large-spmat-object-with-rcpparmadillo >> | >> | For which Dirk answered: >> | >> | Try locally setting ARMA_64BIT_WORD (check how) -- I can't at the >> package level >> | as it breaks existing code. It will likely help you. If you set it >> before the # >> | include <RcppArmadillo.h> things may just work. That at least is the >> pln. (And >> | you should have asked on rcpp-devel to get more than two eyeballs on >> this...) >> | And yes the other answer basically says the same thing. – Dirk >> Eddelbuettel 2 >> | days ago >> | >> | Now I'm asking again but on rcpp-devel =). The issue is that I'm >> concerned on >> | the part that Dirk says that "things may just work". I was reading >> about R not >> | been able to handle int64 and its implications for big matrices, and so >> I'm a >> | little concerned that if passing ARMA_64BIT_WORD works I may loose some >> | information on the way. For example, if I allow a users creating and >> | manipulating 1e6 x 1e6 sparse matrices, is there any chance for them to >> loose >> | information? >> >> It is entirely possibly that nobody stresses RcppArmadillo this way and >> that >> setting ARMA_64BIT_WORD gets unset. I only know that I cannot set it >> globally now as some packages (still) depend on ARMA_32BIT_WORD. >> >> So I encourage you to experiment and to try locally. Maybe something can >> (and even should) be improved. >> >> Dirk >> >> >> | >> | Thanks, >> | >> | George G. Vega Yon >> | +1 (626) 381 8171 >> | http://www.its.caltech.edu/~gvegayon/ >> | _______________________________________________ >> | Rcpp-devel mailing list >> | Rcpp-devel@lists.r-forge.r-project.org >> | https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel >> >> -- >> http://dirk.eddelbuettel.com | @eddelbuettel | e...@debian.org >> > >
_______________________________________________ Rcpp-devel mailing list Rcpp-devel@lists.r-forge.r-project.org https://lists.r-forge.r-project.org/cgi-bin/mailman/listinfo/rcpp-devel
