On Mon, Aug 20, 2018 at 10:52:15AM +0200, Anisa Llaveshi wrote: > Greetings, > > I have recently started using mlpack for a C++ application and I came > across a problem that I haven't been able to solve. I am using Linear > Regression to learn the parameters of a linear model. My training data is a > vector of 1-dimensional points. It consists of a vector of type double > (64-bit). I initialize the data points matrix from a std::vector structure > (where I have the data) using this constructor: arma::mat(std::vector) > Depending on the datasize of the dataset that I use to create the Linear > Regression model I get this error: > > > > error: arma::memory::acquire(): out of memory > > terminate called after throwing an instance of 'std::bad_alloc' > > what(): std::bad_alloc > > > > I am running the application on a machine which has 250GB of memory. When I > use 100k points (less then 1MB of data) I observe that ~28% of the memory > is being used to build the model. When I increase this number to 160k > points I observe that ~50% of the memory is being used and then the process > is killed. When I increase it a bit more the above error is immediately > thrown when trying to build the model. > I was wondering whether it is normal for the model to consume this much > memory for a small amount of data and if this is the case then what can one > do to use a larger dataset?
Hi Anisa, The memory used by a linear regression should be dxd, where d is the number of dimensions in your data. Remember that with mlpack, each observation (or point) in your data should correspond to one column---not one row---since Armadillo is column major. Here's some more information: https://www.mlpack.org/docs/mlpack-3.0.3/doxygen/matrices.html So, my guess is that your matrix currently has N rows and 1 column, causing mlpack to try and invert an NxN matrix (very large), but what you actually want is a matrix with 1 row and N columns. I hope this helps! Let me know if I can clarify anything. Thanks, Ryan -- Ryan Curtin | "Plot is a primitive vulgarity in literature." r...@ratml.org | - Balph Eubank _______________________________________________ mlpack mailing list mlpack@lists.mlpack.org http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack