On Thu, Dec 10, 2020 at 02:03:44PM +0100, Markus Kolb wrote: > Hello, > > do you know about the "high" memory usage during build of mlpack? > With version 3.3.2 a machine with guaranteed physical memory of 4.8G when > using 2 CPUs has been enough. > Now when compiling 3.4.2 a machine with 8G physical memory and using 2 CPUs > is going in OOM. > > ... > > What is being done here during build that needs so much memory? > In relation to the size of mlpack library this looks strange. > Is the memory usage ok or is there anything wrong in the build process?
Hi Markus, Unfortunately it seems that over the years, C++ compilers have become more and more hungry for memory. So that is part of the issue. I'm surprised at your report that 8GB memory can't build on 2 cores, but I can't think of anything that would be wrong there. You could try seeing if something else is using RAM too during the build? There are no significant code changes between 3.3.2 and 3.4.2 that should cause memory blowups; I wonder if the version of the compiler makes a difference here. That said, we are investigating ways to accelerate our builds and figure out how we can reduce the memory footprint. We've been pushing towards removing Boost as a dependency, and I think that this will be very helpful. Hope this helps! Thanks, Ryan -- Ryan Curtin | "What? Facts?" [email protected] | - Joe Cairo _______________________________________________ mlpack mailing list [email protected] http://knife.lugatgt.org/cgi-bin/mailman/listinfo/mlpack
