A Sunday 07 March 2010 20:03:21 Gael Varoquaux escrigué: > On Sun, Mar 07, 2010 at 07:00:03PM +0000, René Dudfield wrote: > > 1. Mmap'd files are useful since you can reuse disk cache as program > > memory. So large files don't waste ram on the disk cache. > > I second that. mmaping has worked very well for me for large datasets, > especialy in the context of reducing memory pressure.
As far as I know, memmap files (or better, the underlying OS) *use* all available RAM for loading data until RAM is exhausted and then start to use SWAP, so the "memory pressure" is still there. But I may be wrong... -- Francesc Alted _______________________________________________ NumPy-Discussion mailing list [email protected] http://mail.scipy.org/mailman/listinfo/numpy-discussion
