On Aug 12, 2009 07:13 -0400, Mag Gam wrote: > Since we have been using lustre for a while now we would like to port > some of them to use I/O features of lustre. Currently, all of the > applications are very memory intensive, meaning they take in the > entire dataset at 1 time load it into memory (64GB) and process. I was > wondering if there are some techniques people are using to > enhance,optimize, and use less memory their applications by using > Lustre. Are there any white papers, algorithms or tricks I can use to > reduce memory footprint (with a slight loss of performance)?
I would steer you away from using memory-mapped IO. This is functional with Lustre, but is not at all the fastest way to do IO with Lustre. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
