Easy way to reduce the client memory used by "Lustre" is to have an Epilogue script run by SGE (or whatever scheduler/resource manager) that does something like this on every node: # sync ; sleep 1 ; sync # echo 3 > /proc/sys/vm/drop_caches
Kevin Nathan Rutman wrote: > Jim, I'm forwarding this to lustre-discuss to get a broader community > input. I'm sure somebody has some experience with this. > > Begin forwarded message: >> >> I am looking for information on how Lustre assigns and holds pages on >> client nodes across jobs. The motivation is that we want to make >> "huge" pages available to users. We have found that it is almost >> impossible to allocate very many "huge" pages since Lustre holds >> scattered small pages across jobs. In fact, typically about 1/3 of >> compute node memory can be allocated as huge pages. >> >> We have done quite a lot of performance studies which show that a >> substantial percentage of jobs on Ranger have TLB misses as a major >> performance bottleneck. We estimate we might recover as much as an >> additional 5%-10% throughput if users could use huge pages. >> >> Therefore we would like to find a way to minimize the client memory >> which Lustre holds across jobs. >> >> Have you had anyone else mention this situation to you? >> >> Regards, >> >> Jim Browne >> >> > > ------------------------------------------------------------------------ > > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss > _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
