Edward Ned Harvey wrote: >> since swap is _extrememly_ expensive to use, you don't actually want to >> use much, if any in a HPC cluster. >> > > > I know this seems counterintuitive - but - I have experience to the > contrary. In traditional thinking, of course, swap is slower so you don't > want to use it, but in modern thinking, having swap available boosts your > system performance because the system can trade swap for cache. > > Here's the reasoning: > > At all times, the kernel will grow to the maximum available ram > (buffering/caching disk reads/writes). So obviously the more memory > available, the better, and the less required in user space the better... > but ... This means at all times the kernel is choosing which disk blocks to > keep in cache, as user processes grow, whatever is deemed to be the least > valuable cached disk block is dropped out of ram. > > If you have plenty of swap available, it gives the kernel another degree of > freedom to work with. The kernel now has the option available to page out > some idle process that it deems to be less valuable than the cached disk > blocks. > > If you run "free" or "top" on your system (assuming linux)... Soon after > booting, you'll see lots of free memory. But if your system has been up for > a week, you'll see zero free memory, and all the rest is consumed by > buffers. > > During the time when there is still "free" memory available, you will get no > performance boost by having swap available (and obviously there would be no > reason to consume any swap). But after the system is up for a long time, > and the kernel has filled all the ram with buffers... Then you get a > performance boost by using swap. >
This is very tricky and presumes that your local disk is faster than your back end storage, which is not necessarily the case. A local disk cache can be your friend or your enemy depending on your job load and your architecture. If you have a big honking storage farm to serve your HPC cluster with lots of memory, you can serve things at nearly wire speed. Again, this depends upon many factors. In our HPC workload, swap is never used except for a very rare series of jobs computing force fields between molecules, and it's extremely painful there, so they tune their workload very carefully. _______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
