> > And bufs don't need it either. Have you actually cranked your buffer > > cache that high? I have test this, on sparc64 which has unlimited DMA > > reach due to the iommu. The system comes to a crawl when there are > > too many mbufs or bufs, probably due to management structures unable > > to handle the pressure. > > No, I didn't know that. I assumed that having a few more GBs of bufcache > would help the performance. Until that is the case, 64bit dma does not > make much sense.
BTW, my tests were on a 128GB sun4v machine. Sun T5140. They are actually fairly cheap used these days. A maximum sized buffer cache should be fast. However there is no need for it to be dma-reachable. Bob's buffer cache flipper can bounce it to high memory easily after it is read the first time, and preserve it in otherwise unused memory. A buffer cache object of that sort is never written back to the io path. Also, it can be discarded in any memory shortage condition without cost.
