Hi Nick, segmap memory consumption is included in the "Page cache" total of memstat. For UFS on Solaris/SPARC as of as of Solaris 10 3/05, segmap is no longer used for normal read and write operations. It is only used in a few oddball places relating to metadata, so you should be able to reduce it from the default 12% without adverse consequences.
If segmap is too small, and is in high demand from some consumer (the QFS filesystem as an example), then you will see increased %sys time and high crosscall counts as shown by the mpstat xcal column - in the millions of xcal/sec system wide. Thus you can monitor mpstat sys and xcal before and after reducing segmap_percent to verify there is little change. - Steve Nicolas Michael wrote: > Hi, > > I have some questions regarding fs cache (cachelist and segmap). > > Environment: - Solaris 10 U4 - SPARC, 64 GB memory - UFS, PxFS file systems > > > Our application is writing some logs to disk (4 GB / hour), flushing some > mmapped files from time to time (4 GB each 15 min), but is not doing much > disk I/O. > > Once our application is started and "warm", it doesn't allocate any further > memory. At this point, we have 3-4 GB of free memory (vmstat) and nothing > paged out to disk (swap -l). When I run long tests, I see free memory > decreasing down to 1 GB (lotsfree, 1/64th of total memory). At this point, > page scanner activity starts, pages are being paged out (and some being > paged in), and the amount of paged-out data (swap -l) increases to > something like 2 GB over time. (Interestingly, 1h after I stop the load, > free memory suddenly increases to 2.5 GB.) > > Since those memory requests are not coming from our application, I assume > that those 5 GB (3 GB less free memory plus 2 GB paged-out data) are used > for the file system cache. I always thought the fs cache would never grow > any more once memory gets short, so it should never cause paging activity > (since the cache list is part of the free list). Reading Solaris Internals, > I just learned that there's not only a cache list, but also a segmap cache. > As I understand this, the segmap cache may very well grow up to 12% of main > memory and may even cause application pages to be paged out, correct? So, > this might be what's happening here. Can I somehow monitor the segmap cache > (since it is kernel memory, is it reported as "Kernel" in ::memstat?)? > > My idea is now to set segmap_percent=1 to decrease the max size of the > segmap cache and this way avoid having pages paged out due to growing fs > cache. In a testrun with this configuration, my free memory doesn't fall > below 3.5 GB any more and nothing is being paged out -- saving me 4.5 GB of > memory! > > On solarisinternals.com I found the statement: --------------- The size of > the segmap can be increased to improve performance of file system intensive > workloads, using the /etc/system tunable segmap_percent, but this tunable > is only recognized on Solaris/SPARC systems. Also, such tuning is no longer > necessary for SPARC as of Solaris 10 3/05 (FCS), because the Kernel > Physical Mapping (KPM) feature greatly reduced the cost of mapping and > unmapping pages in segmap. Segmap is still used, and segmap_percent is > still recognized, but it makes little difference in performance. > --------------- > > Since we don't do much disk I/O, I would assume that we don't gain much > from the segmap cache anyway, so I would like to configure it to 1%. File > system pages will still be cached in the cache list as long as memory is > available, right? With the advantage, that the cache list is basically > "free" memory and would never cause other pages to be paged out. > > I'm not sure, but as I understand it the segmap cache is still used during > read and write operations, right? So, every time we write a file, we always > write into the segmap cache. If this cache is small (let's say: 1% = 640 > MB), we might be slowed done when writing more than 640 MB all at once. > However, if we would only write 64 MB every minute, pages from the segmap > cache would migrate to the cache list and make room for more pages in the > segmap cache, so next time we write 64 MB, would there again be enough > space in the segmap cache for the write operation? > > Also, just to be sure: memory mapped files are never read or written > through the segmap cache, so shrinking that cache has no effect on memory > mapped files, right? > > Are there any other side effects when decreasing the segmap cache to 1%? Or > is this the recommended way to decrease unnecessary memory usage for file > system caching? > > Thanks much for your help, Nick. > > > This message posted from opensolaris.org > _______________________________________________ perf-discuss mailing list > perf-discuss@opensolaris.org _______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
