I would probably tune lotsfree down as well. At 72G of ram currently it's probably reserving around 1.1GB of ram.
http://docs.sun.com/app/docs/doc/819-2724/6n50b07bk?a=view Ethan -----Original Message----- From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Tomas Ögren Sent: Friday, February 26, 2010 6:45 AM To: Ronny Egner Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] ZFS Storage system with 72 GB memoryconstantly has 11 GB free memory On 26 February, 2010 - Ronny Egner sent me these 0,6K bytes: > Dear All, > > our storage system running opensolaris b133 + ZFS has a lot of memory for > caching. 72 GB total. While testing we observed free memory never falls below > 11 GB. > > Even if we create a ram disk free memory drops below 11 GB but will be 11 GB > shortly after (i assume ARC cache is shrunken in this context). > > As far as i know ZFS is designed to use all memory except 1 GB for caching.... http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#arc_init http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/common/fs/zfs/arc.c#arc_reclaim_needed So you have a max limit which it won't try to go past, but also a "keep this much free for the rest of the system". Both are a bit too protective for a pure ZFS/NFS server in my opinion (but can be tuned). You can check most variables with f.ex: echo freemem/D | mdb -k On one server here, I have in /etc/system: * http://www.solarisinternals.com/wiki/index.php/ZFS_Evil_Tuning_Guide#Limiting_the_ARC_Cache * about 7.8*1024*1024*1024, must be < physmem*pagesize (2062222*4096=8446861312 right now) set zfs:zfs_arc_max = 8350000000 set zfs:zfs_arc_meta_limit = 7000000000 * some tuning set ncsize = 500000 set nfs:nrnode = 50000 And I've done runtime modifications to swapfs_minfree to force usage of another chunk of memory. /Tomas -- Tomas Ögren, st...@acc.umu.se, http://www.acc.umu.se/~stric/ |- Student at Computing Science, University of Umeå `- Sysadmin at {cs,acc}.umu.se _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
