Re: [zfs-discuss] zfs_arc_max values
On Wed, May 16, 2012 at 3:35 PM, Paynter, Richard
richard.payn...@infocrossing.com wrote:
Does anyone know what the minimum value for zfs_arc_max should be set to?
Does it depend on the amount of memory on the system, and – if so – is there
a formula, or percentage, to use to determine what the minimum value is?
I would assume that it should not be set any lower than `kstat
zfs::arcstats:c_min`, which I assume you can set with zfs_arc_min in
/etc/system, but that does not really answer your question. Under
Solaris 10 I have set it as low as 1 GB with no (apparent) ill effects
except the desired reduction in the ARC and performance associated
with such. This was NOT for production, but purely testing.
Why are you trying to tune the ARC as _low_ as possible? In my
experience the ARC gives up memory readily for other uses. The only
place I _had_ to tune the ARC in production was a couple systems
running an app that checks for free memory _before_ trying to allocate
it. If the ARC has all but 1 GB in use, the app (which is looking for
about 2.5 GB) does not see enough free memory and tosses an out of
memory error without even trying to actually get the memory it needs.
I have the ARC max tuned to 4 GB on these boxes with 16 GB RAM.
Instead of my ARC using 10-12 GB it is stuck with only 4 GB, but the
application does not quite with an error. Yes, I have flagged this
behavior for the vendor, they don't seem to care.
--
{1-2-3-4-5-6-7-}
Paul Kraus
- Senior Systems Architect, Garnet River ( http://www.garnetriver.com/ )
- Assistant Technical Director, LoneStarCon 3 (http://lonestarcon3.org/)
- Sound Coordinator, Schenectady Light Opera Company (
http://www.sloctheater.org/ )
- Technical Advisor, Troy Civic Theatre Company
- Technical Advisor, RPI Players
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
On Thu, 17 May 2012, Paul Kraus wrote: Why are you trying to tune the ARC as _low_ as possible? In my experience the ARC gives up memory readily for other uses. The only place I _had_ to tune the ARC in production was a couple systems running an app that checks for free memory _before_ trying to allocate it. If the ARC has all but 1 GB in use, the app (which is looking for On my system I adjusted the ARC down due to running user-space applications with very bursty short-term large memory usage. Reducing the ARC assured that there would be no contention between zfs ARC and the applications. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
I'll agree with Bob on this. A specific use case is a VirtualBox server hosting lots of guests. I even made a point of mentioning this tunable in the Solaris 10 Virtualization Essentials section on vbox :) There are several other use cases as well. Bob Bob Sent from my iPad On May 17, 2012, at 9:03 AM, Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Thu, 17 May 2012, Paul Kraus wrote: Why are you trying to tune the ARC as _low_ as possible? In my experience the ARC gives up memory readily for other uses. The only place I _had_ to tune the ARC in production was a couple systems running an app that checks for free memory _before_ trying to allocate it. If the ARC has all but 1 GB in use, the app (which is looking for On my system I adjusted the ARC down due to running user-space applications with very bursty short-term large memory usage. Reducing the ARC assured that there would be no contention between zfs ARC and the applications. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
On 05/17/12 15:03, Bob Friesenhahn wrote: On Thu, 17 May 2012, Paul Kraus wrote: Why are you trying to tune the ARC as _low_ as possible? In my experience the ARC gives up memory readily for other uses. The only place I _had_ to tune the ARC in production was a couple systems running an app that checks for free memory _before_ trying to allocate it. If the ARC has all but 1 GB in use, the app (which is looking for On my system I adjusted the ARC down due to running user-space applications with very bursty short-term large memory usage. Reducing the ARC assured that there would be no contention between zfs ARC and the applications. If the system is running one app which expects to do lots of application level caching (and in theory, the app should be able to work out what's worth caching and what isn't better than any filesystem underneath it can guess), then you should be planning your memory usage accordingly. For example, a database server, you probably want to allocate much of the system's memory to the database cache (in the case of Oracle, the SGA), leaving enough for a smaller ZFS arc and the memory required by the OS and app. Depends on the system and database size, but something like 50% SGA, 25% ZFS ARC, 25% for everything else might be an example, with the SGA disproportionally bigger on larger systems with larger databases. On my desktop system (supposed to be 8GB RAM, but currently 6GB due to a dead DIMM), I have knocked the ARC down to 1GB. I used to find the ARC wouldn't shrink in size until system had got to the point of crawling along showing anon page-ins, and some app (usually firefox or thunderbird) had already become too difficult to use. I must admit I did this a long time ago, and ZFS's shrinking of the ARC may be more proactive now than it was back then, but I don't notice any ZFS performance issues with the ARC restricted to 1GB on a desktop system. It may have increase scrub times, but that happens when I'm in bed, so I don't care. -- Andrew ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
On May 17, 2012, at 5:28 AM, Paul Kraus wrote: On Wed, May 16, 2012 at 3:35 PM, Paynter, Richard richard.payn...@infocrossing.com wrote: Does anyone know what the minimum value for zfs_arc_max should be set to? Does it depend on the amount of memory on the system, and – if so – is there a formula, or percentage, to use to determine what the minimum value is? I would assume that it should not be set any lower than `kstat zfs::arcstats:c_min`, which I assume you can set with zfs_arc_min in /etc/system, but that does not really answer your question. arc_c_max and arc_c_min are used for different things. If you make significant changes there, then it is a good idea to set both zfs_arc_min, zfs_arc_max, and zfs_arc_meta_limit as a set. NB, the minimum size of these is 64MB. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
I have a customer who wants to set zfs_arc_max to 1G for a 16G system, and 2G for a 32G system. Both of these are SAP and/or Oracle db servers. They apparently want to maximize the amount of memory available for the applications. My question, now, is whether it would be advisable to do this, and - if not - what the impact might be. The 32G system is an M3000; the 16G system is a V490. Both are running Solaris 10 Update 8, patched to 147440-09. I have a feeling that setting zfs_arc_max this low is not a good idea, but would like to get some corroborating information/references, if possible. Thanks Rick From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] On Behalf Of Richard Elling Sent: Thursday, May 17, 2012 11:24 AM To: Paul Kraus Cc: ZFS Discussions Subject: Re: [zfs-discuss] zfs_arc_max values On May 17, 2012, at 5:28 AM, Paul Kraus wrote: On Wed, May 16, 2012 at 3:35 PM, Paynter, Richard richard.payn...@infocrossing.com wrote: Does anyone know what the minimum value for zfs_arc_max should be set to? Does it depend on the amount of memory on the system, and - if so - is there a formula, or percentage, to use to determine what the minimum value is? I would assume that it should not be set any lower than `kstat zfs::arcstats:c_min`, which I assume you can set with zfs_arc_min in /etc/system, but that does not really answer your question. arc_c_max and arc_c_min are used for different things. If you make significant changes there, then it is a good idea to set both zfs_arc_min, zfs_arc_max, and zfs_arc_meta_limit as a set. NB, the minimum size of these is 64MB. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 Confidentiality Note: This e-mail, including any attachment to it, may contain material that is confidential, proprietary, privileged and/or Protected Health Information, within the meaning of the regulations under the Health Insurance Portability Accountability Act as amended. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error, and any review, dissemination, distribution or copying of this e-mail, including any attachment to it, is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system. Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
hi it will depend on how zfs is used in the system 1)root zpool only? 2)Db files also? 3)Is this RAC configuration? regards On 5/17/2012 3:19 PM, Paynter, Richard wrote: I have a customer who wants to set zfs_arc_max to 1G for a 16G system, and 2G for a 32G system. Both of these are SAP and/or Oracle db servers. They apparently want to maximize the amount of memory available for the applications. My question, now, is whether it would be advisable to do this, and -- if not -- what the impact might be. The 32G system is an M3000; the 16G system is a V490. Both are running Solaris 10 Update 8, patched to 147440-09. I have a feeling that setting zfs_arc_max this low is not a good idea, but would like to get some corroborating information/references, if possible. Thanks Rick *From:*zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Richard Elling *Sent:* Thursday, May 17, 2012 11:24 AM *To:* Paul Kraus *Cc:* ZFS Discussions *Subject:* Re: [zfs-discuss] zfs_arc_max values On May 17, 2012, at 5:28 AM, Paul Kraus wrote: On Wed, May 16, 2012 at 3:35 PM, Paynter, Richard richard.payn...@infocrossing.com mailto:richard.payn...@infocrossing.com wrote: Does anyone know what the minimum value for zfs_arc_max should be set to? Does it depend on the amount of memory on the system, and -- if so -- is there a formula, or percentage, to use to determine what the minimum value is? I would assume that it should not be set any lower than `kstat zfs::arcstats:c_min`, which I assume you can set with zfs_arc_min in /etc/system, but that does not really answer your question. arc_c_max and arc_c_min are used for different things. If you make significant changes there, then it is a good idea to set both zfs_arc_min, zfs_arc_max, and zfs_arc_meta_limit as a set. NB, the minimum size of these is 64MB. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com mailto:richard.ell...@richardelling.com +1-760-896-4422 Confidentiality Note: This e-mail, including any attachment to it, may contain material that is confidential, proprietary, privileged and/or Protected Health Information, within the meaning of the regulations under the Health Insurance Portability Accountability Act as amended. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error, and any review, dissemination, distribution or copying of this e-mail, including any attachment to it, is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system. Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- attachment: laotsao.vcf___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
sorry what is data storage? ASS U ME the root is local hdd regards On 5/17/2012 3:31 PM, Hung-Sheng Tsao Ph.D. wrote: hi it will depend on how zfs is used in the system 1)root zpool only? 2)Db files also? 3)Is this RAC configuration? regards On 5/17/2012 3:19 PM, Paynter, Richard wrote: I have a customer who wants to set zfs_arc_max to 1G for a 16G system, and 2G for a 32G system. Both of these are SAP and/or Oracle db servers. They apparently want to maximize the amount of memory available for the applications. My question, now, is whether it would be advisable to do this, and – if not – what the impact might be. The 32G system is an M3000; the 16G system is a V490. Both are running Solaris 10 Update 8, patched to 147440-09. I have a feeling that setting zfs_arc_max this low is not a good idea, but would like to get some corroborating information/references, if possible. Thanks Rick *From:*zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-boun...@opensolaris.org] *On Behalf Of *Richard Elling *Sent:* Thursday, May 17, 2012 11:24 AM *To:* Paul Kraus *Cc:* ZFS Discussions *Subject:* Re: [zfs-discuss] zfs_arc_max values On May 17, 2012, at 5:28 AM, Paul Kraus wrote: On Wed, May 16, 2012 at 3:35 PM, Paynter, Richard richard.payn...@infocrossing.com mailto:richard.payn...@infocrossing.com wrote: Does anyone know what the minimum value for zfs_arc_max should be set to? Does it depend on the amount of memory on the system, and – if so – is there a formula, or percentage, to use to determine what the minimum value is? I would assume that it should not be set any lower than `kstat zfs::arcstats:c_min`, which I assume you can set with zfs_arc_min in /etc/system, but that does not really answer your question. arc_c_max and arc_c_min are used for different things. If you make significant changes there, then it is a good idea to set both zfs_arc_min, zfs_arc_max, and zfs_arc_meta_limit as a set. NB, the minimum size of these is 64MB. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com mailto:richard.ell...@richardelling.com +1-760-896-4422 Confidentiality Note: This e-mail, including any attachment to it, may contain material that is confidential, proprietary, privileged and/or Protected Health Information, within the meaning of the regulations under the Health Insurance Portability Accountability Act as amended. If it is not clear that you are the intended recipient, you are hereby notified that you have received this transmittal in error, and any review, dissemination, distribution or copying of this e-mail, including any attachment to it, is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system. Thank you. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss -- -- attachment: laotsao.vcf___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
On May 17, 2012, at 12:19 PM, Paynter, Richard wrote: I have a customer who wants to set zfs_arc_max to 1G for a 16G system, and 2G for a 32G system. Both of these are SAP and/or Oracle db servers. They apparently want to maximize the amount of memory available for the applications. This is the best use case of the zfs_arc_max tunable. My question, now, is whether it would be advisable to do this, and – if not – what the impact might be. For this use case, it is a good idea. The impact of not setting it occurs when you later want to start a memory-hogging program, the request for arc to shrink can take a little while to satisfy, and the memory-hog might not be designed to cope with retrying large memory allocation requests. The 32G system is an M3000; the 16G system is a V490. Both are running Solaris 10 Update 8, patched to 147440-09. I have a feeling that setting zfs_arc_max this low is not a good idea, but would like to get some corroborating information/references, if possible. This use case is why the zfs_arc_max tunable exists. -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] zfs_arc_max values
On May 16, 2012, at 12:35 PM, Paynter, Richard richard.payn...@infocrossing.com wrote: Does anyone know what the minimum value for zfs_arc_max should be set to? Does it depend on the amount of memory on the system, and – if so – is there a formula, or percentage, to use to determine what the minimum value is? It depends on the OS. In general, it is set to size of RAM - 1GB. For many Solaris releases, there is a bug that limits the effective use to 7/8 of RAM. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
