Here's a ZFS tuning guide if you have not seen.
https://wiki.freebsd.org/ZFSTuningGuide 

But only goes to v9.

 

Down the page they ref 2-5GB/TB for dedupe.  Free advice, worth every penny
paid!

 

https://www.freebsd.org/doc/en/books/faq/all-about-zfs.html

 

My NAS4Free server uses 90% of its 4GB RAM for a 3TB volume, configured with
1.75GB arc_max.

 

-----Original Message-----
From: List [mailto:list-boun...@lists.pfsense.org] On Behalf Of Paul Mather
Sent: Tuesday, March 6, 2018 12:09 PM
To: pfSense Support and Discussion Mailing List <list@lists.pfsense.org>
Subject: Re: [pfSense] ZFS on 2.4.2

 

On Mar 6, 2018, at 12:39 PM, Walter Parker < <mailto:walt...@gmail.com>
walt...@gmail.com> wrote:

 

> On Mon, Mar 5, 2018 at 6:38 PM, Curtis Maurand <
<mailto:cmaur...@xyonet.com> cmaur...@xyonet.com> wrote:

> 

>> ZFS is a memory hog.   you need 1 GB of RAM for each TB of disk.

> 

> 

> Curtis, can you provide some more details? I have been testing this 

> for the last couple of weeks and ZFS doesn't require 1G for each TB to 

> function (which is the standard meaning of need).

> From my direct testing and experience 1G per TB is a rule of thumb for 

> suggested memory sizing on general purpose servers. Do you have 

> specific information that violating this rule of thumb will cause
functional issues?

> 

> To be more blunt, was this a case of drive by nerd sniping or do you 

> know something that will cause my specific use case to fail at some 

> point in the future?

 

 

The "1G for each TB" sounds like the rule of thumb for when you plan to
enable deduplication on a dataset.  ZFS deduplication can be a disastrous
memory hog (or else completely ruin your performance if you don't have
sufficient ARC memory/resources), which is why many people do not enable it
unless they've made a serious conscious decision to do so.

 

I ran ZFS on a 1--2 GB RAM FreeBSD/i386 system for years and it was stable.
I have to tune KVM and restrict ARC RAM consumption, but once I did that I
had no problems.  It's my experience that ZFS is more stable and tested on
FreeBSD/amd64.

 

Cheers,

 

Paul.

 

 

> 

> 

> Walter

> 

> 

> 

>> On 3/1/2018 1:49 AM, Walter Parker wrote:

>> 

>>> Forgot to CC the list.

>>> 

>>> On Wed, Feb 28, 2018 at 10:13 PM, Walter Parker <
<mailto:walt...@gmail.com> walt...@gmail.com>

>>> wrote:

>>> 

>>> Thank you for the backup script.

>>>> 

>>>> By my calculations, 2G should be enough. If I limit the ARC cache 

>>>> to 1G, that leaves 1G for applications & kernel memory. As I'm not 

>>>> serving the 6TB drive up as a file server, but using it for one 

>>>> specific task (to receive the backups from one host) I figure that 

>>>> I don't need lots of memory. ZFS as a quick file server or busy 

>>>> server needs lots of memory to be quick.

>>>> I've seen testing showing ZFS doing fast file copies on as little 

>>>> as 768M total system after proper memory tuning.

>>>> 

>>>> I need ZFS because it is the only file system that can receive 

>>>> incremental ZFS snapshots and apply them. I have not setup the ZFS 

>>>> backup software yes, so I'm just using rsnapshot. First time it 

>>>> ran, it filled all 1G of the cache. I rebooted the firewall 

>>>> afterwards and now ZFS with 60-100M of usage (the amount of data 

>>>> that rsync updates on a daily basis is pretty small).

>>>> Right now, the data from the other server is ~8.8G, compressed to 

>>>> 1.7G with lz4.

>>>> 

>>>> When I get the full backup running, I will be ~1.5TB in size. ZFS 

>>>> snapshots should be pretty small and quick (as it can send just the 

>>>> data that was updated without having to walk the entire 

>>>> filesystem). An rsync backup would have to walk the whole system to
find all of the changes.

>>>> Most

>>>> of the data on the system doesn't change (as it is a media library).

>>>> 

>>>> I'll post back more results if people are interested, after I get 

>>>> the backup software working (I'm thinking about using ZapZend).

>>>> 

>>>> 

>>>> Walter

>>>> 

>>>> 

>>>> 

>>>> On Wed, Feb 28, 2018 at 8:54 PM, ED Fochler 

>>>> < <mailto:soek...@liquidbinary.com> soek...@liquidbinary.com>

>>>> wrote:

>>>> 

>>>> I feel like I'm late in responding to this, but I have to say that 

>>>> 2GB of

>>>>> RAM doesn't seem like nearly enough for a 6TB zfs volume.  ZFS is 

>>>>> great in a lot of ways, but is a RAM consuming monster.  For 

>>>>> something RAM limited like the 2220 I'd use a different, simpler 

>>>>> file format.  Then I'd use rsync based snapshots.

>>>>> 

>>>>> Here's my personal backup script.  :-)  I haven't tried it FROM 

>>>>> pfsense, but I've used it to back up pfsense.

>>>>> 

>>>>>         ED.

>>>>> 

>>>>> 

>>>>> 

>>>>> 

>>>>> 

>>>>> On 2018, Feb 21, at 12:23 PM, Walter Parker <
<mailto:walt...@gmail.com> walt...@gmail.com> wrote:

>>>>>> 

>>>>>> Hi,

>>>>>> 

>>>>>> I have 2.4.2 installed on an SG-2220 from Netgate [nice box]. I 

>>>>>> just

>>>>>> 

>>>>> bought

>>>>> 

>>>>>> a 6TB powered USB drive from Costco and it works great (the drive 

>>>>>> has

>>>>>> 

>>>>> its

>>>>> 

>>>>>> own power supply and a USB hub). I want to use it take ZFS 

>>>>>> backups from

>>>>>> 

>>>>> my

>>>>> 

>>>>>> home server.

>>>>>> 

>>>>>> I edited /boot/loader.conf.local and /etc/rc.conf.local to load 

>>>>>> ZFS on

>>>>>> 

>>>>> boot

>>>>> 

>>>>>> and created a pool and a file system. That worked, but the memory 

>>>>>> ran

>>>>>> 

>>>>> low

>>>>> 

>>>>>> so I restricted the ARC cache to 1G to keep a bit more memory 

>>>>>> free and rebooted. When the system rebooted it did not remount 

>>>>>> the pool (and therefore the file system) because the pool what 

>>>>>> marked as in use by another system (itself). That means that the 

>>>>>> pool was not properly exported/umounted at shutdown.

>>>>>> 

>>>>>> Taking a quick look a rc.shutdown, I notice that it calls a 

>>>>>> customized pfsense shutdown script at the beginning and then 

>>>>>> exits. Is there a good place in the configuration where I can 

>>>>>> put/call the proper zfs shutdown script so that the pool is 

>>>>>> properly stopped/exported so that it imports correctly on boot?

>>>>>> 

>>>>>> 

>>>>>> Walter

>>>>>> 

>>>>>> --

>>>>>> The greatest dangers to liberty lurk in insidious encroachment by 

>>>>>> men of

>>>>>> zeal, well-meaning but without understanding.   -- Justice Louis D.

>>>>>> 

>>>>> Brandeis

>>>>> 

>>>>>> _______________________________________________

>>>>>> pfSense mailing list

>>>>>>  <https://lists.pfsense.org/mailman/listinfo/list>
https://lists.pfsense.org/mailman/listinfo/list

>>>>>> Support the project with Gold!  <https://pfsense.org/gold>
https://pfsense.org/gold

>>>>>> 

>>>>> 

>>>>> 

>>>>> 

>>>> --

>>>> The greatest dangers to liberty lurk in insidious encroachment by men
of

>>>> zeal, well-meaning but without understanding.   -- Justice Louis D.

>>>> Brandeis

>>>> 

>>>> 

>>> 

>>> 

>> --

>> Best Regards

>> Curtis Maurand

>> Principal

>> Xyonet Web Hosting

>>  <mailto:cmaur...@xyonet.com> mailto:cmaur...@xyonet.com

>>  <http://www.xyonet.com> http://www.xyonet.com

>> 

>> _______________________________________________

>> pfSense mailing list

>>  <https://lists.pfsense.org/mailman/listinfo/list>
https://lists.pfsense.org/mailman/listinfo/list

>> Support the project with Gold!  <https://pfsense.org/gold>
https://pfsense.org/gold

>> 

> 

> 

> 

> --

> The greatest dangers to liberty lurk in insidious encroachment by men of

> zeal, well-meaning but without understanding.   -- Justice Louis D.
Brandeis

> _______________________________________________

> pfSense mailing list

>  <https://lists.pfsense.org/mailman/listinfo/list>
https://lists.pfsense.org/mailman/listinfo/list

> Support the project with Gold!  <https://pfsense.org/gold>
https://pfsense.org/gold

> 

 

_______________________________________________

pfSense mailing list

 <https://lists.pfsense.org/mailman/listinfo/list>
https://lists.pfsense.org/mailman/listinfo/list

Support the project with Gold!  <https://pfsense.org/gold>
https://pfsense.org/gold

_______________________________________________
pfSense mailing list
https://lists.pfsense.org/mailman/listinfo/list
Support the project with Gold! https://pfsense.org/gold

Reply via email to