Forgot to CC the list. On Wed, Feb 28, 2018 at 10:13 PM, Walter Parker <walt...@gmail.com> wrote:
> Thank you for the backup script. > > By my calculations, 2G should be enough. If I limit the ARC cache to 1G, > that leaves 1G for applications & kernel memory. As I'm not serving the 6TB > drive up as a file server, but using it for one specific task (to receive > the backups from one host) I figure that I don't need lots of memory. ZFS > as a quick file server or busy server needs lots of memory to be quick. > I've seen testing showing ZFS doing fast file copies on as little as 768M > total system after proper memory tuning. > > I need ZFS because it is the only file system that can receive incremental > ZFS snapshots and apply them. I have not setup the ZFS backup software yes, > so I'm just using rsnapshot. First time it ran, it filled all 1G of the > cache. I rebooted the firewall afterwards and now ZFS with 60-100M of usage > (the amount of data that rsync updates on a daily basis is pretty small). > Right now, the data from the other server is ~8.8G, compressed to 1.7G with > lz4. > > When I get the full backup running, I will be ~1.5TB in size. ZFS > snapshots should be pretty small and quick (as it can send just the data > that was updated without having to walk the entire filesystem). An rsync > backup would have to walk the whole system to find all of the changes. Most > of the data on the system doesn't change (as it is a media library). > > I'll post back more results if people are interested, after I get the > backup software working (I'm thinking about using ZapZend). > > > Walter > > > > On Wed, Feb 28, 2018 at 8:54 PM, ED Fochler <soek...@liquidbinary.com> > wrote: > >> I feel like I'm late in responding to this, but I have to say that 2GB of >> RAM doesn't seem like nearly enough for a 6TB zfs volume. ZFS is great in >> a lot of ways, but is a RAM consuming monster. For something RAM limited >> like the 2220 I'd use a different, simpler file format. Then I'd use rsync >> based snapshots. >> >> Here's my personal backup script. :-) I haven't tried it FROM pfsense, >> but I've used it to back up pfsense. >> >> ED. >> >> >> >> >> >> > On 2018, Feb 21, at 12:23 PM, Walter Parker <walt...@gmail.com> wrote: >> > >> > Hi, >> > >> > I have 2.4.2 installed on an SG-2220 from Netgate [nice box]. I just >> bought >> > a 6TB powered USB drive from Costco and it works great (the drive has >> its >> > own power supply and a USB hub). I want to use it take ZFS backups from >> my >> > home server. >> > >> > I edited /boot/loader.conf.local and /etc/rc.conf.local to load ZFS on >> boot >> > and created a pool and a file system. That worked, but the memory ran >> low >> > so I restricted the ARC cache to 1G to keep a bit more memory free and >> > rebooted. When the system rebooted it did not remount the pool (and >> > therefore the file system) because the pool what marked as in use by >> > another system (itself). That means that the pool was not properly >> > exported/umounted at shutdown. >> > >> > Taking a quick look a rc.shutdown, I notice that it calls a customized >> > pfsense shutdown script at the beginning and then exits. Is there a good >> > place in the configuration where I can put/call the proper zfs shutdown >> > script so that the pool is properly stopped/exported so that it imports >> > correctly on boot? >> > >> > >> > Walter >> > >> > -- >> > The greatest dangers to liberty lurk in insidious encroachment by men of >> > zeal, well-meaning but without understanding. -- Justice Louis D. >> Brandeis >> > _______________________________________________ >> > pfSense mailing list >> > https://lists.pfsense.org/mailman/listinfo/list >> > Support the project with Gold! https://pfsense.org/gold >> >> >> > > > -- > The greatest dangers to liberty lurk in insidious encroachment by men of > zeal, well-meaning but without understanding. -- Justice Louis D. Brandeis > -- The greatest dangers to liberty lurk in insidious encroachment by men of zeal, well-meaning but without understanding. -- Justice Louis D. Brandeis _______________________________________________ pfSense mailing list https://lists.pfsense.org/mailman/listinfo/list Support the project with Gold! https://pfsense.org/gold
