On Sun, Aug 31, 2008 at 3:12 PM, Matthew Dillon <[EMAIL PROTECTED]> wrote: > You don't need to use the hardlink trick if backing up to a HAMMER > filesystem. I still need to write utility support to streamline > the user interface but basically all you have to do is use rdist, rsync, > or cpdup (without the hardlink trick) to overwrite the same destination > directory on the HAMMER backup system, then generate a snapshot > softlink. Repeat each day.
In-filesystem snapshot support is such a handy tool. It's something that I really miss on our Linux systems (LVM snapshots are separate volumes, and you have to guesstimate how much room each one will use, and you have to leave empty space in your volume group to support them). We use a similar setup for our remote backups box at work. It's a 2x dual-core Opteron system with 8 GB of RAM and 12x 400 GB SATA HDs on one 3Ware controller and 12x 500 GB SATA HDs on a second 3Ware controller (all configured as Single Disks), running FreeBSD 7-STABLE off a pair of 2 GB CompactFlash cards (gmirror'd). / is on the CF, everything else (/usr, /usr/ports, /usr/local, /usr/ports/distfiles, /usr/src, /usr/obj, /home, /tmp, /var, /storage) are ZFS filesystems (the 24 drives are configured as a single raidz2 pool). There's a quad-port Intel Pro/1000 gigabit NIC configured via lagg(4) as a single load-balancing interface. Every night, a cronjob creates a snapshot of /storage, then the server connects to the remote servers via SSH, runs rsync against the entire harddrive and a directory under /storage. For 37 servers, it takes just under 2 hours for the rsync runs (the initial rsync can takes upwards of 12 hours per server, depending on the amount of data that needs to be transferred). A normal snapshot uses <4 GB. >From the sounds of things, a DragonFlyBSD system could be used in a similar fashion, with Hammer filesystems. <snip> > By my estimation it is going to take at least another 200+ days of > daily backups before I get to that point on my /backup system. I > may speed it up by creating some filler files so I can write and test > a user utility to do the management. We've used just over 1 TB to completely archive 37 servers. Daily snapshots use <5 GB each. This particular server (9 TB) should last us for a couple of years. :) Even after we get the full 75 remote servers being backed up, we should be good to keep at least 6 months of daily backups online. :) > -- > Another way of doing backups is to use the mirroring feature. This only > works when both the source and target filesystems are HAMMER filesystems > though, and the snapshot softlink would have to be created manually > (so we need more utility support to make it easier for userland to do). Or, you can use the mirror feature to mirror your backup server to an offsite server. :) That's what we're planning on doing with ours, using the "snapshot send" and "snapshot receive" features in ZFS. There's lots of great work going on in filesystems right now. It's nice to see the BSDs up near the front (FreeBSD with ZFS, DFlyBSD with Hammer) again. -- Freddie Cash [EMAIL PROTECTED]
