Greetings Rubin, Nice plug for rsync...
Flint On Fri, 7 Oct 2016, Rubin Bennett wrote:
Date: Fri, 7 Oct 2016 10:51:27 -0400 From: Rubin Bennett <ru...@rbtechvt.com> Reply-To: Vermont Area Group of Unix Enthusiasts <VAGUE@list.uvm.edu> To: VAGUE@LIST.UVM.EDU Subject: Re: Backups and File Encryption All of Renee?s points are excellent, and expanding them a bit: There?s a best practice for backups called the 3-2-1 rule, and it calls for 3 copies of your data, on 2 separate media, with 1 being offsite. In practice, you probably want more than 3 copies of your data, especially if you?re looking to cover human error (the file you?ll never need again that was deleted last week, that you need today, or the folder that was inadvertently deleted by someone who shall go unnamed 3 weeks ago that you only noticed was missing today). For our physical Linux boxen we use rsync over ssh from a backup server behind our 2 layer firewalls with SSH key exchange to get a full copy of all the files on the machine. We run a rotating backup where we keep 15 daily copies of the servers. When using tar or rsync or any file level backup, bare metal recovery is always a pain in the ass because the likelihood of the replacement machine having the same RAID controller and storage layout is next to nil, so the LVM volumes and partitions will need to be rebuilt/ reassigned on the new hardware. For virtualized environments we use VMware and Veeam, which work perfectly for us and have a plethora of options for offsite backups, secondary backup targets, etc. R Rubin Bennett Owner & Senior Network Engineer rbTechnologies, LLC 1970 Vermont Rt. 14 South East Montpelier, VT 05601 802.223.4448 <http://rbtechvt.com/> http://rbtechvt.com Thoughtfully managed, custom crafted business computer networks and communications systems since 1997 From: Vermont Area Group of Unix Enthusiasts [mailto:VAGUE@list.uvm.edu] On Behalf Of Rene Churchill Sent: Friday, October 07, 2016 9:01 AM To: VAGUE@LIST.UVM.EDU Subject: Re: Backups and File Encryption Here's my thoughts/rant on backup systems. First of all, you need to consider which problems you're trying to solve with backups: * Hardware failure - To deal with simple hardware failure like the drive itself failing, you just need a copy of the data. The amount of time between copies is the amount of risk you're exposing yourself to. If you can deal with redoing a days' work, then nightly backups/copies are good enough. RAID 1/mirrored drives provide an instantaneous copy. Rsync fired via cron can provide other windows of risk, like hourly, etc. Pick the amount of risk you're willing to deal with. * Human failure - "Oh drat, I really do need that file I deleted last week!" or "That edit turned out wrong, I wish I could go back to the version I had 24hrs ago." To deal with this kind of problem, you need multiple copies of your data spaced out over time. This is where incremental backups help because keeping 30 copies of all your data gets expensive real fast. * Change tracking or Pointing the finger o' blame - Some systems have a need or even a legal responsibility to track/restrict who can make changes and to log when those changes were made. Source code control like SVN/git work well here and can provide a complete history of changes stretching back many years. But you're not going to put commonly changed files like your email Inbox into git. * Catastrophic failure - Things like fire, theft and police raids on the data center seizing all the servers, or the data center going out of business suddenly. This is where you need to have a copy of your data in a separate location. This is also where encryption of your backups becomes important. How much do you trust the other location where a copy of your data resides? * How much data are you dealing with here? - This is incremental vs full backups. A writer can keep backups of their books in progress on a flash drive tucked into their pocket. It'll take them 30 seconds to copy it back and forth. A musician or video editor is going to have multiple TB of data to move around and that takes significant time. The backup need to finish before the next work period starts. If your office/business works multiple shifts, that further reduces the window you have to do the backup. My personal solution, which isn't going to fit everybody, is: * SVN on a central file server for all my projects that require long-term tracking. * Nightly rsync of all my external server data to a large drive on a server here in the office. * For the couple of MySQL servers where I can't afford the nightly 15 minute down time to lock and then back up all of the data, I run a replication server and lock that up instead. I keep 7 nightly copies of the database dumps which are further backed up by the file system backups. * Retrospect (https://www.retrospect.com/) to backup all of the machines here in the office. It's a windows product, but pretty decent and it'll encrypt the backups. I do incremental backups during the week to an external hard drive. On Friday, I swap the drive and it does a full backup on Fri. nightly which takes ~16hrs. I bring the other drive home and I've got 4 of them in rotation, so I've got 4 weeks worth of copies. My office is ~1 mile from my house. My usual joke is that if there's a catastrophe big enough to take out both my office and my home, it's probably taken out me as well, so at that point, I won't give a shit about the backups being unrecoverable. I hope some of that helps, Rene On 10/6/2016 10:08 PM, Joe Golden wrote: I feel like I should get a little more serious about a backup system and encryption on my filesystems. I use ssh for command line access to texty things for projects, etc. I use sshfs for mounting a shared directory for working space between a server, desktop and a couple of laptops. The important bits live on the server. What's the standard recommended encryption for an encrypted home dir? Looks like encfs is a good bet. And for backups, Anthony mentioned Git Annex. Is that a backup solution or something similar? In general I don't need incremental backups, but if didn't cost much and made things faster, all the better. I love git and think git should be in more places, and love the distributed idea. Sorry I know this is a big question. Any recommendations from the list appreciated. PS: we should do beerz sometime. -- _____ René Churchill VP of Development (i.e. Geek #2) WherezIt.com - Your source for Local information r...@wherezit.com <mailto:r...@wherezit.com> 802-244-7880 x527 http://www.wherezit.com/
Kindest Regards, ☮ Paul Flint (802) 479-2360 Home (802) 595-9365 Cell /************************************ Based upon email reliability concerns, please send an acknowledgement in response to this note. Paul Flint 17 Averill Street Barre, VT 05641
