RE: btrfs-progs - failed btrfs replace on RAID1 seems to have left things in a wrong state
Duncan, Thank you for your thorough response to my problem. I am now wiser in my understanding of how btrfs works in RAID1 thanks to your words. Last night I worked with someone in the IRC channel and we essentially came to the exact same conclusion. I used wipefs -a on the errant drive. Rebooted and viola. As of last night the replace was running fine. (Didn't have time to check this morning before heading out) The people on IRC had recommended filing a bug based on the fact that a btrfs filesystem was created during the replace, but if I understand your feedback, this has already been noted and there are patches being considered. As for your backup feedback, it has been thoroughly beaten into my head over the last half-decade that RAID is not backup. Although, I'd argue that RAID on btrfs or ZFS making use of snapshots is pretty darn close. (Since it covers the fat-finger situation - although it doesn't cover the MOBO frying your hard-drives situation) But I do have offsite backup - it's just that it's with a commercial provider (as opposed to, say, a friend's house) so I didn't want to have to download 3TB if things got borked. (I consider that my house burning/theft backup) And it IS in the plans to have a separate backup system in my house. I just haven't spent the money yet as it's currently a bit tight. But I do appreciate that you took the time to explain that in case I didn't know about it. And it's on the mailing list archives now so if someone else is under the misunderstanding that RAID is backup they can also be educated. Anyway, this is running a bit long. I just want to conclude by again offering my thanks at your very thorough response. If I hadn't been able to obtain help on the IRC, this would have put me on the right path. And it came with knowledge rather than just a list of instructions. So thanks for that as well. -- Eric Mesa http://www.ericmesa.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs-progs - failed btrfs replace on RAID1 seems to have left things in a wrong state
tion going on: # btrfs device remove /dev/sda /media/Photos/ ERROR: error removing device '/dev/sda': unable to go below two devices on raid1 # btrfs device remove /dev/sdb /media/Photos/ ERROR: error removing device '/dev/sdb': unable to go below two devices on raid1 # btrfs device remove /dev/sde /media/Photos/ ERROR: error removing device '/dev/sde': unable to go below two devices on raid1 who (filesystem? disk? some program?) maintains the info on what was going on with /dev/sda? I feel like there's some kind bit I need to clear and then it'll work correctly. --- So my question is two-fold. 1) Where do I go from here to get things working for me? I have my photos on these drives (which is why I went RAID1 so I could have high availability backup-ish situation) so I don't want to do anything destructive to the two drives currently working fine in the array. 2) The fact that a failed replace left the (system|disks|filesystem) thinking that the drive is both part of and not part of the RAID1 - does that need to be reported as a bug? -- Eric Mesa http://www.ericmesa.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Duncan 1i5t5.duncan at cox.net writes: *But*, btrfs snapshots by themselves remain on the existing btrfs filesystem, and thus are subject to many of the same risks as the filesystem itself. As you mentioned raid is redundancy not backup, snapshots aren't backup either; snapshots are multiple logical copies thus protecting you from accidental deletion or bad editing, but pointed at the same data blocks without redundancy, and if those data blocks or the entire physical media go bad... Which is where real backups, separate copies on separate physical media, come in, which is where btrfs send/receive, as the ars-technica article was describing, comes in. The idea is to make a read-only snapshot on the local filesystem, read- only so it can't change while it's being sent, and then use btrfs send to send that snapshot to be stored on some other media, which can optionally be over the network to a machine and media at a different site, altho it can be to a different device on the same machine, as well. The first time you do this, there's no existing copy at the other end, so btrfs send sends a full copy and btrfs receive writes it out. After that, the receive side has a snapshot identical to the one created on the send side and further btrfs send/receives to the same set simply duplicate the differences between the reference and the new snapshot from the send end to the receive end. As with local snapshots, old ones can be deleted on both the send and receive ends, as long as at least one common reference snapshot is maintained on both ends, so diffs taken against the send side reference can be applied to an appropriately identical receive side reference, thereby updating the receive side to match the new read-only snapshot on the send side. Hopefully that's clearer now. =:^) Duncan - thanks for this comprehensive explanation. For a huge portion of your reply...I was all wondering why you and others were saying snapshots aren't backups. They certainly SEEMED like backups. But now I see that the problem is one of precise terminology vs colloquialisms. In other words, snapsshots are not backups in and of themselves. They are like Mac's Time Machine. BUT if you take these snapshots and then put them on another media - whether that's local or not - THEN you have backups. Am I right, or am I still missing something subtle? I think the most important thing you said was at the end and I'd like a little clarification on that if it's OK with you. As with local snapshots, old ones can be deleted on both the send and receive ends, as long as at least one common reference snapshot is maintained on both ends, so diffs taken against the send side reference can be applied to an appropriately identical receive side reference, thereby updating the receive side to match the new read-only snapshot on the send side. So, let's say I have everything set up. This means I created the read-only shot on my home btrfs volume and sent it to the backup drive. I'm making hourly snapshots and after each snapshot is made, it's sent to the backup drive. So, obviously the backup drive needs to be at least as big as the home drive so it can store what's on home plus the snapshot-diffs. Now let's be extreme and say that in the course of a year I touch and somehow change every single file on the home drive. That means if I only had one snapshot I'd need home drive x 2 space. (for used space, not unused space, naturally) So I might want my backups to have last's year's data, but wouldn't want to need to upgrade the size of my actual home drive. So I would want to maintain less snapshots on my home drive than my backup drive. (It's possible I'm missing something here...something subtle that makes this not necessary) So do I only need to make sure I have the latest snapshot or maybe latest plus n-1 on the home drive while the backup drive can have all snapshots since the beginning? I THINK that can be the case based on reading your sentence, but I just want to make sure. In case you were wondering, this is based on what's happened to me with Back in Time. I had to reduce the number of backups I was keeping because my home drive wasn't at 100%, but the backupdrive was at 100% because I'd added and deleted some VMs and other large files (video files I think). And Back in Time intelligently does not remove the oldest backup off the top until it knows it has made a new backup - which it couldn't do because it was at 100%. So I had to delete the top 1 or 2 backups and then tell it to keep less backups. Your description of snapshots makes it seems much less likely that this would be an issue. Although Back in Time is an incremental backup, its takes up more space. If I may venture to see if I've learned something from your response, is it because when I change a file Back in Time stores the entire changed file while btrfs only stores the bits that have changed? Also, does it
Understanding btrfs and backups
apologies if this is a resend - it appeared to me that it was rejected because of something in how Gmail was formatting the message. I can't find it in the Gmane archives which leads me to believe it was never delivered. I was hoping to gain some clarification on btrfs snapshops and how they function as backups. I did a bit of Googling and found lots of examples of bash commands, but no one seemed to explain what was going on to a level that would satisfy me for my data needs. I read this Ars Technica article today http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/ First of all, the btrfs-raid1 sounds awesome. Because it helps protect against one of RAID1's failings - bit rot issues. But raid1 is not backup, it's just redundancy. Second, the article mentions using snapshots as a backup method. Page 3 section: Using the features. He makes a snapshot and sends that. Then he sends what changed the second time. He mentions that because btrfs knows what's changed it's a quick process. Right now on my Linux computer I use Back in Time which, I think, is just an rsync frontend. It takes a long time to complete the backup for my 1 TB /home drive. The copy part is nice and quick, but the comparison part takes a long time and hammers the CPU. I have it setup to run at night because if it runs while I'm using the computer, things can crawl. So I was wondering if btrfs snapshots are a substitute for this. Right now if I realize I deleted a file 5 days ago, I can go into Back in Time (the gui) or just navigate to it on the backup drive and restore that one file. From what I've read about btrfs, I'd have to restore the entire home drive, right? Which means I'd lose all the changes from the past five days. If that's the case, it wouldn't really solve my problem - although maybe I'm just not thinking creatively. Also, if I first do the big snapshot backup and then the increments, how do I delete the older snapshots? In other words, the way I'm picturing things working is that I have the main snapshot and every snapshot after that is just a description of what's changed since then. So wouldn't the entire chain be necessary to reconstruct where I'm at now? On a somewhat separate note, I have noticed that many people/utilities for btrfs mention making snapshots every hour. Are the snapshots generally that small that such a think wouldn't quickly fill a hard drive? Thanks for reading my questions, I appreciate the help. When all is said and done I'd certainly like to publish a how-to from my point of undertanding. -- Eric Mesa -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Brian Wong wrote: a snapshot is different than a backup, with a snapshot you're still accessing a read-only version of the live filesystem. i don't know the specifics of btrfs but if you take daily snapshots, you should be able to restore a single file from the five-days-ago snapshot by browsing that snapshot's directory tree and then copying the file to the live version of the filesystem, if that makes sense. in the snapshot case the live filesystem serves the same function as the full backup would if you did full backups then incrementals. the snapshots are the incrementals of the live filesystem, only going backwards in time whereas with backup you would take a full backup then go forward in time with incrementals. the filesystem takes care of making sure every snapshot is complete. in the snapshot case redundancy is then more important because you may not have a bunch of full backups (i.e. full copies) lying around. so full backups still are useful. -- OK, I THINK I understand things a bit better. So from the point of view of restoring a single file, that functionality is there. Excellent. And I guess you're saying that because the snapshots are diffs off the live system, that I'd need a backup of the live system - ie snapshots wouldn't be enough. But what if my first snapshot was a clone of the system at that point (as it seems from the article) And I back that up to a separate drive. Let me illustrate with what I plan to do exactly. Three hard drives: A, B, and C. Hard drives A and B - btrfs RAID-1 so that if one drive dies I can keep using my system until the replacement for the raid arrives. Hard drive C - gets (hourly/daily/weekly/or some combination of the above) snapshots from the RAID. (Starting with the initial state snapshot) Each timepoint another snapshot is copied to hard drive C. So in the case of a file disappearing on me or being over-written or w/e - I reach into the directory of the snapshot that contains the file just as I would now with the backup. So if that's what I'm doing, do snapshots become a way to do backups? Thanks -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Brian Wong wrote: a snapshot is different than a backup, with a snapshot you're still accessing a read-only version of the live filesystem. i don't know the specifics of btrfs but if you take daily snapshots, you should be able to restore a single file from the five-days-ago snapshot by browsing that snapshot's directory tree and then copying the file to the live version of the filesystem, if that makes sense. in the snapshot case the live filesystem serves the same function as the full backup would if you did full backups then incrementals. the snapshots are the incrementals of the live filesystem, only going backwards in time whereas with backup you would take a full backup then go forward in time with incrementals. the filesystem takes care of making sure every snapshot is complete. in the snapshot case redundancy is then more important because you may not have a bunch of full backups (i.e. full copies) lying around. so full backups still are useful. -- OK, I THINK I understand things a bit better. So from the point of view of restoring a single file, that functionality is there. Excellent. And I guess you're saying that because the snapshots are diffs off the live system, that I'd need a backup of the live system - ie snapshots wouldn't be enough. But what if my first snapshot was a clone of the system at that point (as it seems from the article) And I back that up to a separate drive. Let me illustrate with what I plan to do exactly. Three hard drives: A, B, and C. Hard drives A and B - btrfs RAID-1 so that if one drive dies I can keep using my system until the replacement for the raid arrives. Hard drive C - gets (hourly/daily/weekly/or some combination of the above) snapshots from the RAID. (Starting with the initial state snapshot) Each timepoint another snapshot is copied to hard drive C. So in the case of a file disappearing on me or being over-written or w/e - I reach into the directory of the snapshot that contains the file just as I would now with the backup. So if that's what I'm doing, do snapshots become a way to do backups? Thanks -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
