Rich Freeman posted on Mon, 13 Oct 2014 16:42:14 -0400 as excerpted: > On Mon, Oct 13, 2014 at 4:27 PM, David Arendt <ad...@prnet.org> wrote: >> From my own experience and based on what other people are saying, I >> think there is a random btrfs filesystem corruption problem in kernel >> 3.17 at least related to snapshots, therefore I decided to post using >> another subject to draw attention from people not concerned about btrfs >> send to it. More information can be found in the brtfs send posts. >> >> Did the filesystem you tried to balance contain snapshots ? Read only >> ones ? > > The filesystem contains numerous subvolumes and snapshots, many of which > are read-only. I'm managing many with snapper. > > The similarity of the transid verify errors made me think this issue is > related, and the root cause may have nothing to do with btrfs send. > > As far as I can tell these errors aren't having any affect on my data - > hopefully the system is catching the problems before there are actual > disk writes/etc.
Summarizing what I've seen on the threads... 1) The bug seems to be read-only snapshot related. The connection to send is that send creates read-only snapshots, but people creating read- only snapshots for other purposes are now reporting the same problem, so it's not send, it's the read-only snapshots. 2) Writable snapshots haven't been implicated yet, and the working set from which the snapshots are taken doesn't seem to be affected, either. So in that sense it's not affecting ordinary usage, only the read-only snapshots themselves. 3) More problematic, however, is the fact that these apparently corrupted read-only snapshots often are not listed properly and can't be deleted, tho I'm not sure if that's /all/ the corrupted snapshots or only part of them. So while it may not affect ordinary operation in the short term, over time until there's a fix, people routinely doing read-only snapshots are going to be getting more and more of these undeletable snapshots, and depending on whether the eventual patch only prevents more or can actually fix the bad ones (possibly via btrfs check or the like), affected filesystems may ultimately have to be blown away and recreated with a fresh mkfs, in ordered to kill the currently undeletable snapshots. So the first thing to do would be to shut off whatever's making read-only snapshots, so you don't make the problem worse while it's being investigated. For those who can do that without too big an interruption to their normal routine (who don't depend on send/receive, for instance), just keep it off for the time being. For those who depend on read-only snapshots (send-receive for backup and the data is too valuable to not do the backups for a few days), consider switching back to 3.16-stable -- from 3.16.3 at least, the patch for the compress bug is there, so that shouldn't be a problem. And if you're affected, be aware that until we have a fix, we don't know if it'll be possible to remove the affected and currently undeletable snapshots. If it's not, at some point you'll need to do a fresh mkfs.btrfs, to get rid of the damage. Since the bug doesn't appear to affect writable snapshots or the "head" from which snapshots are made, it's not urgent, and a full fix is likely to include a patch to detect and fix the problem as well, but until we know what the problem is we can't be sure of that, so be prepared to do that mkfs at some point, as at this point it's possible that's the only way you'll be able to kill the corrupted snapshots. 4) Total speculation on my part, but given the wanted transid (aka generation, in different contexts) is significantly lower than the found transid, and the fact that the problem appears to be limited to /read-only/ snapshots, my first suspicion is that something's getting updated that would normally apply to all snapshots, but the read-only nature of the snapshots is preventing the full update there. The transid of the block is updated, but the snapshot being read-only is preventing update of the pointer in that snapshot accordingly. What I do /not/ know is whether the bug is that something's getting updated that should NOT be, and it's simply the read-only snapshots letting us know about it since the writable snapshots are fully updated, even if that breaks the snapshot (breaking writable snapshots in a different and currently undetected way), or if instead, it's a legitimate update, like a balance simply moving the snapshot around but not affecting it otherwise, and the bug is that the read-only snapshots aren't allowing the legitimate update. Either way, this more or less developed over the weekend, and it's Monday now, so the devs should be on it. If it's anything like the 3.15/3.16 compression bug, it'll take some time for them to properly trace it, and then to figure out an appropriate fix, but they will. Chances are we'll have at least some decent progress on a trace by Friday, and maybe even a good-to-go patch. =:^) -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
