Niccolò Belli posted on Thu, 12 May 2016 15:56:20 +0200 as excerpted: > Thanks for the detailed explanation, hopefully in the future someone > will be able to make defrag snapshot/reflink aware in a scalable manner.
It's still planned, AFAIK, but one of the scaling issues in particular, quotas, have turned out to be a particularly challenging thing to even actually get working correctly. They've rewritten the quota code twice (so they're on their third attempted solution), and it's still broken in certain corner-cases ATM, to the point that while they're still actually trying to get the existing third try to work in the tough corner-cases as well, they're already talking about an eventual third rewrite (4th attempt, having scrapped three) once they actually have the corner-cases working, to try to bring better performance once they know the tough corner-cases and can actually design a solution with both them and performance in mind from the beginning. So in practice an actually scalable snapshot-aware defrag is likely to be years out, as it's going to need actually working and scalable quota code, and even then, that's only part of the full scalable snapshot/ reflink-aware defrag solution. The good news is that while there's still work to be done, progress has been healthy in other areas, so once the quota code both actually works and is scalable, the other aspects should hopefully fall into place relatively fast, as they've already been maturing on their own, separately. > I will not use use defrag anymore, but what do you suggest me to do to > reclaim the lost space? Get rid of my current snapshots or maybe simply > running bedup? Neither snapshots nor dedup are one of my direct use-cases so my practical knowledge there is limited, but removing the snapshots should indeed clear the space (but you'll likely have to remove all of them covering a specific subvolume in ordered to free the space) as in doing so you'll be removing all references locking the old extents in place. If you already have them backed up (using send/receive, for instance) elsewhere or don't actually need them, however, it's a viable alternative. In theory the various btrfs dedup solutions out there should work as well, while letting you keep the snapshots (at least to the extent they're either writable snapshots so can be reflink modified, or a single read-only snapshot that the others including the freshly defragged working copy can be reflinked to), since that's their mechanism of operation -- finding identical block sequences and reflinking them so there's only one actual copy on the filesystem, with the rest being reflinks to it -- so in effect it should undo the reflink-breaking you did with the defrag. *But*, without any personal experience with them, I have no idea either how effective they are in practice in a situation like this, or how practical vs. convoluted the commandlines are going to be to actually accomplish your goal. Best-case, it's a simple and fast command to run and it not only undoes the defrag reflink breakage, but actually finds enough duplication in the dataset to reduce usage even further than before, worst-case it's multiple complex commands that take a week or longer to run and don't actually help much. So in practice, you have a choice between EITHER deleting all the snapshots and along with them everything locking down the old extents, thus leaving you with only the new, fresh copy (which by itself should be smaller than before), but at the expense of losing your snapshots, OR the at least from my knowledge relative unknown of the various btrfs dedup solutions, which in theory should work well, but in practice... I simply don't know. AND of course you have the option of basically doing nothing, leaving things as they are. However, given the context of this thread, it seems you don't consider that a viable longer term option as apparently you were trying to clear space, not use MORE of it, and presumably you actually need that space for something else, and that precludes just letting things be, unless of course you can afford to simply buy your way out of the problem with more storage devices. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
