Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Dan Ritter writes: > Peter Wiersig wrote: > > ZFS is now in two incompatible versions: Oracle's, and ZFSonLinux, > which is now effectively the parent for all the other efforts including > FreeBSD's ZFS. The biggest problem is the incompatible license which makes the code untouchable. How Oracle is able to distribute a Linux kernel with ZFS support without either releasing the whole thing under a GPL compatible license or violating the kernel GPL is a miracle I didn't found the time or need to investigate. I consider ZFS a poisoned gift from Oracle to the Linux community and will not be surprised when they go for one vendor, on their whim. If they wanted to, they could release their code under a compatible license. My knowledge on the whole affair is bit vague as I had no real interest past reading LWN or cks articles on ZFS, and I noted the problems around the CDDL once Illumos was released. I understand from cks that the real development nowadays happens on ZFSonLinux as the other OSes were stopped in development, but I would not touch that patch on the future prospect on a lawsuit analog to the SCO one. from https://github.com/zfsonlinux/zfs/wiki/FAQ#licensing """Licensing ZFS is licensed under the Common Development and Distribution License (CDDL), and the Linux kernel is licensed under the GNU General Public License Version 2 (GPLv2). While both are free open source licenses they are restrictive licenses. The combination of them causes problems because it prevents using pieces of code exclusively available under one license with pieces of code exclusively available under the other in the same binary. In the case of the kernel, this prevents us from distributing ZFS on Linux as part of the kernel binary. However, there is nothing in either license that prevents distributing it in the form of a binary module or in the form of source code.""" Peter
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Peter Wiersig wrote: > Matthew Crews writes: > > > > Here is a good talk on the subject by Michael Lucas, one of the premier > > experts on ZFS. Its worth noting that a lot of the concepts apply to > > BTRFS to varying degrees: > > > > https://www.youtube.com/watch?v=x9A0dX2WqW8 > > I don't have time yet, I think I will watch the whole thing later the > week. > > But I hate the ZFS pushing going around, I think it's a almost fine idea > from the technical standpoint, it will work if you decide to use it, but > > I surely don't trust the commercial parties behind ZFS as > - they have acted malicious in the past, There are no commercial parties behind ZFS. The code came from Sun, who open-sourced it. Oracle bought Sun, and closed further development. Open Solaris was forked as Illumos, Open Indiana, and others, and continued ZFS development. FreeBSD adopted it fairly quickly. All these efforts came together in OpenZFS. ZFS is now in two incompatible versions: Oracle's, and ZFSonLinux, which is now effectively the parent for all the other efforts including FreeBSD's ZFS. Oracle? Yeah, you can't trust Oracle any further than you can spit Larry Ellison. They aren't involved in anything we do with ZFS anymore. > I would never put my business on ZFS, I'd always go the "more backups, > and maybe less perfect GPL solution" route. You have the choice. -dsr-
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Matthew Crews writes: > > Here is a good talk on the subject by Michael Lucas, one of the premier > experts on ZFS. Its worth noting that a lot of the concepts apply to > BTRFS to varying degrees: > > https://www.youtube.com/watch?v=x9A0dX2WqW8 I don't have time yet, I think I will watch the whole thing later the week. But I hate the ZFS pushing going around, I think it's a almost fine idea from the technical standpoint, it will work if you decide to use it, but I surely don't trust the commercial parties behind ZFS as - they have acted malicious in the past, - cost us in the linux community a whole lot of energy with that lawsuit - were bought by the most twisting corporation which put Microsofts embrace, extend and extinguish campaigns back on the amateur level. I would never put my business on ZFS, I'd always go the "more backups, and maybe less perfect GPL solution" route. I would really like if the hostile Linux patchset would be off-topic here and sorry BSD guys, while I know that there were (or are) things like debian-kfreebsd, but I'm just your average (A/L)GPL-fanboy which only grudgingly accepts BSD licensed code. Because FSFs license protects me better as a developer and as a user, is a non-malicous cancer from my point of view. And yes, the FSF GFDL is a mistake. Peter
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
On Sat, Apr 13, 2019 at 7:08 PM Felix Miata wrote: > > Anders Andersson composed on 2019-04-13 17:31 (UTC+0200): > > > Felix Miata wrote: > > >> Because of its snapshotting, BTRFS requires considerably more space than > >> older > >> filesystems, as much as double. > > > A btrfs snapshot takes approximately zero space. Where did you get > > this idea from? > > (not an exhaustive list) > > 1: "Disk Space Full Because of Snapper" on https://en.opensuse.org/SDB:BTRFS > > 2: Since 2015, BTRFS has been the default / filesystem on openSUSE, which > recommends minimum / filesystem size of 20GB for EXT4, compared to 40GB for > BTRFS. > > 3: Much more common / filesystem freespace exhausted threads on mailing lists > and > web forums from BTRFS users compared to EXT4 users, with the usual > recommendation > to delete one or more snapshots to free space. > > 4: > https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_I_ran_out_of_disk_space.21 > > 5: > https://btrfs.wiki.kernel.org/index.php/FAQ#Why_is_free_space_so_complicated.3F > > 6: > https://www.linuxquestions.org/questions/linux-containers-122/docker-on-btrfs-using-much-space-in-var-lib-docker-btrfs-4175622037/#post5811463 Of course you will run out of space if you keep taking snapshots! Btrfs never does this, but you can do it *manually* or with third-party tools. When you take a snapshot, btrfs will keep everything until it's deleted. That's why everyone says that if you run out of space, you can delete snapshots. If the advise to the user is to delete a snapshot, it is something that the user did *because* they wanted to retain those files. The "Snapper" user has *installed* a tool that takes snapshots all the time - obviously you will run out of space because every file you modify or delete will still be stored in its original version until you remove the snapshot.
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
On 4/13/19 5:40 PM, Peter Wiersig wrote: > Peter Wiersig writes: >> >> I would be pissed if my OS removes snapshots I might or might not need >> in the future. That's a release critical bug in my eyes. Yeah, I know >> Microsoft and Apple do that automatically if your capacity runs out, but >> that's also why I don't recommend them at all. > > Ok, I checked https://en.opensuse.org/Portal:Snapper and they do support > LVM and ext4, and they have a bullet of auto-removing old snapshots. I > hope they did it right, perhaps I need to make a new test drive with the > latest release. > > Snapshots on ZFS can't be zero cost, so you need to account for them > there, too. > > Peter > ZFS Snapshots are nearly zero cost to create the snapshot, since ZFS (and likewise BTRFS) are copy-on-write file systems. What it does is records the deltas after the snapshots. This is a good thinkg as it saves on disk space. For example: You create a random 10 MB file, and take a snapshot. You then alter the 5 tail MBs and add five more MBs at the tail. You are left with: Pre-Snapshot | 5MB chunk #1 | 5MB chunk #2 | - total 10MBs Post-Snapshot | 5MB chunk #1 | 5MB chunk #3 | 5MB chunk #4 | - total 15MBs Actual disk usage will be 20 MBs, since the 5MB chunk #1 is only recorded on disk once, not twice. Here is a good talk on the subject by Michael Lucas, one of the premier experts on ZFS. Its worth noting that a lot of the concepts apply to BTRFS to varying degrees: https://www.youtube.com/watch?v=x9A0dX2WqW8 -Matt
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Peter Wiersig writes: > > I would be pissed if my OS removes snapshots I might or might not need > in the future. That's a release critical bug in my eyes. Yeah, I know > Microsoft and Apple do that automatically if your capacity runs out, but > that's also why I don't recommend them at all. Ok, I checked https://en.opensuse.org/Portal:Snapper and they do support LVM and ext4, and they have a bullet of auto-removing old snapshots. I hope they did it right, perhaps I need to make a new test drive with the latest release. Snapshots on ZFS can't be zero cost, so you need to account for them there, too. Peter
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Felix Miata writes: > Anders Andersson composed on 2019-04-13 17:31 (UTC+0200): > >> Felix Miata wrote: > >>> Because of its snapshotting, BTRFS requires considerably more space than >>> older >>> filesystems, as much as double. > >> A btrfs snapshot takes approximately zero space. Where did you get >> this idea from? > > 1: "Disk Space Full Because of Snapper" on https://en.opensuse.org/SDB:BTRFS i only skimmed that, good resource for anyone even not on SUSE. > 2: Since 2015, BTRFS has been the default / filesystem on openSUSE, which > recommends minimum / filesystem size of 20GB for EXT4, compared to 40GB for > BTRFS. >From my experience I think they only use snapshotting on BTRFS volumes, I don't know if they support it with LVM, but is it even possible with ext4 only? And yeah, no surprise, if you want snapshots, you'll need more capacity, it would probably the same recommendation with LVM And the effects of using snapshots is assuring, you simply set the system to auto-update everything, and if your system doesn't boot, you simply select the older snapshot from the grub menu and have your system running in no time. (This was my scenario with a SUSE desktop, I tried tumbleweed aka SUSEs unstable/experimental distribution, where some instability was expected) I would never recommend something like that for server updates/upgrades, as there a other far more procedures you can follow to test verify and prevent service loss on failed updates, if you simply cluster your services. I would be pissed if my OS removes snapshots I might or might not need in the future. That's a release critical bug in my eyes. Yeah, I know Microsoft and Apple do that automatically if your capacity runs out, but that's also why I don't recommend them at all. Have a monitoring on all your systems, track each and every possible value in compact rrd databases, calculate trends from those values and you'll never be surprised by filled up disks, growing defects detected by SMART etc. I'd like a debian desktop/notebook/tablet where snapshots were implemented in a openSUSE manner, so that I can simply forget about updates, have them installed in background und can go to the snapshot if problems arise. Peter
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Felix Miata wrote: > Anders Andersson composed on 2019-04-13 17:31 (UTC+0200): > >> Felix Miata wrote: > >>> Because of its snapshotting, BTRFS requires considerably more space than >>> older filesystems, as much as double. > >> A btrfs snapshot takes approximately zero space. Where did you get >> this idea from? > > (not an exhaustive list) > > 1: "Disk Space Full Because of Snapper" on > https://en.opensuse.org/SDB:BTRFS > > 2: Since 2015, BTRFS has been the default / filesystem on openSUSE, which > recommends minimum / filesystem size of 20GB for EXT4, compared to 40GB > for BTRFS. > > 3: Much more common / filesystem freespace exhausted threads on mailing > lists and web forums from BTRFS users compared to EXT4 users, with the > usual recommendation to delete one or more snapshots to free space. > > 4: > https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_I_ran_out_of_disk_space.21 > > 5: > https://btrfs.wiki.kernel.org/index.php/FAQ#Why_is_free_space_so_complicated.3F > > 6: > https://www.linuxquestions.org/questions/linux-containers-122/docker-on-btrfs-using-much-space-in-var-lib-docker-btrfs-4175622037/#post5811463 This time I like you :) and agree with you fully. ext4 or xfs - these are most convenient, although also zfs might be an option if one can manage it regards
Re: BTRFS snapshot space consumption (was: New laptop: need advice on choice...)
Anders Andersson composed on 2019-04-13 17:31 (UTC+0200): > Felix Miata wrote: >> Because of its snapshotting, BTRFS requires considerably more space than >> older >> filesystems, as much as double. > A btrfs snapshot takes approximately zero space. Where did you get > this idea from? (not an exhaustive list) 1: "Disk Space Full Because of Snapper" on https://en.opensuse.org/SDB:BTRFS 2: Since 2015, BTRFS has been the default / filesystem on openSUSE, which recommends minimum / filesystem size of 20GB for EXT4, compared to 40GB for BTRFS. 3: Much more common / filesystem freespace exhausted threads on mailing lists and web forums from BTRFS users compared to EXT4 users, with the usual recommendation to delete one or more snapshots to free space. 4: https://btrfs.wiki.kernel.org/index.php/FAQ#Help.21_I_ran_out_of_disk_space.21 5: https://btrfs.wiki.kernel.org/index.php/FAQ#Why_is_free_space_so_complicated.3F 6: https://www.linuxquestions.org/questions/linux-containers-122/docker-on-btrfs-using-much-space-in-var-lib-docker-btrfs-4175622037/#post5811463 -- Evolution as taught in public schools is religion, not science. Team OS/2 ** Reg. Linux User #211409 ** a11y rocks! Felix Miata *** http://fm.no-ip.com/
