Re: Move data and mount point to subvolume
On 09/18/2018 08:10 PM, Marc Joliet wrote: > Am Sonntag, 16. September 2018, 14:50:04 CEST schrieb Hans van Kranenburg: >> The last example, where you make a subvolume and move everything into >> it, will not do what you want. Since a subvolume is a separate new >> directoty/file hierarchy, mv will turn into a cp and rm operation >> (without warning you) probably destroying information about data shared >> between files. > > I thought that wasn't true anymore. The NEWS file to coreutils contains this > (for version 8.24): > > mv will try a reflink before falling back to a standard copy, which is > more efficient when moving files across BTRFS subvolume boundaries. I was wrong when saying that mv will copy/rm between subvolumes indeed, because you can reflink files between subvolumes, as long as they're under the same mount point. (You still can NOT between mount points iirc, even when they are mounts from the same btrfs.) But still, mv silently does one or the other, which is also confusing, because if it starts copying/removing things while that was not your intention, at first you're like "hm, takes longer than I tought" and then "oh wait, n", and then it's too late already. -- Hans van Kranenburg
Re: Move data and mount point to subvolume
Am Sonntag, 16. September 2018, 14:50:04 CEST schrieb Hans van Kranenburg: > The last example, where you make a subvolume and move everything into > it, will not do what you want. Since a subvolume is a separate new > directoty/file hierarchy, mv will turn into a cp and rm operation > (without warning you) probably destroying information about data shared > between files. I thought that wasn't true anymore. The NEWS file to coreutils contains this (for version 8.24): mv will try a reflink before falling back to a standard copy, which is more efficient when moving files across BTRFS subvolume boundaries. -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup signature.asc Description: This is a digitally signed message part.
Re: Move data and mount point to subvolume
On 16/09/18, Chris Murphy (li...@colorremedies.com) wrote: > On Sun, Sep 16, 2018 at 12:40 PM, Rory Campbell-Lange > wrote: > > I'm a bit confused about the difference between / and backup, which is > > at /bkp/backup. > > top level, subvolid=5, subvolid=0, subvol=/, FS_TREE are all the same > thing. This is the subvolume that's created at mkfs time, it has no > name, it can't be deleted, and at mkfs time if you do > > # btrfs sub get-default > ID 5 (FS_TREE) > > So long as you haven't changed the default subvolume, the top level > subvolume is what gets mounted, unless you use "-o subvol=" or "-o > subvolid=" mount option. > > If you do > # btrfs sub list -ta /bkp > > It might become a bit more clear what the layout is on disk. And for > an even more verbose output you can do: > > # btrfs insp dump-t -t fs_tree /dev/### for this you need to > specify device not mountpoint, you don't need to umount, it's a read > only command > > Anything that in the "top level" or the "file system root" you will > see listed. The first number is the inode, you'll see 256 is a special > inode for subvolumes. You can do 'ls -li' and compare. Any subvolume > you create is not FS_TREE, it is a "file tree". And note that each > subvolume has it's own pile of inode numbers, meaning > files/directories only have unique inode numbers *in a given > subvolume*. Those inode numbers start over in a new subvolume. > > Subvolumes share extent, chunk, csum, uuid and other trees, so a > subvolume is not a completely isolated "file system". Thanks for the wonderfully clear explanation. > > Anyhow I've verified I can snapshot /bkp/backup to another subvolume. > > This means I don't need to move anything, simply remount /bkp at > > /bkp/backup. > > Uhh, that's the reverse of what you said in the first message. I'm not > sure what you want to do. It sounds like you want to mount the > subvolume "backup" at /bkp/ so that all the other files/dirs on this > Btrfs volume are not visible through the /bkp/ mount path? > > Anyway if you want to explicitly mount the subvolume "backup" > somewhere, you use -o subvol=backup to specify "the subvolume named > backup, not the top level subvolume". Yes, apologies, my intentions changed after you pointed out /bkp/backup was a subvolume. My simple aim is to mount backup at /bkp. Thanks for the note about the mount subvolume option. > > Presumably I can therefore remount /bkp at subvolume /backup? > > > > # btrfs subvolume show /bkp/backup | egrep -i 'name|uuid|subvol' > > Name: backup > > UUID: d17cf2ca-a6db-ca43-8054-1fd76533e84b > > Parent UUID:- > > Received UUID: - > > Subvolume ID: 258 > > > > My fstab is presently > > > > UUID=da90602a-b98e-4f0b-959a-ce431ac0cdfa /bkp btrfs > > noauto,noatime,compress=lzo 0 2 > > > > I guess it would now be > > > > UUID=d17cf2ca-a6db-ca43-8054-1fd76533e84b /bkp btrfs > > noauto,noatime,compress=lzo 0 2 > > No you can't mount by subvolume UUID. You continue to specify the > volume UUID, but then add a mount option > > > noauto,noatime,compress=lzo,subvol=backup > > or > > noauto,noatime,compress=lzo,subvolid=258 > > The advantage of subvolid is that it doesn't change when you rename > the subvolume. That is terrific advice; thanks a lot. Rory
Re: Move data and mount point to subvolume
On Sun, Sep 16, 2018 at 12:40 PM, Rory Campbell-Lange wrote: > Thanks very much for spotting my error, Chris. > > # mount | grep bkp > /dev/mapper/cdisk2 on /bkp type btrfs > (rw,noatime,compress=lzo,space_cache,subvolid=5,subvol=/) > > # btrfs subvol list /bkp > ID 258 gen 313636 top level 5 path backup > > I'm a bit confused about the difference between / and backup, which is > at /bkp/backup. top level, subvolid=5, subvolid=0, subvol=/, FS_TREE are all the same thing. This is the subvolume that's created at mkfs time, it has no name, it can't be deleted, and at mkfs time if you do # btrfs sub get-default ID 5 (FS_TREE) So long as you haven't changed the default subvolume, the top level subvolume is what gets mounted, unless you use "-o subvol=" or "-o subvolid=" mount option. If you do # btrfs sub list -ta /bkp It might become a bit more clear what the layout is on disk. And for an even more verbose output you can do: # btrfs insp dump-t -t fs_tree /dev/### for this you need to specify device not mountpoint, you don't need to umount, it's a read only command Anything that in the "top level" or the "file system root" you will see listed. The first number is the inode, you'll see 256 is a special inode for subvolumes. You can do 'ls -li' and compare. Any subvolume you create is not FS_TREE, it is a "file tree". And note that each subvolume has it's own pile of inode numbers, meaning files/directories only have unique inode numbers *in a given subvolume*. Those inode numbers start over in a new subvolume. Subvolumes share extent, chunk, csum, uuid and other trees, so a subvolume is not a completely isolated "file system". > > Anyhow I've verified I can snapshot /bkp/backup to another subvolume. > This means I don't need to move anything, simply remount /bkp at > /bkp/backup. Uhh, that's the reverse of what you said in the first message. I'm not sure what you want to do. It sounds like you want to mount the subvolume "backup" at /bkp/ so that all the other files/dirs on this Btrfs volume are not visible through the /bkp/ mount path? Anyway if you want to explicitly mount the subvolume "backup" somewhere, you use -o subvol=backup to specify "the subvolume named backup, not the top level subvolume". > > Presumably I can therefore remount /bkp at subvolume /backup? > > # btrfs subvolume show /bkp/backup | egrep -i 'name|uuid|subvol' > Name: backup > UUID: d17cf2ca-a6db-ca43-8054-1fd76533e84b > Parent UUID:- > Received UUID: - > Subvolume ID: 258 > > My fstab is presently > > UUID=da90602a-b98e-4f0b-959a-ce431ac0cdfa /bkp btrfs > noauto,noatime,compress=lzo 0 2 > > I guess it would now be > > UUID=d17cf2ca-a6db-ca43-8054-1fd76533e84b /bkp btrfs > noauto,noatime,compress=lzo 0 2 No you can't mount by subvolume UUID. You continue to specify the volume UUID, but then add a mount option noauto,noatime,compress=lzo,subvol=backup or noauto,noatime,compress=lzo,subvolid=258 The advantage of subvolid is that it doesn't change when you rename the subvolume. > >> If you snapshot a subvolume, which itself contains subvolumes, the >> nested subvolumes are not snapshot. In the snapshot, the nested >> subvolumes are empty directories. >> >> > >> > # btrfs fi du -s /bkp/backup-subvol/backup >> > Total Exclusive Set shared Filename >> > ERROR: cannot check space of '/bkp/backup-subvol/backup': Inappropriate >> > ioctl for device >> >> That's a bug in older btrfs-progs. It's been fixed, but I'm not sure >> what version, maybe by 4.14? > > Sounds about right -- my version is 4.7.3. It's not dangerous to use it (maybe --repair is more dangerous but don't use it without advice first, no matter version). You just don't get new features and bug fixes. It's also not dangerous to use something much newer, again if the user space tools are very new and the kernel is old, you just don't get certain features. -- Chris Murphy
Re: Move data and mount point to subvolume
On 16/09/18, Chris Murphy (li...@colorremedies.com) wrote: > > So I did this: > > > > btrfs subvol snapshot /bkp /bkp/backup-subvol > > > > strangely while /bkp/backup has lots of files in it, > > /bkp/backup-subvol/backup has none. > > > > # btrfs subvol list /bkp > > ID 258 gen 313585 top level 5 path backup > > ID 4782 gen 313590 top level 5 path backup-subvol > > OK so previously you said "/bkp which is a top level subvolume. There > are no other subvolumes." > > But in fact backup is already a subvolume. So now it's confusing what > you were asking for in the first place, maybe you didn't realize > backup is not a dir but it is a subvolume. Thanks very much for spotting my error, Chris. # mount | grep bkp /dev/mapper/cdisk2 on /bkp type btrfs (rw,noatime,compress=lzo,space_cache,subvolid=5,subvol=/) # btrfs subvol list /bkp ID 258 gen 313636 top level 5 path backup I'm a bit confused about the difference between / and backup, which is at /bkp/backup. Anyhow I've verified I can snapshot /bkp/backup to another subvolume. This means I don't need to move anything, simply remount /bkp at /bkp/backup. Presumably I can therefore remount /bkp at subvolume /backup? # btrfs subvolume show /bkp/backup | egrep -i 'name|uuid|subvol' Name: backup UUID: d17cf2ca-a6db-ca43-8054-1fd76533e84b Parent UUID:- Received UUID: - Subvolume ID: 258 My fstab is presently UUID=da90602a-b98e-4f0b-959a-ce431ac0cdfa /bkp btrfs noauto,noatime,compress=lzo 0 2 I guess it would now be UUID=d17cf2ca-a6db-ca43-8054-1fd76533e84b /bkp btrfs noauto,noatime,compress=lzo 0 2 > If you snapshot a subvolume, which itself contains subvolumes, the > nested subvolumes are not snapshot. In the snapshot, the nested > subvolumes are empty directories. > > > > > # btrfs fi du -s /bkp/backup-subvol/backup > > Total Exclusive Set shared Filename > > ERROR: cannot check space of '/bkp/backup-subvol/backup': Inappropriate > > ioctl for device > > That's a bug in older btrfs-progs. It's been fixed, but I'm not sure > what version, maybe by 4.14? Sounds about right -- my version is 4.7.3. > > Any ideas about what could be going on? > > > > In the mean time I'm trying: > > > > btrfs subvol create /bkp/backup-subvol > > cp -prv --reflink=always /bkp/backup/* /bkp/backup-subvol/ > > Yeah that will take a lot of writes that are not necessary, now that > you see backup is a subvolume already. If you want a copy of it, just > snapshot it. Makes sense. Thanks very much Rory
Re: Move data and mount point to subvolume
> So I did this: > > btrfs subvol snapshot /bkp /bkp/backup-subvol > > strangely while /bkp/backup has lots of files in it, > /bkp/backup-subvol/backup has none. > > # btrfs subvol list /bkp > ID 258 gen 313585 top level 5 path backup > ID 4782 gen 313590 top level 5 path backup-subvol OK so previously you said "/bkp which is a top level subvolume. There are no other subvolumes." But in fact backup is already a subvolume. So now it's confusing what you were asking for in the first place, maybe you didn't realize backup is not a dir but it is a subvolume. If you snapshot a subvolume, which itself contains subvolumes, the nested subvolumes are not snapshot. In the snapshot, the nested subvolumes are empty directories. > > # btrfs fi du -s /bkp/backup-subvol/backup > Total Exclusive Set shared Filename > ERROR: cannot check space of '/bkp/backup-subvol/backup': Inappropriate > ioctl for device That's a bug in older btrfs-progs. It's been fixed, but I'm not sure what version, maybe by 4.14? > > Any ideas about what could be going on? > > In the mean time I'm trying: > > btrfs subvol create /bkp/backup-subvol > cp -prv --reflink=always /bkp/backup/* /bkp/backup-subvol/ Yeah that will take a lot of writes that are not necessary, now that you see backup is a subvolume already. If you want a copy of it, just snapshot it. -- Chris Murphy
Re: Move data and mount point to subvolume
On Sun, Sep 16, 2018 at 5:14 AM, Rory Campbell-Lange wrote: > Hi > > We have a backup machine that has been happily running its backup > partitions on btrfs (on top of a luks encrypted disks) for a few years. > > Our backup partition is on /bkp which is a top level subvolume. > Data, RAID1: total=2.52TiB, used=1.36TiB > There are no other subvolumes. and > /dev/mapper/cdisk2 on /bkp type btrfs > (rw,noatime,compress=lzo,space_cache,subvolid=5,subvol=/) I like Hans' 2nd email advice to snapshot the top level subvolume. I would start out with: btrfs sub snap -r /bkp /bkp/toplevel.ro And that way I shouldn't be able to F this up irreversibly if I make a mistake. :-D And then do another snapshot that's rw: btrfs sub snap /bkp /bkp/bkpsnap cd /bkp/bkpsnap Now remove everything except "backupdir". Then move everything out of backupdir including any hidden files. Then rmdir backupdir. Then you can rename the snapshot/subvolume cd .. mv bkpsnap backup That's less metadata writes than creating a new subvolume, and reflink copying the backup dir, e.g. cp -a --reflink /bkp/backupdir /bkp/backupsubvol That could take a long time because all the metadata is fully read, modified (new inodes) and written out. But either way it should work. -- Chris Murphy
Re: Move data and mount point to subvolume
On 16/09/18, Hans van Kranenburg (hans.van.kranenb...@mendix.com) wrote: > On 09/16/2018 02:37 PM, Hans van Kranenburg wrote: > > On 09/16/2018 01:14 PM, Rory Campbell-Lange wrote: ... > >> We have /bkp with /bkp/backup in it. We would like to mount /bkp/backup > >> at /bkp instead. Note that /bkp/backup has a lot of hardlinked files. ... > So, what you can do is (similar to what I earlier typed): > > * You now have subvol 5 (at /bkp) > * btrfs sub snap /bkp /bkp/backup-subvol > * Now you have a new subvol (256 or higher number) which already > contains everything > * Then inside /bkp/backup-subvol you will again see > /bkp/backup-subvol/backup (since you snapshotted the toplevel which also > contains it) > * Now mv /bkp/backup/backup-subvol/* /bkp/backup-subvol (so the mv > operations stays within the same subvolume) > * Then remove everything outside /bkp/backup-subvol and mv > /bkp/backup-subvol /bkp/backup, and then voila... you can now use > subvol=/backup to mount it. Thanks for the advice, Hans. Your suggestion makes complete sense. So I did this: btrfs subvol snapshot /bkp /bkp/backup-subvol strangely while /bkp/backup has lots of files in it, /bkp/backup-subvol/backup has none. # btrfs subvol list /bkp ID 258 gen 313585 top level 5 path backup ID 4782 gen 313590 top level 5 path backup-subvol # btrfs fi du -s /bkp/backup-subvol/backup Total Exclusive Set shared Filename ERROR: cannot check space of '/bkp/backup-subvol/backup': Inappropriate ioctl for device Any ideas about what could be going on? In the mean time I'm trying: btrfs subvol create /bkp/backup-subvol cp -prv --reflink=always /bkp/backup/* /bkp/backup-subvol/ One worry as I do this is that the Metadata seems to be building quickly as I run this. At the start: # btrfs fi df /bkp Data, RAID1: total=2.52TiB, used=1.37TiB System, RAID1: total=32.00MiB, used=480.00KiB Metadata, RAID1: total=9.00GiB, used=4.80GiB <--- ** GlobalReserve, single: total=512.00MiB, used=0.00B 15 minutes into the cp run: # btrfs fi df /bkp Data, RAID1: total=2.52TiB, used=1.37TiB System, RAID1: total=32.00MiB, used=480.00KiB Metadata, RAID1: total=9.00GiB, used=5.56GiB <--- ** GlobalReserve, single: total=512.00MiB, used=80.00KiB Thanks very much for any further advice Rory
Re: Move data and mount point to subvolume
Thanks very much for the reply, Hans. I'm just responding to your note about mout options here. On 16/09/18, Hans van Kranenburg (hans.van.kranenb...@mendix.com) wrote: > > The machine is btrfs-progs v4.7.3 Linux 4.9.0-8-amd64 on Debian. The > > coreutils version is 8.26-3. > > Another note: since it seems you have about 100% more data space > allocated (set apart for data purpose) than you're actually using, or in > other words, having the 1GiB chunks on average just for 50% filled... > >Data, RAID1: total=2.52TiB, used=1.36TiB > > ...in combination with using Linux 4.9, I suspect there's also 'ssd' in > your mount options (not in fstab, but enabled by btrfs while mounting, > see /proc/mounts or mount command output)? > > If so, this is a nice starting point for more info about what might also > be happening to your filesystem: > https://www.spinics.net/lists/linux-btrfs/msg70622.html My mount options are (according to 'mount'): /dev/mapper/cdisk2 on /bkp type btrfs (rw,noatime,compress=lzo,space_cache,subvolid=5,subvol=/) Perhaps I need to run a balance, as suggested in your message recorded on spinics? mount doesn't report the ssd option though, as you suggest it might. Perhaps the compression flag is the culprit here. Many thanks Rory
Re: Move data and mount point to subvolume
On 09/16/2018 02:37 PM, Hans van Kranenburg wrote: > On 09/16/2018 01:14 PM, Rory Campbell-Lange wrote: >> Hi >> >> We have a backup machine that has been happily running its backup >> partitions on btrfs (on top of a luks encrypted disks) for a few years. >> >> Our backup partition is on /bkp which is a top level subvolume. >> Data, RAID1: total=2.52TiB, used=1.36TiB >> There are no other subvolumes. >> >> We have /bkp with /bkp/backup in it. We would like to mount /bkp/backup >> at /bkp instead. Note that /bkp/backup has a lot of hardlinked files. >> >> I guess I could do >> >> cd /bkp/backup >> mv * ../ >> rmdir backup > > Doing it the other way around is easier, since you don't have to think > about hardlinked/reflinked/etc/anything while copying data: Oh wait, I'm stupid, I was reading too quickly. Let me finish my coffee first. > btrfs sub snap /bkp /bkp/backup > > Now you have the exact identical thing in /backup, and you can start > throwing away everything outside of /backup. > > To reduce chance for accidental errors, you can snapshot with -r to make > the new /backup read-only first. Then after removing everything outside > it, make it rw again with: > > btrfs property set -ts /bkp/backup ro false > >> But would it also be possible to do something like >> >> cd /bkp >> btrfs subvol create backup-subvol >> mv /bkp/backup/* /bkp/backup-subvol >> ... then mount /bkp/backup-subvol at /bkp >> >> Would this second approach work, and preserve hardlinks? You essentially want to turn a subdirectory into a subvolume. The last example, where you make a subvolume and move everything into it, will not do what you want. Since a subvolume is a separate new directoty/file hierarchy, mv will turn into a cp and rm operation (without warning you) probably destroying information about data shared between files. So, what you can do is (similar to what I earlier typed): * You now have subvol 5 (at /bkp) * btrfs sub snap /bkp /bkp/backup-subvol * Now you have a new subvol (256 or higher number) which already contains everything * Then inside /bkp/backup-subvol you will again see /bkp/backup-subvol/backup (since you snapshotted the toplevel which also contains it) * Now mv /bkp/backup/backup-subvol/* /bkp/backup-subvol (so the mv operations stays within the same subvolume) * Then remove everything outside /bkp/backup-subvol and mv /bkp/backup-subvol /bkp/backup, and then voila... you can now use subvol=/backup to mount it. >> The machine is btrfs-progs v4.7.3 Linux 4.9.0-8-amd64 on Debian. The >> coreutils version is 8.26-3. > > Another note: since it seems you have about 100% more data space > allocated (set apart for data purpose) than you're actually using, or in > other words, having the 1GiB chunks on average just for 50% filled... > >Data, RAID1: total=2.52TiB, used=1.36TiB > > ...in combination with using Linux 4.9, I suspect there's also 'ssd' in > your mount options (not in fstab, but enabled by btrfs while mounting, > see /proc/mounts or mount command output)? > > If so, this is a nice starting point for more info about what might also > be happening to your filesystem: > https://www.spinics.net/lists/linux-btrfs/msg70622.html Have fun, -- Hans van Kranenburg
Re: Move data and mount point to subvolume
On 09/16/2018 01:14 PM, Rory Campbell-Lange wrote: > Hi > > We have a backup machine that has been happily running its backup > partitions on btrfs (on top of a luks encrypted disks) for a few years. > > Our backup partition is on /bkp which is a top level subvolume. > Data, RAID1: total=2.52TiB, used=1.36TiB > There are no other subvolumes. > > We have /bkp with /bkp/backup in it. We would like to mount /bkp/backup > at /bkp instead. Note that /bkp/backup has a lot of hardlinked files. > > I guess I could do > > cd /bkp/backup > mv * ../ > rmdir backup Doing it the other way around is easier, since you don't have to think about hardlinked/reflinked/etc/anything while copying data: btrfs sub snap /bkp /bkp/backup Now you have the exact identical thing in /backup, and you can start throwing away everything outside of /backup. To reduce chance for accidental errors, you can snapshot with -r to make the new /backup read-only first. Then after removing everything outside it, make it rw again with: btrfs property set -ts /bkp/backup ro false > But would it also be possible to do something like > > cd /bkp > btrfs subvol create backup-subvol > mv /bkp/backup/* /bkp/backup-subvol > ... then mount /bkp/backup-subvol at /bkp > > Would this second approach work, and preserve hardlinks? > > The machine is btrfs-progs v4.7.3 Linux 4.9.0-8-amd64 on Debian. The > coreutils version is 8.26-3. Another note: since it seems you have about 100% more data space allocated (set apart for data purpose) than you're actually using, or in other words, having the 1GiB chunks on average just for 50% filled... Data, RAID1: total=2.52TiB, used=1.36TiB ...in combination with using Linux 4.9, I suspect there's also 'ssd' in your mount options (not in fstab, but enabled by btrfs while mounting, see /proc/mounts or mount command output)? If so, this is a nice starting point for more info about what might also be happening to your filesystem: https://www.spinics.net/lists/linux-btrfs/msg70622.html -- Hans van Kranenburg
Move data and mount point to subvolume
Hi We have a backup machine that has been happily running its backup partitions on btrfs (on top of a luks encrypted disks) for a few years. Our backup partition is on /bkp which is a top level subvolume. Data, RAID1: total=2.52TiB, used=1.36TiB There are no other subvolumes. We have /bkp with /bkp/backup in it. We would like to mount /bkp/backup at /bkp instead. Note that /bkp/backup has a lot of hardlinked files. I guess I could do cd /bkp/backup mv * ../ rmdir backup But would it also be possible to do something like cd /bkp btrfs subvol create backup-subvol mv /bkp/backup/* /bkp/backup-subvol ... then mount /bkp/backup-subvol at /bkp Would this second approach work, and preserve hardlinks? The machine is btrfs-progs v4.7.3 Linux 4.9.0-8-amd64 on Debian. The coreutils version is 8.26-3. Thanks for any comments Rory
