Re: shrinking btrfs (was Re: Boot problems after adding 2 new disks)
Hi, I have used gparted to clear the partition sdc2 in preparation for the addition to the btrfs RAID type set up. It does not have a label such as sdc2 and is reported by gparted as "unallocated" Is that the correct description for the purpose? as my machine no longer boots. It enters an emergency stage where I can press CTRL+D to continue and then see the logs with another command, but it just tries to enter default mode and repeats. Also, I note for the first time that UEFI is mentioned in the boot selections. I have travelled, so far, without UEFI , although the M/B has UEFI setup option on the startup screen. Lost again Thanks Andrew ,On 24/2/19 4:08 pm, Craig Sanders via luv-main wrote: On Sun, Feb 24, 2019 at 03:15:19PM +1100, Andrew Greig wrote: This should be my last message on this issue (I sincerely hope so as I have probably redefined the meaning of "needy") I lost the message related to the setting up of one btrfs drive and then using the force (-f) feature to get it to add the device to the array i don't know which message you're referring to. i already re-sent one to you, but it seems that wasn't the right one. you can find all messages in this thread in the LUV archives at: https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: shrinking btrfs (was Re: Boot problems after adding 2 new disks)
That was a great help, thanks for directing me there. On 24/2/19 4:08 pm, Craig Sanders via luv-main wrote: i don't know which message you're referring to. i already re-sent one to you, but it seems that wasn't the right one. you can find all messages in this thread in the LUV archives at: https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: shrinking btrfs (was Re: Boot problems after adding 2 new disks)
On Sun, Feb 24, 2019 at 03:15:19PM +1100, Andrew Greig wrote: > This should be my last message on this issue (I sincerely hope so as I have > probably redefined the meaning of "needy") > > I lost the message related to the setting up of one btrfs drive and then > using the force (-f) feature to get it to add the device to the array i don't know which message you're referring to. i already re-sent one to you, but it seems that wasn't the right one. you can find all messages in this thread in the LUV archives at: https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: shrinking btrfs (was Re: Boot problems after adding 2 new disks)
Hi Craig, This should be my last message on this issue (I sincerely hope so as I have probably redefined the meaning of "needy") I lost the message related to the setting up of one btrfs drive and then using the force (-f) feature to get it to add the device to the array run sudo umount /dev/sdc1 prior to the force flag it was sudo umount /dev/sdb1 /dev/sdc1 then use gparted to blitz the partitions -- so only blitz /dev/sdc1 ?? but If I Recall you said the -f could be used with /dev/sdb1 live and mounted I am guessing, really, would the -f be placed here btrfs device add -f /dev/sdc1/ btrfs balance start -dconvert=raid1 -mconvert=raid1 / I am ready to jump on this if the above is correct. Many thanks Andrew On 24/2/19 11:47 am, Craig Sanders via luv-main wrote: [ you accidentally sent this Q as private mail. replying back to the luv-main list ] On Sun, Feb 24, 2019 at 08:33:25AM +1100, pushin.linux wrote: Hi Craig,I was wondering if btrfs allows "shrinking" a patition to create free space, and if swap at the end of an SSD was better than at the start of a standard SATA drive that's the kind of question that a search engine like google or duckduckgo is good for. Also Q sites like https://askubuntu.com/ or https://unix.stackexchange.com/ It's been years since I used btrfs for anything real (i use ZFS), so I searched for "shrink btrfs partition" and found that it is possible. But first you need to know that resizing ANY filesystem always involves two steps: resizing the fs itself, and resizing the partition that it's on. and the order of those two steps depends on whether you are shrinking or enlarging the fs. to shrink an fs, you first shrink the fs itself and then the partition. to enlarge, you first enlarge the partition and then the fs. For single-disk btrfs like on your root fs, it's fairly easy, just boot with the "gparted live" CD/USB[1] and tell it to resize your root btrfs partition (sda2, i think). That will resize both the fs and the partition. For a btrfs pool with multiple partitions/disks, it's more complicated because gparted operates on individual drives so it doesn't resize all of the drives/partitions in a btrfs fs at once. You have to resize each partition in the btrfs pool separately. e.g. if you wanted to resize your /data filesystem, you'd first have to run "gparted /dev/sdb", resize sdb2, and then "gparted /dev/sdc" and be careful to change sdc2 to EXACTLY the same size as sdb1. Personally, for a relatively trivial 4 or 8GB of swap space, i don't think it's worth the bother or the risk - messing with partitions is always a risk, it is very easy to make a mistake and that leads to data loss. swap *IS* faster on an SSD (everything is faster on an SSD), but when you get 24GB RAM installed your system isn't going to be swapping much - it certainly won't be thrashing stuff in and out of swap and causing performance problems. swap usage will be occasional data + code that hasn't been in use for a while. [1] https://gparted.org/ the gparted web site also has lots of useful info about partitioning and filesystems, so is a good place to learn the whys and wherefores of all this stuff. craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
shrinking btrfs (was Re: Boot problems after adding 2 new disks)
[ you accidentally sent this Q as private mail. replying back to the luv-main list ] On Sun, Feb 24, 2019 at 08:33:25AM +1100, pushin.linux wrote: > Hi Craig,I was wondering if btrfs allows "shrinking" a patition to create > free space, and if swap at the end of an SSD was better than at the start of > a standard SATA drive that's the kind of question that a search engine like google or duckduckgo is good for. Also Q sites like https://askubuntu.com/ or https://unix.stackexchange.com/ It's been years since I used btrfs for anything real (i use ZFS), so I searched for "shrink btrfs partition" and found that it is possible. But first you need to know that resizing ANY filesystem always involves two steps: resizing the fs itself, and resizing the partition that it's on. and the order of those two steps depends on whether you are shrinking or enlarging the fs. to shrink an fs, you first shrink the fs itself and then the partition. to enlarge, you first enlarge the partition and then the fs. For single-disk btrfs like on your root fs, it's fairly easy, just boot with the "gparted live" CD/USB[1] and tell it to resize your root btrfs partition (sda2, i think). That will resize both the fs and the partition. For a btrfs pool with multiple partitions/disks, it's more complicated because gparted operates on individual drives so it doesn't resize all of the drives/partitions in a btrfs fs at once. You have to resize each partition in the btrfs pool separately. e.g. if you wanted to resize your /data filesystem, you'd first have to run "gparted /dev/sdb", resize sdb2, and then "gparted /dev/sdc" and be careful to change sdc2 to EXACTLY the same size as sdb1. Personally, for a relatively trivial 4 or 8GB of swap space, i don't think it's worth the bother or the risk - messing with partitions is always a risk, it is very easy to make a mistake and that leads to data loss. swap *IS* faster on an SSD (everything is faster on an SSD), but when you get 24GB RAM installed your system isn't going to be swapping much - it certainly won't be thrashing stuff in and out of swap and causing performance problems. swap usage will be occasional data + code that hasn't been in use for a while. [1] https://gparted.org/ the gparted web site also has lots of useful info about partitioning and filesystems, so is a good place to learn the whys and wherefores of all this stuff. craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
Re: Boot problems after adding 2 new disks
On Sat, Feb 23, 2019 at 06:32:43PM +1100, Andrew Greig wrote:
> > 1. unmount both of them
>
> $sudo umount /dev/sdb1 && umount /dev/sdc1 ?
or "sudo umount /data0 /data1"
as long as no process has any file open under those directories (and that
includes having a shell with it's current working directory in either of
them - you can't unmount a filesystem that is being actively used), both
directories will be unmounted.
> > 2. remount ONE of them (say, data0) as /data (and edit /etc/fstab so that it
> > gets mounted as /data on every reboot. also delete the line in fstab that
> > mounts data1).
>
> Here is my current fstab (please note, partition manager took me an hour and
> a half to negotiate and I was unable to install swap on my SSD so I put a
> swap partition on each of the two SATA drives so that they would be exactly
> the same size. )
That's a shame because swap on SSD is much faster than swap on HDD. Of course
when you upgrade your RAM, it probably won't swap much.
Once you get your RAM upgrade installed, I strongly recommend that you install
libvirt and virt-manager and create some VMs to play with. e.g. make a VM and
give it three 5GB disk image files (i.e. similar to your current system with
three drives).
Then install ubuntu onto it. you can mess around with the partition manager
(or even fdisk on the command line) until you understand how it works without
risking anything on your real system. and try different variations on the
build (e.g. install ubuntu onto one of the VM's virtual disks, boot it up,
and then manually partition the other two virtual disks and aformat them with
btrfs and add them to fstab. and experiment also with other filesystems and/or
mdadm and/or lvm2 if you like).
That's one of the things VMs are good for, to experiment and test things and
especially to learn. In fact, they're an excellent way to learn stuff. Things
like partition management and formatting partitions are hard and a bit scary
because they are things that are very rarely done by most people - only when
building a new machine or adding new drives to a machine. Practice is the only
thing that will make it familiar and comfortable. Do this every few months
to keep the memory fresh so that you will know what to do and how to do it
if/when you ever need to.
> # /data0 was on /dev/sdb2 during installation
> UUID=0e8718c8-03bf-4f1a-915f-df03fe117dc0 /data0 btrfs defaults 0
>2
edit this line, change data0 to data.
> # /data1 was on /dev/sdc2 during installation
> UUID=5969127b-f5e0-40dc-98ba-ea7252c9ee41 /data1 btrfs defaults 0
>2
delete or comment out this line.
then, save & exit, and run "sudo mount /data"
> # /efi was on /dev/sda1 during installation
> UUID=b588608e-8cf7-43be-8a53-03dfde6f8f15 /efibtrfs defaults 0
>2
the EFI partition should be FAT32. UEFI can't use btrfs. I guess that means
it's not being used at all - your machine is either old-fashioned BIOS or, if
UEFI, it's configured for legacy (BIOS) boot.
> > 3. destroy the partition table on the data1 drive, and recreate it (again,
> > one big partition for the entire disk[1])
>
> So by deleting the partition we eliminate the FS (btrfs) and in the addition
> step the FS is rebuilt?? but specifically to control both disks?
No, it's just deleting and re-creating the partition. creating a partition and
formatting it are two different things. A partition is just a chunk of disk
space reserved for some particular use. That use can be to be formatted as
one of several different filesystems (ext4, xfs, btrfs, fat32, etc etc), to be
used as swap space, for an lvm physical volume (PV), or just left unused.
But now that i know you've got a swap partition on there, DON'T DELETE THE
ENTIRE PARTITION TABLE. Just delete /dev/sdc2. better yet, don't bother
deleting it at all, this step can be skipped.
You can actually skip step 3 entirely: the '-f' option used in step 4 ('btrfs
device add -f ...') should force it to use /dev/sdc2 even though it is already
formatted as btrfs.
> Can /dev/sdc2 can be deleted with gparted?
yes.
> > 4. add that drive to the existing btrfs array on /data
> >
> > e.g. *IF* /data1 was sdc1, you'd do something like:
> >
> > sudo btrfs device add -f /dev/sdc1 /data
> > sudo btrfs balance start -dconvert=raid1 -mconvert=raid1 /data
change sdc1 here to sdc2.
> > The earlier you do this (i.e. the less data is already on it), the faster
> > this conversion to raid1 will be. Nearly instant if there's little or no
> > data. Much longer if there's a lot of data that needs to be synced to the
> > other drive.
> >
> > i.e. best to do it before copying the data from your old drive.
>
> I have about 4Gb only of data from this morning's photo shoot, I can move
> that back to /home/andrew easily enough. I just tried the Data drive to see
> how my CHOWN went. ( I cheat, I use mc)
No need. 4GB of data will be synced in very little time.
craig
--
Re: Boot problems after adding 2 new disks
Very happy vegemite atm, I disconnected my optical drive so I could hook up my old SATA HDD. Well, it was found by the system and automounted. I was getting ready for a mount operation. No Need. I am loading my SSD at present and then I will put a bit of data in the /Data directory to see how the balancing deal goes. When that is OK I will just dump the rest of the data. Thank you very much. Truly grateful Andrew On 23/2/19 5:31 pm, Craig Sanders via luv-main wrote: On Sat, Feb 23, 2019 at 04:26:25PM +1100, Andrew Greig wrote: Referring to an earlier message about my data drives, do I need to CHOWN those drives to andrew:andrew and then set the permissions to rwx? I think i said perms should be 664. that was wrong. the execute bit is needed to access a directory, so it should be 775 (rwxrwxr-x). 770 (rwxrwx---) would also work if you didn't want any other accounts on the system (other than root and andrew, and any accounts that you add to group andrew) to access it. the chown and chmod commands need to be run so that your user is able to read and write to the /data directory. Otherwise it'll be owned by root and only writable by root. NOTE: the chown and chmod need to be done while /data is mounted. This only needs to be done once, and will retain the owner & permissions metadata whenever it is remounted (e.g. on a reboot). if you do the chown & chmod while the /data fs isn't mounted, you'll only be changing the permissions of the empty mount-point directory, not of the filesystem. I think you mentioned a symlink, would that be necessary if I have done the CHOWN? the symlink was for convenience only. useful but not necessary. mostly so that you can just navigate to your home dir and double-click on the symlink in any GUI file chooser dialog. or from the command line "cd ~/data". How do I set up the RAID1 on the Data0 and Data1 drives, please? see my previous message. you should have only a /data fs combining both the 2TB drives into a single btrfs raid1 array. I have btrfs on all drives. I am amazed at the speed of an SSD. Yeah, they're bloody fast, aren't they? and NVME SSDs are even faster. I will pick up the RAM and a cradle for the SSD as it does not fit anywhere in my case. It is just sitting in there at present. There are no moving parts in an SSD, so it's safe to leave it just hanging loose indefinitely until you get a cradle for it. I wouldn't do that for a HDD except in some of data-recovery emergency, but it's not a problem for an SSD. craig -- craig sanders ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main ___ luv-main mailing list luv-main@luv.asn.au https://lists.luv.asn.au/cgi-bin/mailman/listinfo/luv-main
