Re: [CentOS] Troubles expanding file system. - Solved
To follow-up and provide a conclusion to my issue, in case anyone else runs into a similar situation. TLDR - go to bottom and read item #7. To recap the issue: I have a Dell PowerEdge server with a CentOS KVM host (Earth) with one CentOS guest (Sequoia) that I am trying to expand the partition and filesystem on. I have LVM logical volumes on the host system (Earth), which are used as devices/partitions on the guest system (Sequoia). In this particular situation I have successfully extended the logical volume (lv_SeqEco) on Earth from 500GB to 700GB. 1. Checking the disk information (lsblk) on Earth shows that the logical volume (lv_SeqEco) is now listed as 700GB. 2. Checking disk information (lsblk) on Sequoia shows that the disk /dev/vde is still listed as 500GB, and partition /dev/vde1 where the mount point /ecosystem is located is also listed as 500GB. 3. I had tried using the resize2fs command to expand the filesystem on /dev/vde1, but it returned with the result that there was nothing to do. Which makes sense now after I checked the disk information, since /dev/vde on Sequoia has not increased from 500GB to 700GB. 4. On previous occasions when I have done this task, I would just start GParted on Sequoia and use the GUI to expand the partition and filesystem. I am unable to do this now, as the VGA adapter on my server has died and I have no graphical output to the attached monitor. 5. My goal was to not have to not have to reboot the system. Especially since it is a 10-year old server and there is no longer VGA output to view the boot process. What I tried, and what worked to solve my issue: 1. First, I tried rescannning the device on the guest: echo 1 > /sys/devices/pci:00/:00:01.1/host1/target1:0:0/1:0:0:0/rescan This resulted in no output, and checking lsblk on the guest (Sequoia) showed no change: vde 500GB disk vde1 500GB part /ecosystem 2. Second, I tried using "virsh blockresize" from the host (Earth) system. Get the block size information earth# virsh domblkinfo SequoiaVM vde output: vde capacity 751619276800 Resize the block (express the size in Bytes) earth# virsh blockresize SequioaVM vde 751619276800B output: Block device vde is resized. Check the block device on the guest to confirm. sequoia# lsblk vde 700GB disk vde1 500GB part /ecosystem Therefore, virsh blockresize was successful. 3. I now have the larger disk recognized on the guest system, but still need to expand the partition and the filesystem. Tried the following options. resize2fs -p /dev/vde1 output: filesystem is already xx blocks long, nothing to do. growpart /dev/vde 1 output: This showed output indicating the partition had changed changed: partition=1 start=63 old: size=1048575937 end=1048576000 new: size=1468003572 end=1468003635 However, checking lsblk on the guest still showed the same result vde 700GB disk vde1 500GB part /ecosystem On the off chance that growpart was successful, I ran resize2fs again and it produced the same result as above. partprobe output: failed to re-read the partition table on all attached devices (including /dev/vde1); device or resource busy. Tried using "parted" and its resize command. First, using the print command to get the parameters of /dev/vde (output: end=751617861119B, size=751617828864B). Then changing parted to select (use) /dev/vde1 and used print command to view parameters of /dev/vde1 (output: end=536870879743B, size=536870879744B). (parted)# resize 1 0 751617861119 output: The location 751617861119 is outside of the device /dev/vde1 Looked at using fdisk; however, the documentation stated that the only way to change the partition size using fdisk is by deleting it and recreating it. I didn't want to do this. I finally decided that everything indicated that I would not be able to complete this while the system was online and/or mounted, despite all my research information showing that. 4. Unmounted the filesystem of the guest and run partprobe. sequoia# service smb stop sequoia# umount /ecosystem sequoia# partprobe output: This produced the same warnings as before (failed to re-read the partition, device busy) for all partitions that were still mounted, but did not produce the warning for partition /dev/vde1 that had been unmounted. sequoia# lsblk vde 700GB disk vde1 700GB part /ecosystem Success. The new partition is now recognized in the kernel, and I should now be able to resize the filesystem. 5. Run resize2fs sequoia# resize2fs -p /dev/vde1 output: please run e2fsck -f /dev/vde1 first sequoia# e2fsck -f -C 0 /dev/vde1 (-C 0 will display the progress) output: system passed all checks. sequoia# resize2fs -p /dev/vde1 output: resizing the filesystem on /dev/vde1 to 183500446 (4k) blocks, begin Pass 1, extending inode table, filesystem is now 183500446 blocks
Re: [CentOS] Troubles expanding file system.
> On 9/2/2021 10:28 PM, Simon Matter wrote: >> There is one thing that I couldn't find a solution for no matter what I >> tried: When the root/boot disk of the guest is being resized, it's not >> possible to modify and reread the new partition table without reboot. > > I'm curious to know if this works for you. Suppose /dev/sda is the boot > disk. Determine the highest number primary partition in use on the > drive. Let's say it's 3. > > # growpart /dev/sda 3 > > In addition, you might try > > # printf "F\n" | parted ---pretend-input-tty -l > > I have these in an Ansible playbook for creating CentOS 7 and Ubuntu > Focal VMs. They require cloud-int (for growpart) and gdisk. Note: > growpart doesn't work if the highest partition is not a primary > partition, i.e., greater than 4. I think the problem in my case was that EL6 didn't support this. From growpart man page: "this requires kernel support and 'partx --update'" Maybe it works for newer systems but didn't work when I last had to do it on EL6. Simon ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
On 9/2/2021 10:28 PM, Simon Matter wrote: There is one thing that I couldn't find a solution for no matter what I tried: When the root/boot disk of the guest is being resized, it's not possible to modify and reread the new partition table without reboot. I'm curious to know if this works for you. Suppose /dev/sda is the boot disk. Determine the highest number primary partition in use on the drive. Let's say it's 3. # growpart /dev/sda 3 In addition, you might try # printf "F\n" | parted ---pretend-input-tty -l I have these in an Ansible playbook for creating CentOS 7 and Ubuntu Focal VMs. They require cloud-int (for growpart) and gdisk. Note: growpart doesn't work if the highest partition is not a primary partition, i.e., greater than 4. Jack ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
Hi Jeff, > I realized I was still on receiving the daily digest format last night, so > I have probably screwed up the threading on this now. If you cc me > directly maybe I can maintain the future threading. > > Ok, looking at Parted it looks like the resize (or resizepart) command > will be what I will need. But that doesn't appear to help recognize the > expanded disk, so I think I need something before that. That is what I > thought the echo 1 > rescan would do for me. I'm not sure what your current state is but I'd like to point out what I usually do to online resize disks on KVM hosts/guests. After expanding the disk image/logical volume, let the quest know it has changed. To do so, run like this on the host: virsh blockresize db01 /var/lib/libvirt/images/db01.var.img 7516192768B or virsh blockresize db01 vdb 7516192768B Note: I'm using bytes (B) as unit to make very sure it's correct! After doing so, you should see that disk size has changed on the guest, in syslog or with dmesg. Now you can modify partition tables or pvresize. After that, resize the fillesystems with resize2fs or xfs_growfs. That's all possible while the guest is online with one exception outlined below. There is one thing that I couldn't find a solution for no matter what I tried: When the root/boot disk of the guest is being resized, it's not possible to modify and reread the new partition table without reboot. Regards, Simon ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
On Thu, Sep 2, 2021 at 7:16 PM Jeff Boyce wrote: > > > > > 6. I suspect that I need to rescan the devices on Sequoia so that it > > recognizes the increased space that has been allocated from the extended > > the logical volume. But when I did that (command below) it came back > > with a no such file or directory. > > > > echo 1 > /sys/class/block/vde1/device/rescan > > > Not sure that would do anything. > > https://lists.centos.org/mailman/listinfo/centos What are the CentOS versions of your host and guest? As far as I know, virtio disks have never supported rescan from inside the guest and so there isn't the corresponding "rescan" special file. I found this old, but interesting thread: https://forum.proxmox.com/threads/extend-rescan-virtio-disk-in-guest-vm.7989/ Can you confirm your host libvirt version and your guest kernel are ok with what is referred? In case you could try with the qemu monitor command. See also here for a reference of possible qemu monitor commands: https://qemu-project.gitlab.io/qemu/interop/qemu-qmp-ref.html And here for consideration regarding RH EL 6 host: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/virtualization_administration_guide/sub-sect-domain_commands-using_blockresize_to_change_the_size_of_a_domain_path Also here you find reference of command to issue, but somehow the syntax changes depending on components' versions: https://www.humblec.com/is-it-possible-to-do-online-resizing-of-guest-block-devices-or-without-shutdown/ HIH, Gianluca ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
I realized I was still on receiving the daily digest format last night, so I have probably screwed up the threading on this now. If you cc me directly maybe I can maintain the future threading. Ok, looking at Parted it looks like the resize (or resizepart) command will be what I will need. But that doesn't appear to help recognize the expanded disk, so I think I need something before that. That is what I thought the echo 1 > rescan would do for me. I will look more into fdisk to understand the capabilities there. I am going to take advantage of the holiday weekend in a few days to take care of this so I am trying to understand all of the options available to me before diving into the task. In response to Gordon also, I did rescan the drive as suggested and got the same results; no such file or directory. So then I did a search for the rescan file to see where it was present. Found it in a few locations, but this one looks to be the one that I would want to try. /sys/devices/pci:00/:00:01.1/host1/target1:0:0/1:0:0:0/ The rescan file was also located in just: /sys/bus/pci but don't know if that would do the job for the specific device. Thanks for everyone's input. Very helpful. More suggestions are welcome while I am still reading up on options. Jeff Date: Wed, 1 Sep 2021 13:15:37 -0400 From: Stephen John Smoogen To: CentOS mailing list Subject: Re: [CentOS] Troubles expanding file system. Message-ID: Content-Type: text/plain; charset="UTF-8" On Wed, 1 Sept 2021 at 12:42, Jeff Boyce wrote: Greetings - I have tried posting this four times now, from two different email addresses (on the 25th, 27th, 30th, and 31st) and it never appeared. I don't see it in the archives, so it appears to be getting dropped in transition for some reason. I am not getting messages from the email system saying it is undeliverable, or is bounced; I am sending as plain text, not HTML, I stripped off my signature. If this makes it through, someone please give me a clue why the others might not have. But that is not as important as the real issue that I am trying to get addressed below. Thanks for any assistance. I have a Dell PowerEdge server with a CentOS KVM host (Earth) with one CentOS guest (Sequoia) that I am trying to expand the partition and filesystem on. I have LVM logical volumes on the host system (Earth), which are used as devices/partitions on the guest system (Sequoia). In this particular situation I have successfully extended the logical volume (lv_SeqEco) on Earth from 500GB to 700GB. 1. Checking the disk information (lsblk) on Earth shows that the logical volume (lv_SeqEco) is now listed as 700GB. 2. Checking disk information (lsblk) on Sequoia shows that the disk /dev/vde is still listed as 500GB, and partition /dev/vde1 where the mount point /ecosystem is located is also listed as 500GB. 3. I had tried using the resize2fs command to expand the filesystem on /dev/vde1, but it returned with the result that there was nothing to do. Which makes sense now after I checked the disk information, since /dev/vde on Sequoia has not increased from 500GB to 700GB. Thanks for the long list of items of what you have done. In Fedora Infrastructure, we used this method to resize images in the past https://pagure.io/infra-docs/blob/main/f/docs/sysadmin-guide/sops/guestdisk.rst The guest system usually needs to have the `fdisk` , `gdisk` or `parted` commands rerun to resize the disk to its new size. 4. On previous occasions when I have done this task, I would just start GParted on Sequoia and use the GUI to expand the partition and filesystem. A real quick and simple solution. 5. The problem I have now is that the VGA adapter on my server has died and I have no graphical output to the attached monitor, nor to the iDrac console display. So I am stuck doing this entirely by the command line while logged into the system remotely. 6. I suspect that I need to rescan the devices on Sequoia so that it recognizes the increased space that has been allocated from the extended the logical volume. But when I did that (command below) it came back with a no such file or directory. echo 1 > /sys/class/block/vde1/device/rescan Not sure that would do anything. 7. This server is being retired in the next few months, but I need this additional space prior to migrating to the new system. Can someone give me some guidance on what I am missing in this sequence? Let me know if I haven't been clear enough in the explanation of my systems and objective. Thanks. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
> On 9/1/21 9:42 AM, Jeff Boyce wrote: >> 6. I suspect that I need to rescan the devices on Sequoia so that it >> recognizes the increased space that has been allocated from the >> extended the logical volume. But when I did that (command below) it >> came back with a no such file or directory. >> >> echo 1 > /sys/class/block/vde1/device/rescan > > > If you look at the content of /sys/class/block/vde and vde1, you'll see > that vde has a device subdir, and vde1 does not. You can't rescan a > partition. Rescan the *drive* > > echo 1 > /sys/class/block/vde/device/rescan And, on additional disks, one can put filesystems directly on the disk and so not have to care about useless partition tables (that's not on the root/boot disk). To add more flexibility, one can also use LVM on the device directly without having to mess with partition tables. Simon ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
On 9/1/21 9:42 AM, Jeff Boyce wrote: 6. I suspect that I need to rescan the devices on Sequoia so that it recognizes the increased space that has been allocated from the extended the logical volume. But when I did that (command below) it came back with a no such file or directory. echo 1 > /sys/class/block/vde1/device/rescan If you look at the content of /sys/class/block/vde and vde1, you'll see that vde has a device subdir, and vde1 does not. You can't rescan a partition. Rescan the *drive* echo 1 > /sys/class/block/vde/device/rescan ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Troubles expanding file system.
On Wed, 1 Sept 2021 at 12:42, Jeff Boyce wrote: > > Greetings - > > I have tried posting this four times now, from two different email > addresses (on the 25th, 27th, 30th, and 31st) and it never appeared. I > don't see it in the archives, so it appears to be getting dropped in > transition for some reason. I am not getting messages from the email > system saying it is undeliverable, or is bounced; I am sending as plain > text, not HTML, I stripped off my signature. If this makes it through, > someone please give me a clue why the others might not have. But that > is not as important as the real issue that I am trying to get addressed > below. Thanks for any assistance. > > I have a Dell PowerEdge server with a CentOS KVM host (Earth) with > one CentOS guest (Sequoia) that I am trying to expand the partition and > filesystem on. I have LVM logical volumes on the host system (Earth), > which are used as devices/partitions on the guest system (Sequoia). In > this particular situation I have successfully extended the logical > volume (lv_SeqEco) on Earth from 500GB to 700GB. > > 1. Checking the disk information (lsblk) on Earth shows that the > logical volume (lv_SeqEco) is now listed as 700GB. > > 2. Checking disk information (lsblk) on Sequoia shows that the disk > /dev/vde is still listed as 500GB, and partition /dev/vde1 where the > mount point /ecosystem is located is also listed as 500GB. > > 3. I had tried using the resize2fs command to expand the filesystem on > /dev/vde1, but it returned with the result that there was nothing to > do. Which makes sense now after I checked the disk information, since > /dev/vde on Sequoia has not increased from 500GB to 700GB. > Thanks for the long list of items of what you have done. In Fedora Infrastructure, we used this method to resize images in the past https://pagure.io/infra-docs/blob/main/f/docs/sysadmin-guide/sops/guestdisk.rst The guest system usually needs to have the `fdisk` , `gdisk` or `parted` commands rerun to resize the disk to its new size. > 4. On previous occasions when I have done this task, I would just start > GParted on Sequoia and use the GUI to expand the partition and > filesystem. A real quick and simple solution. > > 5. The problem I have now is that the VGA adapter on my server has died > and I have no graphical output to the attached monitor, nor to the iDrac > console display. So I am stuck doing this entirely by the command line > while logged into the system remotely. > > 6. I suspect that I need to rescan the devices on Sequoia so that it > recognizes the increased space that has been allocated from the extended > the logical volume. But when I did that (command below) it came back > with a no such file or directory. > > echo 1 > /sys/class/block/vde1/device/rescan > Not sure that would do anything. > 7. This server is being retired in the next few months, but I need this > additional space prior to migrating to the new system. Can someone give > me some guidance on what I am missing in this sequence? > > Let me know if I haven't been clear enough in the explanation of my > systems and objective. Thanks. > > Jeff > ___ > CentOS mailing list > CentOS@centos.org > https://lists.centos.org/mailman/listinfo/centos -- Stephen J Smoogen. I've seen things you people wouldn't believe. Flame wars in sci.astro.orion. I have seen SPAM filters overload because of Godwin's Law. All those moments will be lost in time... like posts on a BBS... time to shutdown -h now. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
[CentOS] Troubles expanding file system.
Greetings - I have tried posting this four times now, from two different email addresses (on the 25th, 27th, 30th, and 31st) and it never appeared. I don't see it in the archives, so it appears to be getting dropped in transition for some reason. I am not getting messages from the email system saying it is undeliverable, or is bounced; I am sending as plain text, not HTML, I stripped off my signature. If this makes it through, someone please give me a clue why the others might not have. But that is not as important as the real issue that I am trying to get addressed below. Thanks for any assistance. I have a Dell PowerEdge server with a CentOS KVM host (Earth) with one CentOS guest (Sequoia) that I am trying to expand the partition and filesystem on. I have LVM logical volumes on the host system (Earth), which are used as devices/partitions on the guest system (Sequoia). In this particular situation I have successfully extended the logical volume (lv_SeqEco) on Earth from 500GB to 700GB. 1. Checking the disk information (lsblk) on Earth shows that the logical volume (lv_SeqEco) is now listed as 700GB. 2. Checking disk information (lsblk) on Sequoia shows that the disk /dev/vde is still listed as 500GB, and partition /dev/vde1 where the mount point /ecosystem is located is also listed as 500GB. 3. I had tried using the resize2fs command to expand the filesystem on /dev/vde1, but it returned with the result that there was nothing to do. Which makes sense now after I checked the disk information, since /dev/vde on Sequoia has not increased from 500GB to 700GB. 4. On previous occasions when I have done this task, I would just start GParted on Sequoia and use the GUI to expand the partition and filesystem. A real quick and simple solution. 5. The problem I have now is that the VGA adapter on my server has died and I have no graphical output to the attached monitor, nor to the iDrac console display. So I am stuck doing this entirely by the command line while logged into the system remotely. 6. I suspect that I need to rescan the devices on Sequoia so that it recognizes the increased space that has been allocated from the extended the logical volume. But when I did that (command below) it came back with a no such file or directory. echo 1 > /sys/class/block/vde1/device/rescan 7. This server is being retired in the next few months, but I need this additional space prior to migrating to the new system. Can someone give me some guidance on what I am missing in this sequence? Let me know if I haven't been clear enough in the explanation of my systems and objective. Thanks. Jeff ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
