Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
Great, thanks for the update. Jason On Fri, Apr 13, 2018 at 11:06 PM, Alex Gorbachev wrote: > On Thu, Apr 12, 2018 at 9:38 AM, Alex Gorbachev > wrote: >> On Thu, Apr 12, 2018 at 7:57 AM, Jason Dillaman wrote: >>> If you run "partprobe" after you resize in your second example, is the >>> change visible in "parted"? >> >> No, partprobe does not help: >> >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End SizeFile system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> root@lumd1:~# partprobe >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End SizeFile system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> >> >>> >>> On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev >>> wrote: On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote: > I've tested the patch on both 4.14.0 and 4.16.0 and it appears to > function correctly for me. parted can see the newly added free-space > after resizing the RBD image and our stress tests once again pass > successfully. Do you have any additional details on the issues you are > seeing? I recompiled again with 4.14-24 and tested, the resize shows up OK when the filesystem is not mounted. dmesg shows also the "detected capacity change" message. However, if I create a filesystem and mount it, the capacity change is no longer detected. Steps as follows: root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4 root@lumd1:~# rbd-nbd map matte/n4 /dev/nbd2 root@lumd1:~# mkfs.xfs /dev/nbd2 meta-data=/dev/nbd2 isize=512agcount=4, agsize=65536 blks = sectsz=512 attr=2, projid32bit=1 = crc=1finobt=1, sparse=0 data = bsize=4096 blocks=262144, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal log bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 root@lumd1:~# parted /dev/nbd2 p Model: Unknown (unknown) Disk /dev/nbd2: 1074MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 1074MB 1074MB xfs root@lumd1:~# rbd resize --pool matte --image n4 --size 2048 Resizing image: 100% complete...done. root@lumd1:~# parted /dev/nbd2 p Model: Unknown (unknown) Disk /dev/nbd2: 2147MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 2147MB 2147MB xfs -- All is well so far, now let's mount the fs root@lumd1:~# mount /dev/nbd2 /mnt root@lumd1:~# rbd resize --pool matte --image n4 --size 3072 Resizing image: 100% complete...done. root@lumd1:~# parted /dev/nbd2 p Model: Unknown (unknown) Disk /dev/nbd2: 2147MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 2147MB 2147MB xfs -- Now the change is not detected > > On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman > wrote: >> I'll give it a try locally and see if I can figure it out. Note that >> this commit [1] also dropped the call to "bd_set_size" within >> "nbd_size_update", which seems suspicious to me at initial glance. >> >> [1] >> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180 >> >> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev >> wrote: On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev wrote: > On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub > wrote: >> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: >> >>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size >>> to >>> reflect the correct siz on change. However, partptobe and parted >>> still do not detect the change, complete unmap and remap of rbd-nbd >>> device and remount of the filesystem is required. >> >> Does your rbd-nbd include this fix [1], targeted for v12.2.3? >> >> [1] http://tracker.ceph.com/issues/22172 > > It should, t
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Thu, Apr 12, 2018 at 9:38 AM, Alex Gorbachev wrote: > On Thu, Apr 12, 2018 at 7:57 AM, Jason Dillaman wrote: >> If you run "partprobe" after you resize in your second example, is the >> change visible in "parted"? > > No, partprobe does not help: > > root@lumd1:~# parted /dev/nbd2 p > Model: Unknown (unknown) > Disk /dev/nbd2: 2147MB > Sector size (logical/physical): 512B/512B > Partition Table: loop > Disk Flags: > > Number Start End SizeFile system Flags > 1 0.00B 2147MB 2147MB xfs > > root@lumd1:~# partprobe > root@lumd1:~# parted /dev/nbd2 p > Model: Unknown (unknown) > Disk /dev/nbd2: 2147MB > Sector size (logical/physical): 512B/512B > Partition Table: loop > Disk Flags: > > Number Start End SizeFile system Flags > 1 0.00B 2147MB 2147MB xfs > > > >> >> On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev >> wrote: >>> On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote: I've tested the patch on both 4.14.0 and 4.16.0 and it appears to function correctly for me. parted can see the newly added free-space after resizing the RBD image and our stress tests once again pass successfully. Do you have any additional details on the issues you are seeing? >>> >>> I recompiled again with 4.14-24 and tested, the resize shows up OK >>> when the filesystem is not mounted. dmesg shows also the "detected >>> capacity change" message. However, if I create a filesystem and mount >>> it, the capacity change is no longer detected. Steps as follows: >>> >>> root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4 >>> root@lumd1:~# rbd-nbd map matte/n4 >>> /dev/nbd2 >>> root@lumd1:~# mkfs.xfs /dev/nbd2 >>> meta-data=/dev/nbd2 isize=512agcount=4, agsize=65536 blks >>> = sectsz=512 attr=2, projid32bit=1 >>> = crc=1finobt=1, sparse=0 >>> data = bsize=4096 blocks=262144, imaxpct=25 >>> = sunit=0 swidth=0 blks >>> naming =version 2 bsize=4096 ascii-ci=0 ftype=1 >>> log =internal log bsize=4096 blocks=2560, version=2 >>> = sectsz=512 sunit=0 blks, lazy-count=1 >>> realtime =none extsz=4096 blocks=0, rtextents=0 >>> root@lumd1:~# parted /dev/nbd2 p >>> Model: Unknown (unknown) >>> Disk /dev/nbd2: 1074MB >>> Sector size (logical/physical): 512B/512B >>> Partition Table: loop >>> Disk Flags: >>> >>> Number Start End SizeFile system Flags >>> 1 0.00B 1074MB 1074MB xfs >>> >>> root@lumd1:~# rbd resize --pool matte --image n4 --size 2048 >>> Resizing image: 100% complete...done. >>> root@lumd1:~# parted /dev/nbd2 p >>> Model: Unknown (unknown) >>> Disk /dev/nbd2: 2147MB >>> Sector size (logical/physical): 512B/512B >>> Partition Table: loop >>> Disk Flags: >>> >>> Number Start End SizeFile system Flags >>> 1 0.00B 2147MB 2147MB xfs >>> >>> -- All is well so far, now let's mount the fs >>> >>> root@lumd1:~# mount /dev/nbd2 /mnt >>> root@lumd1:~# rbd resize --pool matte --image n4 --size 3072 >>> Resizing image: 100% complete...done. >>> root@lumd1:~# parted /dev/nbd2 p >>> Model: Unknown (unknown) >>> Disk /dev/nbd2: 2147MB >>> Sector size (logical/physical): 512B/512B >>> Partition Table: loop >>> Disk Flags: >>> >>> Number Start End SizeFile system Flags >>> 1 0.00B 2147MB 2147MB xfs >>> >>> -- Now the change is not detected >>> >>> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman wrote: > I'll give it a try locally and see if I can figure it out. Note that > this commit [1] also dropped the call to "bd_set_size" within > "nbd_size_update", which seems suspicious to me at initial glance. > > [1] > https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180 > > On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev > wrote: >>> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev >>> wrote: On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub wrote: > On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: > >> So Josef fixed the one issue that enables e.g. lsblk and sysfs size >> to >> reflect the correct siz on change. However, partptobe and parted >> still do not detect the change, complete unmap and remap of rbd-nbd >> device and remount of the filesystem is required. > > Does your rbd-nbd include this fix [1], targeted for v12.2.3? > > [1] http://tracker.ceph.com/issues/22172 It should, the rbd-nbd version is 12.2.4 root@lumd1:~# rbd-nbd -v ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous (stable) ___ >>>
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Thu, Apr 12, 2018 at 7:57 AM, Jason Dillaman wrote: > If you run "partprobe" after you resize in your second example, is the > change visible in "parted"? No, partprobe does not help: root@lumd1:~# parted /dev/nbd2 p Model: Unknown (unknown) Disk /dev/nbd2: 2147MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 2147MB 2147MB xfs root@lumd1:~# partprobe root@lumd1:~# parted /dev/nbd2 p Model: Unknown (unknown) Disk /dev/nbd2: 2147MB Sector size (logical/physical): 512B/512B Partition Table: loop Disk Flags: Number Start End SizeFile system Flags 1 0.00B 2147MB 2147MB xfs > > On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev > wrote: >> On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote: >>> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to >>> function correctly for me. parted can see the newly added free-space >>> after resizing the RBD image and our stress tests once again pass >>> successfully. Do you have any additional details on the issues you are >>> seeing? >> >> I recompiled again with 4.14-24 and tested, the resize shows up OK >> when the filesystem is not mounted. dmesg shows also the "detected >> capacity change" message. However, if I create a filesystem and mount >> it, the capacity change is no longer detected. Steps as follows: >> >> root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4 >> root@lumd1:~# rbd-nbd map matte/n4 >> /dev/nbd2 >> root@lumd1:~# mkfs.xfs /dev/nbd2 >> meta-data=/dev/nbd2 isize=512agcount=4, agsize=65536 blks >> = sectsz=512 attr=2, projid32bit=1 >> = crc=1finobt=1, sparse=0 >> data = bsize=4096 blocks=262144, imaxpct=25 >> = sunit=0 swidth=0 blks >> naming =version 2 bsize=4096 ascii-ci=0 ftype=1 >> log =internal log bsize=4096 blocks=2560, version=2 >> = sectsz=512 sunit=0 blks, lazy-count=1 >> realtime =none extsz=4096 blocks=0, rtextents=0 >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 1074MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End SizeFile system Flags >> 1 0.00B 1074MB 1074MB xfs >> >> root@lumd1:~# rbd resize --pool matte --image n4 --size 2048 >> Resizing image: 100% complete...done. >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End SizeFile system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> -- All is well so far, now let's mount the fs >> >> root@lumd1:~# mount /dev/nbd2 /mnt >> root@lumd1:~# rbd resize --pool matte --image n4 --size 3072 >> Resizing image: 100% complete...done. >> root@lumd1:~# parted /dev/nbd2 p >> Model: Unknown (unknown) >> Disk /dev/nbd2: 2147MB >> Sector size (logical/physical): 512B/512B >> Partition Table: loop >> Disk Flags: >> >> Number Start End SizeFile system Flags >> 1 0.00B 2147MB 2147MB xfs >> >> -- Now the change is not detected >> >> >>> >>> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman >>> wrote: I'll give it a try locally and see if I can figure it out. Note that this commit [1] also dropped the call to "bd_set_size" within "nbd_size_update", which seems suspicious to me at initial glance. [1] https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180 On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev wrote: >> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev >> wrote: >>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub >>> wrote: On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: > So Josef fixed the one issue that enables e.g. lsblk and sysfs size to > reflect the correct siz on change. However, partptobe and parted > still do not detect the change, complete unmap and remap of rbd-nbd > device and remount of the filesystem is required. Does your rbd-nbd include this fix [1], targeted for v12.2.3? [1] http://tracker.ceph.com/issues/22172 >>> >>> It should, the rbd-nbd version is 12.2.4 >>> >>> root@lumd1:~# rbd-nbd -v >>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous >>> (stable) >>> ___ > > On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman > wrote: >> Do you have a preliminary patch that we can test against? >> Jason > > Hi Jason, > > This is the one Josef did, which
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
If you run "partprobe" after you resize in your second example, is the
change visible in "parted"?
On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev
wrote:
> On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote:
>> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to
>> function correctly for me. parted can see the newly added free-space
>> after resizing the RBD image and our stress tests once again pass
>> successfully. Do you have any additional details on the issues you are
>> seeing?
>
> I recompiled again with 4.14-24 and tested, the resize shows up OK
> when the filesystem is not mounted. dmesg shows also the "detected
> capacity change" message. However, if I create a filesystem and mount
> it, the capacity change is no longer detected. Steps as follows:
>
> root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4
> root@lumd1:~# rbd-nbd map matte/n4
> /dev/nbd2
> root@lumd1:~# mkfs.xfs /dev/nbd2
> meta-data=/dev/nbd2 isize=512agcount=4, agsize=65536 blks
> = sectsz=512 attr=2, projid32bit=1
> = crc=1finobt=1, sparse=0
> data = bsize=4096 blocks=262144, imaxpct=25
> = sunit=0 swidth=0 blks
> naming =version 2 bsize=4096 ascii-ci=0 ftype=1
> log =internal log bsize=4096 blocks=2560, version=2
> = sectsz=512 sunit=0 blks, lazy-count=1
> realtime =none extsz=4096 blocks=0, rtextents=0
> root@lumd1:~# parted /dev/nbd2 p
> Model: Unknown (unknown)
> Disk /dev/nbd2: 1074MB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
> Disk Flags:
>
> Number Start End SizeFile system Flags
> 1 0.00B 1074MB 1074MB xfs
>
> root@lumd1:~# rbd resize --pool matte --image n4 --size 2048
> Resizing image: 100% complete...done.
> root@lumd1:~# parted /dev/nbd2 p
> Model: Unknown (unknown)
> Disk /dev/nbd2: 2147MB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
> Disk Flags:
>
> Number Start End SizeFile system Flags
> 1 0.00B 2147MB 2147MB xfs
>
> -- All is well so far, now let's mount the fs
>
> root@lumd1:~# mount /dev/nbd2 /mnt
> root@lumd1:~# rbd resize --pool matte --image n4 --size 3072
> Resizing image: 100% complete...done.
> root@lumd1:~# parted /dev/nbd2 p
> Model: Unknown (unknown)
> Disk /dev/nbd2: 2147MB
> Sector size (logical/physical): 512B/512B
> Partition Table: loop
> Disk Flags:
>
> Number Start End SizeFile system Flags
> 1 0.00B 2147MB 2147MB xfs
>
> -- Now the change is not detected
>
>
>>
>> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman wrote:
>>> I'll give it a try locally and see if I can figure it out. Note that
>>> this commit [1] also dropped the call to "bd_set_size" within
>>> "nbd_size_update", which seems suspicious to me at initial glance.
>>>
>>> [1]
>>> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
>>>
>>> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev
>>> wrote:
> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
> wrote:
>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
>> wrote:
>>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>>>
So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
reflect the correct siz on change. However, partptobe and parted
still do not detect the change, complete unmap and remap of rbd-nbd
device and remount of the filesystem is required.
>>>
>>> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>>>
>>> [1] http://tracker.ceph.com/issues/22172
>>
>> It should, the rbd-nbd version is 12.2.4
>>
>> root@lumd1:~# rbd-nbd -v
>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
>> (stable)
>> ___
On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman
wrote:
> Do you have a preliminary patch that we can test against?
> Jason
Hi Jason,
This is the one Josef did, which fixes detection of the new size via
sysfs (lsblk eetc.), but parted still requires complete unmapping and
remapping of the NBD device to detect the change.
I built a kernel with this patch based on 4.14.24, and also tested
with the latest mainline 4.16.1 - same behavior.
cc: sta...@vger.kernel.org
Fixes: 639812a ("nbd: don't set the device size until we're connected")
Signed-off-by: Josef Bacik
---
drivers/block/nbd.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 86258b00a1d4..7106b98a35fb 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -243,6 +243,8 @@
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote:
> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to
> function correctly for me. parted can see the newly added free-space
> after resizing the RBD image and our stress tests once again pass
> successfully. Do you have any additional details on the issues you are
> seeing?
I recompiled again with 4.14-24 and tested, the resize shows up OK
when the filesystem is not mounted. dmesg shows also the "detected
capacity change" message. However, if I create a filesystem and mount
it, the capacity change is no longer detected. Steps as follows:
root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4
root@lumd1:~# rbd-nbd map matte/n4
/dev/nbd2
root@lumd1:~# mkfs.xfs /dev/nbd2
meta-data=/dev/nbd2 isize=512agcount=4, agsize=65536 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1finobt=1, sparse=0
data = bsize=4096 blocks=262144, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
root@lumd1:~# parted /dev/nbd2 p
Model: Unknown (unknown)
Disk /dev/nbd2: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End SizeFile system Flags
1 0.00B 1074MB 1074MB xfs
root@lumd1:~# rbd resize --pool matte --image n4 --size 2048
Resizing image: 100% complete...done.
root@lumd1:~# parted /dev/nbd2 p
Model: Unknown (unknown)
Disk /dev/nbd2: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End SizeFile system Flags
1 0.00B 2147MB 2147MB xfs
-- All is well so far, now let's mount the fs
root@lumd1:~# mount /dev/nbd2 /mnt
root@lumd1:~# rbd resize --pool matte --image n4 --size 3072
Resizing image: 100% complete...done.
root@lumd1:~# parted /dev/nbd2 p
Model: Unknown (unknown)
Disk /dev/nbd2: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: loop
Disk Flags:
Number Start End SizeFile system Flags
1 0.00B 2147MB 2147MB xfs
-- Now the change is not detected
>
> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman wrote:
>> I'll give it a try locally and see if I can figure it out. Note that
>> this commit [1] also dropped the call to "bd_set_size" within
>> "nbd_size_update", which seems suspicious to me at initial glance.
>>
>> [1]
>> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
>>
>> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev
>> wrote:
On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
wrote:
> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
> wrote:
>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>>
>>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
>>> reflect the correct siz on change. However, partptobe and parted
>>> still do not detect the change, complete unmap and remap of rbd-nbd
>>> device and remount of the filesystem is required.
>>
>> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>>
>> [1] http://tracker.ceph.com/issues/22172
>
> It should, the rbd-nbd version is 12.2.4
>
> root@lumd1:~# rbd-nbd -v
> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
> (stable)
> ___
>>>
>>> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman
>>> wrote:
Do you have a preliminary patch that we can test against?
Jason
>>>
>>> Hi Jason,
>>>
>>> This is the one Josef did, which fixes detection of the new size via
>>> sysfs (lsblk eetc.), but parted still requires complete unmapping and
>>> remapping of the NBD device to detect the change.
>>>
>>> I built a kernel with this patch based on 4.14.24, and also tested
>>> with the latest mainline 4.16.1 - same behavior.
>>>
>>> cc: sta...@vger.kernel.org
>>> Fixes: 639812a ("nbd: don't set the device size until we're connected")
>>> Signed-off-by: Josef Bacik
>>> ---
>>> drivers/block/nbd.c | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>>> index 86258b00a1d4..7106b98a35fb 100644
>>> --- a/drivers/block/nbd.c
>>> +++ b/drivers/block/nbd.c
>>> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
>>> loff_t blocksize,
>>> struct nbd_config *config = nbd->config;
>>> config->blksize = blocksize;
>>> config->bytesize = blocksize * nr_blocks;
>>> + if (nbd->task_recv != NULL)
>>> + nbd_size_update(nbd);
>>> }
>>>
>>> static void nb
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman wrote:
> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to
> function correctly for me. parted can see the newly added free-space
> after resizing the RBD image and our stress tests once again pass
> successfully. Do you have any additional details on the issues you are
> seeing?
Jason, I am recompiling now and will retest right after. Maybe I got
the wrong modules built - I will build NBD as in-kernel option.
>
> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman wrote:
>> I'll give it a try locally and see if I can figure it out. Note that
>> this commit [1] also dropped the call to "bd_set_size" within
>> "nbd_size_update", which seems suspicious to me at initial glance.
>>
>> [1]
>> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
>>
>> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev
>> wrote:
On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
wrote:
> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
> wrote:
>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>>
>>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
>>> reflect the correct siz on change. However, partptobe and parted
>>> still do not detect the change, complete unmap and remap of rbd-nbd
>>> device and remount of the filesystem is required.
>>
>> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>>
>> [1] http://tracker.ceph.com/issues/22172
>
> It should, the rbd-nbd version is 12.2.4
>
> root@lumd1:~# rbd-nbd -v
> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
> (stable)
> ___
>>>
>>> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman
>>> wrote:
Do you have a preliminary patch that we can test against?
Jason
>>>
>>> Hi Jason,
>>>
>>> This is the one Josef did, which fixes detection of the new size via
>>> sysfs (lsblk eetc.), but parted still requires complete unmapping and
>>> remapping of the NBD device to detect the change.
>>>
>>> I built a kernel with this patch based on 4.14.24, and also tested
>>> with the latest mainline 4.16.1 - same behavior.
>>>
>>> cc: sta...@vger.kernel.org
>>> Fixes: 639812a ("nbd: don't set the device size until we're connected")
>>> Signed-off-by: Josef Bacik
>>> ---
>>> drivers/block/nbd.c | 2 ++
>>> 1 file changed, 2 insertions(+)
>>>
>>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>>> index 86258b00a1d4..7106b98a35fb 100644
>>> --- a/drivers/block/nbd.c
>>> +++ b/drivers/block/nbd.c
>>> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
>>> loff_t blocksize,
>>> struct nbd_config *config = nbd->config;
>>> config->blksize = blocksize;
>>> config->bytesize = blocksize * nr_blocks;
>>> + if (nbd->task_recv != NULL)
>>> + nbd_size_update(nbd);
>>> }
>>>
>>> static void nbd_complete_rq(struct request *req)
>>>
>>> Regards,
>>> Alex
>>
>>
>>
>> --
>> Jason
>
>
>
> --
> Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
I've tested the patch on both 4.14.0 and 4.16.0 and it appears to
function correctly for me. parted can see the newly added free-space
after resizing the RBD image and our stress tests once again pass
successfully. Do you have any additional details on the issues you are
seeing?
On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman wrote:
> I'll give it a try locally and see if I can figure it out. Note that
> this commit [1] also dropped the call to "bd_set_size" within
> "nbd_size_update", which seems suspicious to me at initial glance.
>
> [1]
> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
>
> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev
> wrote:
>>> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
>>> wrote:
On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
wrote:
> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>
>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
>> reflect the correct siz on change. However, partptobe and parted
>> still do not detect the change, complete unmap and remap of rbd-nbd
>> device and remount of the filesystem is required.
>
> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>
> [1] http://tracker.ceph.com/issues/22172
It should, the rbd-nbd version is 12.2.4
root@lumd1:~# rbd-nbd -v
ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
(stable)
___
>>
>> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman wrote:
>>> Do you have a preliminary patch that we can test against?
>>> Jason
>>
>> Hi Jason,
>>
>> This is the one Josef did, which fixes detection of the new size via
>> sysfs (lsblk eetc.), but parted still requires complete unmapping and
>> remapping of the NBD device to detect the change.
>>
>> I built a kernel with this patch based on 4.14.24, and also tested
>> with the latest mainline 4.16.1 - same behavior.
>>
>> cc: sta...@vger.kernel.org
>> Fixes: 639812a ("nbd: don't set the device size until we're connected")
>> Signed-off-by: Josef Bacik
>> ---
>> drivers/block/nbd.c | 2 ++
>> 1 file changed, 2 insertions(+)
>>
>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>> index 86258b00a1d4..7106b98a35fb 100644
>> --- a/drivers/block/nbd.c
>> +++ b/drivers/block/nbd.c
>> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
>> loff_t blocksize,
>> struct nbd_config *config = nbd->config;
>> config->blksize = blocksize;
>> config->bytesize = blocksize * nr_blocks;
>> + if (nbd->task_recv != NULL)
>> + nbd_size_update(nbd);
>> }
>>
>> static void nbd_complete_rq(struct request *req)
>>
>> Regards,
>> Alex
>
>
>
> --
> Jason
--
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
I'll give it a try locally and see if I can figure it out. Note that
this commit [1] also dropped the call to "bd_set_size" within
"nbd_size_update", which seems suspicious to me at initial glance.
[1]
https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev
wrote:
>> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
>> wrote:
>>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
>>> wrote:
On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
> reflect the correct siz on change. However, partptobe and parted
> still do not detect the change, complete unmap and remap of rbd-nbd
> device and remount of the filesystem is required.
Does your rbd-nbd include this fix [1], targeted for v12.2.3?
[1] http://tracker.ceph.com/issues/22172
>>>
>>> It should, the rbd-nbd version is 12.2.4
>>>
>>> root@lumd1:~# rbd-nbd -v
>>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
>>> (stable)
>>> ___
>
> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman wrote:
>> Do you have a preliminary patch that we can test against?
>> Jason
>
> Hi Jason,
>
> This is the one Josef did, which fixes detection of the new size via
> sysfs (lsblk eetc.), but parted still requires complete unmapping and
> remapping of the NBD device to detect the change.
>
> I built a kernel with this patch based on 4.14.24, and also tested
> with the latest mainline 4.16.1 - same behavior.
>
> cc: sta...@vger.kernel.org
> Fixes: 639812a ("nbd: don't set the device size until we're connected")
> Signed-off-by: Josef Bacik
> ---
> drivers/block/nbd.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 86258b00a1d4..7106b98a35fb 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
> loff_t blocksize,
> struct nbd_config *config = nbd->config;
> config->blksize = blocksize;
> config->bytesize = blocksize * nr_blocks;
> + if (nbd->task_recv != NULL)
> + nbd_size_update(nbd);
> }
>
> static void nbd_complete_rq(struct request *req)
>
> Regards,
> Alex
--
Jason
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev
> wrote:
>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub
>> wrote:
>>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>>>
So Josef fixed the one issue that enables e.g. lsblk and sysfs size to
reflect the correct siz on change. However, partptobe and parted
still do not detect the change, complete unmap and remap of rbd-nbd
device and remount of the filesystem is required.
>>>
>>> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>>>
>>> [1] http://tracker.ceph.com/issues/22172
>>
>> It should, the rbd-nbd version is 12.2.4
>>
>> root@lumd1:~# rbd-nbd -v
>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous
>> (stable)
>> ___
On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman wrote:
> Do you have a preliminary patch that we can test against?
> Jason
Hi Jason,
This is the one Josef did, which fixes detection of the new size via
sysfs (lsblk eetc.), but parted still requires complete unmapping and
remapping of the NBD device to detect the change.
I built a kernel with this patch based on 4.14.24, and also tested
with the latest mainline 4.16.1 - same behavior.
cc: sta...@vger.kernel.org
Fixes: 639812a ("nbd: don't set the device size until we're connected")
Signed-off-by: Josef Bacik
---
drivers/block/nbd.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 86258b00a1d4..7106b98a35fb 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
loff_t blocksize,
struct nbd_config *config = nbd->config;
config->blksize = blocksize;
config->bytesize = blocksize * nr_blocks;
+ if (nbd->task_recv != NULL)
+ nbd_size_update(nbd);
}
static void nbd_complete_rq(struct request *req)
Regards,
Alex
___
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
Do you have a preliminary patch that we can test against? On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev wrote: > On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub wrote: >> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: >> >>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to >>> reflect the correct siz on change. However, partptobe and parted >>> still do not detect the change, complete unmap and remap of rbd-nbd >>> device and remount of the filesystem is required. >> >> Does your rbd-nbd include this fix [1], targeted for v12.2.3? >> >> [1] http://tracker.ceph.com/issues/22172 > > It should, the rbd-nbd version is 12.2.4 > > root@lumd1:~# rbd-nbd -v > ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous > (stable) > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com -- Jason ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub wrote: > On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: > >> So Josef fixed the one issue that enables e.g. lsblk and sysfs size to >> reflect the correct siz on change. However, partptobe and parted >> still do not detect the change, complete unmap and remap of rbd-nbd >> device and remount of the filesystem is required. > > Does your rbd-nbd include this fix [1], targeted for v12.2.3? > > [1] http://tracker.ceph.com/issues/22172 It should, the rbd-nbd version is 12.2.4 root@lumd1:~# rbd-nbd -v ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) luminous (stable) ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote: > So Josef fixed the one issue that enables e.g. lsblk and sysfs size to > reflect the correct siz on change. However, partptobe and parted > still do not detect the change, complete unmap and remap of rbd-nbd > device and remount of the filesystem is required. Does your rbd-nbd include this fix [1], targeted for v12.2.3? [1] http://tracker.ceph.com/issues/22172 -- Mykola Golub ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Sun, Mar 11, 2018 at 3:50 PM, Alex Gorbachev wrote: > On Sun, Mar 11, 2018 at 4:23 AM, Mykola Golub wrote: >> On Sat, Mar 10, 2018 at 08:25:15PM -0500, Alex Gorbachev wrote: >>> I am running into the problem described in >>> https://lkml.org/lkml/2018/2/19/565 and >>> https://tracker.ceph.com/issues/23137 >>> >>> I went ahead and built a custom kernel reverting the change >>> https://github.com/torvalds/linux/commit/639812a1ed9bf49ae2c026086fbf975339cd1eef >>> >>> After that a resize shows in lsblk and /sys/block/nbdX/size, but not >>> in parted for a mounted filesystem. >>> >>> Unmapping and remapping the NBD device shows the size in parted. >> >> Note 639812a is only a part of changes. The more invasive changes are >> in 29eaadc [1]. For me the most suspicious looks removing >> bd_set_size() in nbd_size_update(), but this is just a wild guess. >> >> I would recommend to contact the authors of the change. This would >> also be a gentle remainder for Josef that he promised to fix this. >> >> [1] >> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b > So Josef fixed the one issue that enables e.g. lsblk and sysfs size to reflect the correct siz on change. However, partptobe and parted still do not detect the change, complete unmap and remap of rbd-nbd device and remount of the filesystem is required. In limbo with this issue at this point. ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Sun, Mar 11, 2018 at 4:23 AM, Mykola Golub wrote: > On Sat, Mar 10, 2018 at 08:25:15PM -0500, Alex Gorbachev wrote: >> I am running into the problem described in >> https://lkml.org/lkml/2018/2/19/565 and >> https://tracker.ceph.com/issues/23137 >> >> I went ahead and built a custom kernel reverting the change >> https://github.com/torvalds/linux/commit/639812a1ed9bf49ae2c026086fbf975339cd1eef >> >> After that a resize shows in lsblk and /sys/block/nbdX/size, but not >> in parted for a mounted filesystem. >> >> Unmapping and remapping the NBD device shows the size in parted. > > Note 639812a is only a part of changes. The more invasive changes are > in 29eaadc [1]. For me the most suspicious looks removing > bd_set_size() in nbd_size_update(), but this is just a wild guess. > > I would recommend to contact the authors of the change. This would > also be a gentle remainder for Josef that he promised to fix this. > > [1] > https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b Got it, I am on it, thanks. Alex > > -- > Mykola Golub ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
Re: [ceph-users] rbd-nbd not resizing even after kernel tweaks
On Sat, Mar 10, 2018 at 08:25:15PM -0500, Alex Gorbachev wrote: > I am running into the problem described in > https://lkml.org/lkml/2018/2/19/565 and > https://tracker.ceph.com/issues/23137 > > I went ahead and built a custom kernel reverting the change > https://github.com/torvalds/linux/commit/639812a1ed9bf49ae2c026086fbf975339cd1eef > > After that a resize shows in lsblk and /sys/block/nbdX/size, but not > in parted for a mounted filesystem. > > Unmapping and remapping the NBD device shows the size in parted. Note 639812a is only a part of changes. The more invasive changes are in 29eaadc [1]. For me the most suspicious looks removing bd_set_size() in nbd_size_update(), but this is just a wild guess. I would recommend to contact the authors of the change. This would also be a gentle remainder for Josef that he promised to fix this. [1] https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b -- Mykola Golub ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
[ceph-users] rbd-nbd not resizing even after kernel tweaks
I am running into the problem described in https://lkml.org/lkml/2018/2/19/565 and https://tracker.ceph.com/issues/23137 I went ahead and built a custom kernel reverting the change https://github.com/torvalds/linux/commit/639812a1ed9bf49ae2c026086fbf975339cd1eef After that a resize shows in lsblk and /sys/block/nbdX/size, but not in parted for a mounted filesystem. Unmapping and remapping the NBD device shows the size in parted. Thank you for any help -- Alex Gorbachev Storcium ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
