Great, thanks for the update.

Jason

On Fri, Apr 13, 2018 at 11:06 PM, Alex Gorbachev <a...@iss-integration.com> 
wrote:
> On Thu, Apr 12, 2018 at 9:38 AM, Alex Gorbachev <a...@iss-integration.com> 
> wrote:
>> On Thu, Apr 12, 2018 at 7:57 AM, Jason Dillaman <jdill...@redhat.com> wrote:
>>> If you run "partprobe" after you resize in your second example, is the
>>> change visible in "parted"?
>>
>> No, partprobe does not help:
>>
>> root@lumd1:~# parted /dev/nbd2 p
>> Model: Unknown (unknown)
>> Disk /dev/nbd2: 2147MB
>> Sector size (logical/physical): 512B/512B
>> Partition Table: loop
>> Disk Flags:
>>
>> Number  Start  End     Size    File system  Flags
>>  1      0.00B  2147MB  2147MB  xfs
>>
>> root@lumd1:~# partprobe
>> root@lumd1:~# parted /dev/nbd2 p
>> Model: Unknown (unknown)
>> Disk /dev/nbd2: 2147MB
>> Sector size (logical/physical): 512B/512B
>> Partition Table: loop
>> Disk Flags:
>>
>> Number  Start  End     Size    File system  Flags
>>  1      0.00B  2147MB  2147MB  xfs
>>
>>
>>
>>>
>>> On Wed, Apr 11, 2018 at 11:01 PM, Alex Gorbachev <a...@iss-integration.com> 
>>> wrote:
>>>> On Wed, Apr 11, 2018 at 2:13 PM, Jason Dillaman <jdill...@redhat.com> 
>>>> wrote:
>>>>> I've tested the patch on both 4.14.0 and 4.16.0 and it appears to
>>>>> function correctly for me. parted can see the newly added free-space
>>>>> after resizing the RBD image and our stress tests once again pass
>>>>> successfully. Do you have any additional details on the issues you are
>>>>> seeing?
>>>>
>>>> I recompiled again with 4.14-24 and tested, the resize shows up OK
>>>> when the filesystem is not mounted.  dmesg shows also the "detected
>>>> capacity change" message.  However, if I create a filesystem and mount
>>>> it, the capacity change is no longer detected.  Steps as follows:
>>>>
>>>> root@lumd1:~# rbd create -s 1024 --image-format 2 matte/n4
>>>> root@lumd1:~# rbd-nbd map matte/n4
>>>> /dev/nbd2
>>>> root@lumd1:~# mkfs.xfs /dev/nbd2
>>>> meta-data=/dev/nbd2              isize=512    agcount=4, agsize=65536 blks
>>>>          =                       sectsz=512   attr=2, projid32bit=1
>>>>          =                       crc=1        finobt=1, sparse=0
>>>> data     =                       bsize=4096   blocks=262144, imaxpct=25
>>>>          =                       sunit=0      swidth=0 blks
>>>> naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
>>>> log      =internal log           bsize=4096   blocks=2560, version=2
>>>>          =                       sectsz=512   sunit=0 blks, lazy-count=1
>>>> realtime =none                   extsz=4096   blocks=0, rtextents=0
>>>> root@lumd1:~# parted /dev/nbd2 p
>>>> Model: Unknown (unknown)
>>>> Disk /dev/nbd2: 1074MB
>>>> Sector size (logical/physical): 512B/512B
>>>> Partition Table: loop
>>>> Disk Flags:
>>>>
>>>> Number  Start  End     Size    File system  Flags
>>>>  1      0.00B  1074MB  1074MB  xfs
>>>>
>>>> root@lumd1:~# rbd resize --pool matte --image n4 --size 2048
>>>> Resizing image: 100% complete...done.
>>>> root@lumd1:~# parted /dev/nbd2 p
>>>> Model: Unknown (unknown)
>>>> Disk /dev/nbd2: 2147MB
>>>> Sector size (logical/physical): 512B/512B
>>>> Partition Table: loop
>>>> Disk Flags:
>>>>
>>>> Number  Start  End     Size    File system  Flags
>>>>  1      0.00B  2147MB  2147MB  xfs
>>>>
>>>> -- All is well so far, now let's mount the fs
>>>>
>>>> root@lumd1:~# mount /dev/nbd2 /mnt
>>>> root@lumd1:~# rbd resize --pool matte --image n4 --size 3072
>>>> Resizing image: 100% complete...done.
>>>> root@lumd1:~# parted /dev/nbd2 p
>>>> Model: Unknown (unknown)
>>>> Disk /dev/nbd2: 2147MB
>>>> Sector size (logical/physical): 512B/512B
>>>> Partition Table: loop
>>>> Disk Flags:
>>>>
>>>> Number  Start  End     Size    File system  Flags
>>>>  1      0.00B  2147MB  2147MB  xfs
>>>>
>>>> -- Now the change is not detected
>>>>
>>>>
>>>>>
>>>>> On Wed, Apr 11, 2018 at 12:06 PM, Jason Dillaman <jdill...@redhat.com> 
>>>>> wrote:
>>>>>> I'll give it a try locally and see if I can figure it out. Note that
>>>>>> this commit [1] also dropped the call to "bd_set_size" within
>>>>>> "nbd_size_update", which seems suspicious to me at initial glance.
>>>>>>
>>>>>> [1] 
>>>>>> https://github.com/torvalds/linux/commit/29eaadc0364943b6352e8994158febcb699c9f9b#diff-bc9273bcb259fef182ae607a1d06a142L180
>>>>>>
>>>>>> On Wed, Apr 11, 2018 at 11:09 AM, Alex Gorbachev 
>>>>>> <a...@iss-integration.com> wrote:
>>>>>>>> On Wed, Apr 11, 2018 at 10:27 AM, Alex Gorbachev 
>>>>>>>> <a...@iss-integration.com> wrote:
>>>>>>>>> On Wed, Apr 11, 2018 at 2:43 AM, Mykola Golub 
>>>>>>>>> <to.my.troc...@gmail.com> wrote:
>>>>>>>>>> On Tue, Apr 10, 2018 at 11:14:58PM -0400, Alex Gorbachev wrote:
>>>>>>>>>>
>>>>>>>>>>> So Josef fixed the one issue that enables e.g. lsblk and sysfs size 
>>>>>>>>>>> to
>>>>>>>>>>> reflect the correct siz on change.  However, partptobe and parted
>>>>>>>>>>> still do not detect the change, complete unmap and remap of rbd-nbd
>>>>>>>>>>> device and remount of the filesystem is required.
>>>>>>>>>>
>>>>>>>>>> Does your rbd-nbd include this fix [1], targeted for v12.2.3?
>>>>>>>>>>
>>>>>>>>>> [1] http://tracker.ceph.com/issues/22172
>>>>>>>>>
>>>>>>>>> It should, the rbd-nbd version is 12.2.4
>>>>>>>>>
>>>>>>>>> root@lumd1:~# rbd-nbd -v
>>>>>>>>> ceph version 12.2.4 (52085d5249a80c5f5121a76d6288429f35e4e77b) 
>>>>>>>>> luminous (stable)
>>>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> On Wed, Apr 11, 2018 at 10:39 AM, Jason Dillaman <jdill...@redhat.com> 
>>>>>>> wrote:
>>>>>>>> Do you have a preliminary patch that we can test against?
>>>>>>>> Jason
>>>>>>>
>>>>>>> Hi Jason,
>>>>>>>
>>>>>>> This is the one Josef did, which fixes detection of the new size via
>>>>>>> sysfs (lsblk eetc.), but parted still requires complete unmapping and
>>>>>>> remapping of the NBD device to detect the change.
>>>>>>>
>>>>>>> I built a kernel with this patch based on 4.14.24, and also tested
>>>>>>> with the latest mainline 4.16.1 - same behavior.
>>>>>>>
>>>>>>> cc: sta...@vger.kernel.org
>>>>>>> Fixes: 639812a ("nbd: don't set the device size until we're connected")
>>>>>>> Signed-off-by: Josef Bacik <jba...@fb.com>
>>>>>>> ---
>>>>>>>  drivers/block/nbd.c | 2 ++
>>>>>>>  1 file changed, 2 insertions(+)
>>>>>>>
>>>>>>> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
>>>>>>> index 86258b00a1d4..7106b98a35fb 100644
>>>>>>> --- a/drivers/block/nbd.c
>>>>>>> +++ b/drivers/block/nbd.c
>>>>>>> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
>>>>>>> loff_t blocksize,
>>>>>>>         struct nbd_config *config = nbd->config;
>>>>>>>         config->blksize = blocksize;
>>>>>>>         config->bytesize = blocksize * nr_blocks;
>>>>>>> +       if (nbd->task_recv != NULL)
>>>>>>> +               nbd_size_update(nbd);
>>>>>>>  }
>>>>>>>
>>>>>>>  static void nbd_complete_rq(struct request *req)
>>>>>>>
>
>
> Josef submitted three patches to nbd.c today, which completely resolve
> the problem with the online resize of rbd-nbd device.  For
> convenience, I am listing them below:
>
> Fixes: c6a4759ea0c9 ("nbd: add device refcounting")
> cc: sta...@vger.kernel.org
> Signed-off-by: Josef Bacik <jba...@fb.com>
> Tested-by: Alex Gorbachev <a...@iss-integration.com>
> ---
>  drivers/block/nbd.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 86258b00a1d4..e33da3e6aa20 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -174,8 +174,8 @@ static void nbd_dev_remove(struct nbd_device *nbd)
>  {
>         struct gendisk *disk = nbd->disk;
>         if (disk) {
> -               del_gendisk(disk);
>                 blk_cleanup_queue(disk->queue);
> +               del_gendisk(disk);
>                 blk_mq_free_tag_set(&nbd->tag_set);
>                 disk->private_data = NULL;
>                 put_disk(disk);
>
> cc: sta...@vger.kernel.org
> Fixes: 639812a ("nbd: don't set the device size until we're connected")
> Signed-off-by: Josef Bacik <jba...@fb.com>
> Tested-by: Alex Gorbachev <a...@iss-integration.com>
> ---
>  drivers/block/nbd.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index e33da3e6aa20..1520383b12f6 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -243,6 +243,8 @@ static void nbd_size_set(struct nbd_device *nbd,
> loff_t blocksize,
>         struct nbd_config *config = nbd->config;
>         config->blksize = blocksize;
>         config->bytesize = blocksize * nr_blocks;
> +       if (nbd->task_recv != NULL)
> +               nbd_size_update(nbd);
>  }
>
>  static void nbd_complete_rq(struct request *req)
>
> Fixes: 29eaadc ("nbd: stop using the bdev everywhere")
> cc: sta...@vger.kernel.org
> Signed-off-by: Josef Bacik <jba...@fb.com>
> Tested-by: Alex Gorbachev <a...@iss-integration.com>
> ---
>  drivers/block/nbd.c | 7 ++++++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 1520383b12f6..61dd95aa47fa 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -231,9 +231,15 @@ static void nbd_size_clear(struct nbd_device *nbd)
>  static void nbd_size_update(struct nbd_device *nbd)
>  {
>         struct nbd_config *config = nbd->config;
> +       struct block_device *bdev = bdget_disk(nbd->disk, 0);
> +
>         blk_queue_logical_block_size(nbd->disk->queue, config->blksize);
>         blk_queue_physical_block_size(nbd->disk->queue, config->blksize);
>         set_capacity(nbd->disk, config->bytesize >> 9);
> +       if (bdev) {
> +               bd_set_size(bdev, config->bytesize);
> +               bdput(bdev);
> +       }
>         kobject_uevent(&nbd_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  }
>
> @@ -1111,7 +1117,6 @@ static int nbd_start_device_ioctl(struct
> nbd_device *nbd, struct block_device *b
>         if (ret)
>                 return ret;
>
> -       bd_set_size(bdev, config->bytesize);
>         if (max_part)
>                 bdev->bd_invalidated = 1;
>         mutex_unlock(&nbd->config_lock);



-- 
Jason
_______________________________________________
ceph-users mailing list
ceph-users@lists.ceph.com
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Reply via email to