Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On Wed, May 13, 2020 at 12:22 PM Christoph Hellwig wrote: > > On Wed, May 13, 2020 at 09:07:43AM +0200, Martijn Coenen wrote: > > On Wed, May 13, 2020 at 4:30 AM Jens Axboe wrote: > > > > Looks acceptable to me, but I'm getting a failure applying it to > > > > for-5.8/drivers on this patch: > > > > > > > > Applying: loop: Refactor loop_set_status() size calculation > > > > > > > > So you'll probably want to respin on the right branch. > > > > This series depends on a separate bugfix I sent to LKML earlier - see > > https://lkml.org/lkml/2020/3/31/755 . I mentioned it in [00/10] of > > this series, but perhaps I should have just included that patch. > > > > I just verified that patch + this series still applies cleanly on your > > for-5.8/drivers tree, but if you prefer I send a v5 with that patch > > going first let me know. > > You probably want to resend with the fix includes as the first patch. > And drop the truncation check now that we figured out that we don't > actually need it. Just sent v5, thanks! Martijn
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On Wed, May 13, 2020 at 09:07:43AM +0200, Martijn Coenen wrote: > On Wed, May 13, 2020 at 4:30 AM Jens Axboe wrote: > > > Looks acceptable to me, but I'm getting a failure applying it to > > > for-5.8/drivers on this patch: > > > > > > Applying: loop: Refactor loop_set_status() size calculation > > > > > > So you'll probably want to respin on the right branch. > > This series depends on a separate bugfix I sent to LKML earlier - see > https://lkml.org/lkml/2020/3/31/755 . I mentioned it in [00/10] of > this series, but perhaps I should have just included that patch. > > I just verified that patch + this series still applies cleanly on your > for-5.8/drivers tree, but if you prefer I send a v5 with that patch > going first let me know. You probably want to resend with the fix includes as the first patch. And drop the truncation check now that we figured out that we don't actually need it.
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On Wed, May 13, 2020 at 4:30 AM Jens Axboe wrote: > > Looks acceptable to me, but I'm getting a failure applying it to > > for-5.8/drivers on this patch: > > > > Applying: loop: Refactor loop_set_status() size calculation > > > > So you'll probably want to respin on the right branch. This series depends on a separate bugfix I sent to LKML earlier - see https://lkml.org/lkml/2020/3/31/755 . I mentioned it in [00/10] of this series, but perhaps I should have just included that patch. I just verified that patch + this series still applies cleanly on your for-5.8/drivers tree, but if you prefer I send a v5 with that patch going first let me know. Thanks, Martijn > > Then you can also drop patch #1. > > -- > Jens Axboe >
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On 5/12/20 8:29 PM, Jens Axboe wrote: > On 5/12/20 12:46 AM, Martijn Coenen wrote: >> Hi Jens, >> >> What do you think of this series? > > Looks acceptable to me, but I'm getting a failure applying it to > for-5.8/drivers on this patch: > > Applying: loop: Refactor loop_set_status() size calculation > > So you'll probably want to respin on the right branch. Then you can also drop patch #1. -- Jens Axboe
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On 5/12/20 12:46 AM, Martijn Coenen wrote: > Hi Jens, > > What do you think of this series? Looks acceptable to me, but I'm getting a failure applying it to for-5.8/drivers on this patch: Applying: loop: Refactor loop_set_status() size calculation So you'll probably want to respin on the right branch. -- Jens Axboe
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
Hi Jens,
What do you think of this series?
Thanks,
Martijn
On Wed, Apr 29, 2020 at 4:03 PM Martijn Coenen wrote:
>
> This allows userspace to completely setup a loop device with a single
> ioctl, removing the in-between state where the device can be partially
> configured - eg the loop device has a backing file associated with it,
> but is reading from the wrong offset.
>
> Besides removing the intermediate state, another big benefit of this
> ioctl is that LOOP_SET_STATUS can be slow; the main reason for this
> slowness is that LOOP_SET_STATUS(64) calls blk_mq_freeze_queue() to
> freeze the associated queue; this requires waiting for RCU
> synchronization, which I've measured can take about 15-20ms on this
> device on average.
>
> In addition to doing what LOOP_SET_STATUS can do, LOOP_CONFIGURE can
> also be used to:
> - Set the correct block size immediately by setting
> loop_config.block_size (avoids LOOP_SET_BLOCK_SIZE)
> - Explicitly request direct I/O mode by setting LO_FLAGS_DIRECT_IO
> in loop_config.info.lo_flags (avoids LOOP_SET_DIRECT_IO)
> - Explicitly request read-only mode by setting LO_FLAGS_READ_ONLY
> in loop_config.info.lo_flags
>
> Here's setting up ~70 regular loop devices with an offset on an x86
> Android device, using LOOP_SET_FD and LOOP_SET_STATUS:
>
> vsoc_x86:/system/apex # time for i in `seq 30 100`;
> do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done
> 0m03.40s real 0m00.02s user 0m00.03s system
>
> Here's configuring ~70 devices in the same way, but using a modified
> losetup that uses the new LOOP_CONFIGURE ioctl:
>
> vsoc_x86:/system/apex # time for i in `seq 30 100`;
> do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done
> 0m01.94s real 0m00.01s user 0m00.01s system
>
> Signed-off-by: Martijn Coenen
> ---
> drivers/block/loop.c | 107 +++---
> include/uapi/linux/loop.h | 21
> 2 files changed, 99 insertions(+), 29 deletions(-)
>
> diff --git a/drivers/block/loop.c b/drivers/block/loop.c
> index cfbdd99fdb1a..a353ce55fd18 100644
> --- a/drivers/block/loop.c
> +++ b/drivers/block/loop.c
> @@ -241,6 +241,19 @@ loop_validate_size(loff_t size)
> return 0;
> }
>
> +/**
> + * loop_validate_block_size() - validates the passed in block size
> + * @bsize: size to validate
> + */
> +static int
> +loop_validate_block_size(unsigned short bsize)
> +{
> + if (bsize < 512 || bsize > PAGE_SIZE || !is_power_of_2(bsize))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> /**
> * loop_set_size() - sets device size and notifies userspace
> * @lo: struct loop_device to set the size for
> @@ -1063,23 +1076,24 @@ loop_set_status_from_info(struct loop_device *lo,
> return 0;
> }
>
> -static int loop_set_fd(struct loop_device *lo, fmode_t mode,
> - struct block_device *bdev, unsigned int arg)
> +static int loop_configure(struct loop_device *lo, fmode_t mode,
> + struct block_device *bdev,
> + const struct loop_config *config)
> {
> struct file *file;
> struct inode*inode;
> struct address_space *mapping;
> struct block_device *claimed_bdev = NULL;
> - int lo_flags = 0;
> int error;
> loff_t size;
> boolpartscan;
> + unsigned short bsize;
>
> /* This is safe, since we have a reference from open(). */
> __module_get(THIS_MODULE);
>
> error = -EBADF;
> - file = fget(arg);
> + file = fget(config->fd);
> if (!file)
> goto out;
>
> @@ -1088,7 +1102,7 @@ static int loop_set_fd(struct loop_device *lo, fmode_t
> mode,
> * here to avoid changing device under exclusive owner.
> */
> if (!(mode & FMODE_EXCL)) {
> - claimed_bdev = bd_start_claiming(bdev, loop_set_fd);
> + claimed_bdev = bd_start_claiming(bdev, loop_configure);
> if (IS_ERR(claimed_bdev)) {
> error = PTR_ERR(claimed_bdev);
> goto out_putf;
> @@ -1110,44 +1124,58 @@ static int loop_set_fd(struct loop_device *lo,
> fmode_t mode,
> mapping = file->f_mapping;
> inode = mapping->host;
>
> - if (!(file->f_mode & FMODE_WRITE) || !(mode & FMODE_WRITE) ||
> - !file->f_op->write_iter)
> - lo_flags |= LO_FLAGS_READ_ONLY;
> -
> size = get_loop_size(lo, file);
> error = loop_validate_size(size);
> if (error)
> goto out_unlock;
> +
> + if ((config->info.lo_flags & ~LOOP_CONFIGURE_SETTABLE_FLAGS) != 0) {
> + error = -EINVAL;
> + goto out_unlock;
> + }
> +
> + if (config->block_size) {
> + error = loop_validate_block_size(config->block_size);
> + if (error)
> +
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On Wed, May 6, 2020 at 11:44 AM Michael Kerrisk (man-pages) > > Can we have also a patch for the loop.4 manual page please? Ack, will do when the series lands. Best, Martijn > > Thanks, > > Michael > > > -- > Michael Kerrisk > Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ > Linux/UNIX System Programming Training: http://man7.org/training/
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
On Wed, May 6, 2020 at 8:09 AM Christoph Hellwig wrote: > > Thanks, this looks very nice! Thanks! And thanks for all the feedback, it has made this series a lot cleaner. > > Reviewed-by: Christoph Hellwig > > Are you also going to submit a patch to util-linux to use the new > ioctl? Yeah, I'm happy to - will do it as soon as the series lands. Best, Martijn
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
Hi Martijn, On 4/29/20 4:03 PM, Martijn Coenen wrote: > This allows userspace to completely setup a loop device with a single > ioctl, removing the in-between state where the device can be partially > configured - eg the loop device has a backing file associated with it, > but is reading from the wrong offset. > > Besides removing the intermediate state, another big benefit of this > ioctl is that LOOP_SET_STATUS can be slow; the main reason for this > slowness is that LOOP_SET_STATUS(64) calls blk_mq_freeze_queue() to > freeze the associated queue; this requires waiting for RCU > synchronization, which I've measured can take about 15-20ms on this > device on average. > > In addition to doing what LOOP_SET_STATUS can do, LOOP_CONFIGURE can > also be used to: > - Set the correct block size immediately by setting > loop_config.block_size (avoids LOOP_SET_BLOCK_SIZE) > - Explicitly request direct I/O mode by setting LO_FLAGS_DIRECT_IO > in loop_config.info.lo_flags (avoids LOOP_SET_DIRECT_IO) > - Explicitly request read-only mode by setting LO_FLAGS_READ_ONLY > in loop_config.info.lo_flags > > Here's setting up ~70 regular loop devices with an offset on an x86 > Android device, using LOOP_SET_FD and LOOP_SET_STATUS: > > vsoc_x86:/system/apex # time for i in `seq 30 100`; > do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done > 0m03.40s real 0m00.02s user 0m00.03s system > > Here's configuring ~70 devices in the same way, but using a modified > losetup that uses the new LOOP_CONFIGURE ioctl: > > vsoc_x86:/system/apex # time for i in `seq 30 100`; > do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done > 0m01.94s real 0m00.01s user 0m00.01s system > > Signed-off-by: Martijn Coenen Can we have also a patch for the loop.4 manual page please? Thanks, Michael -- Michael Kerrisk Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/ Linux/UNIX System Programming Training: http://man7.org/training/
Re: [PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
Thanks, this looks very nice! Reviewed-by: Christoph Hellwig Are you also going to submit a patch to util-linux to use the new ioctl?
[PATCH v4 10/10] loop: Add LOOP_CONFIGURE ioctl
This allows userspace to completely setup a loop device with a single
ioctl, removing the in-between state where the device can be partially
configured - eg the loop device has a backing file associated with it,
but is reading from the wrong offset.
Besides removing the intermediate state, another big benefit of this
ioctl is that LOOP_SET_STATUS can be slow; the main reason for this
slowness is that LOOP_SET_STATUS(64) calls blk_mq_freeze_queue() to
freeze the associated queue; this requires waiting for RCU
synchronization, which I've measured can take about 15-20ms on this
device on average.
In addition to doing what LOOP_SET_STATUS can do, LOOP_CONFIGURE can
also be used to:
- Set the correct block size immediately by setting
loop_config.block_size (avoids LOOP_SET_BLOCK_SIZE)
- Explicitly request direct I/O mode by setting LO_FLAGS_DIRECT_IO
in loop_config.info.lo_flags (avoids LOOP_SET_DIRECT_IO)
- Explicitly request read-only mode by setting LO_FLAGS_READ_ONLY
in loop_config.info.lo_flags
Here's setting up ~70 regular loop devices with an offset on an x86
Android device, using LOOP_SET_FD and LOOP_SET_STATUS:
vsoc_x86:/system/apex # time for i in `seq 30 100`;
do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done
0m03.40s real 0m00.02s user 0m00.03s system
Here's configuring ~70 devices in the same way, but using a modified
losetup that uses the new LOOP_CONFIGURE ioctl:
vsoc_x86:/system/apex # time for i in `seq 30 100`;
do losetup -r -o 4096 /dev/block/loop$i com.android.adbd.apex; done
0m01.94s real 0m00.01s user 0m00.01s system
Signed-off-by: Martijn Coenen
---
drivers/block/loop.c | 107 +++---
include/uapi/linux/loop.h | 21
2 files changed, 99 insertions(+), 29 deletions(-)
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index cfbdd99fdb1a..a353ce55fd18 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -241,6 +241,19 @@ loop_validate_size(loff_t size)
return 0;
}
+/**
+ * loop_validate_block_size() - validates the passed in block size
+ * @bsize: size to validate
+ */
+static int
+loop_validate_block_size(unsigned short bsize)
+{
+ if (bsize < 512 || bsize > PAGE_SIZE || !is_power_of_2(bsize))
+ return -EINVAL;
+
+ return 0;
+}
+
/**
* loop_set_size() - sets device size and notifies userspace
* @lo: struct loop_device to set the size for
@@ -1063,23 +1076,24 @@ loop_set_status_from_info(struct loop_device *lo,
return 0;
}
-static int loop_set_fd(struct loop_device *lo, fmode_t mode,
- struct block_device *bdev, unsigned int arg)
+static int loop_configure(struct loop_device *lo, fmode_t mode,
+ struct block_device *bdev,
+ const struct loop_config *config)
{
struct file *file;
struct inode*inode;
struct address_space *mapping;
struct block_device *claimed_bdev = NULL;
- int lo_flags = 0;
int error;
loff_t size;
boolpartscan;
+ unsigned short bsize;
/* This is safe, since we have a reference from open(). */
__module_get(THIS_MODULE);
error = -EBADF;
- file = fget(arg);
+ file = fget(config->fd);
if (!file)
goto out;
@@ -1088,7 +1102,7 @@ static int loop_set_fd(struct loop_device *lo, fmode_t
mode,
* here to avoid changing device under exclusive owner.
*/
if (!(mode & FMODE_EXCL)) {
- claimed_bdev = bd_start_claiming(bdev, loop_set_fd);
+ claimed_bdev = bd_start_claiming(bdev, loop_configure);
if (IS_ERR(claimed_bdev)) {
error = PTR_ERR(claimed_bdev);
goto out_putf;
@@ -1110,44 +1124,58 @@ static int loop_set_fd(struct loop_device *lo, fmode_t
mode,
mapping = file->f_mapping;
inode = mapping->host;
- if (!(file->f_mode & FMODE_WRITE) || !(mode & FMODE_WRITE) ||
- !file->f_op->write_iter)
- lo_flags |= LO_FLAGS_READ_ONLY;
-
size = get_loop_size(lo, file);
error = loop_validate_size(size);
if (error)
goto out_unlock;
+
+ if ((config->info.lo_flags & ~LOOP_CONFIGURE_SETTABLE_FLAGS) != 0) {
+ error = -EINVAL;
+ goto out_unlock;
+ }
+
+ if (config->block_size) {
+ error = loop_validate_block_size(config->block_size);
+ if (error)
+ goto out_unlock;
+ }
+
+ error = loop_set_status_from_info(lo, >info);
+ if (error)
+ goto out_unlock;
+
+ if (!(file->f_mode & FMODE_WRITE) || !(mode & FMODE_WRITE) ||
+ !file->f_op->write_iter)
+ lo->lo_flags |= LO_FLAGS_READ_ONLY;
+
error = loop_prepare_queue(lo);
if
