Re: [PATCH 0/5] nbd improvements

[Markus Pargmann]

Hi Jens,

On 2016 M09 8, Thu 14:13:46 CEST Jens Axboe wrote:
> On 09/08/2016 01:33 PM, Josef Bacik wrote:
> > This is a patch series aimed at bringing NBD into 2016.  The two big
> > components of this series is converting nbd over to using blkmq and
> > then allowing us to provide more than one connection for a nbd device.
> > The NBD user space server doesn't care about how many connections it
> > has to a particular device, so we can easily open multiple
> > connections to the server and allow blkmq to handle multi-plexing over
> > the different connections.  This drastically improves performance
> > with multi-threaded workloads over NBD.  The multi-connection support
> > requires some changes to the NBD userspace client code, and you can
> > find that code here
> > 
> > https://github.com/josefbacik/nbd
> > 
> > I have been testing this for a few months and it is pretty solid and gives
> > excellent performance improvements.  Thanks,
> 
> Not only are these nice performance improvements, it also cleans up the
> code a lot and kills crufty old driver code in favor of using the proper
> APIs instead. I have applied the series for 4.9.

These patches conflict with the patches I picked up for 4.9 that fix the 
issues of dying filesystems on killed NBD block devices.
As I am a kernel hobbiest now, I had to move the git repository recently:
https://github.com/scosu/linux-nbd

Best Regards,

Markus

signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 0/5] nbd improvements

[Markus Pargmann]

Hi Jens,

On 2016 M09 8, Thu 14:13:46 CEST Jens Axboe wrote:
> On 09/08/2016 01:33 PM, Josef Bacik wrote:
> > This is a patch series aimed at bringing NBD into 2016.  The two big
> > components of this series is converting nbd over to using blkmq and
> > then allowing us to provide more than one connection for a nbd device.
> > The NBD user space server doesn't care about how many connections it
> > has to a particular device, so we can easily open multiple
> > connections to the server and allow blkmq to handle multi-plexing over
> > the different connections.  This drastically improves performance
> > with multi-threaded workloads over NBD.  The multi-connection support
> > requires some changes to the NBD userspace client code, and you can
> > find that code here
> > 
> > https://github.com/josefbacik/nbd
> > 
> > I have been testing this for a few months and it is pretty solid and gives
> > excellent performance improvements.  Thanks,
> 
> Not only are these nice performance improvements, it also cleans up the
> code a lot and kills crufty old driver code in favor of using the proper
> APIs instead. I have applied the series for 4.9.

These patches conflict with the patches I picked up for 4.9 that fix the 
issues of dying filesystems on killed NBD block devices.
As I am a kernel hobbiest now, I had to move the git repository recently:
https://github.com/scosu/linux-nbd

Best Regards,

Markus

signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v5 3/4] make nbd device wait for its users

[Markus Pargmann]

Hi,

On Saturday 16 July 2016 16:06:36 Pranay Kr Srivastava wrote:
>   When a timeout occurs or a recv fails, then
>   instead of abruptly killing nbd block device
>   wait for its users to finish.
> 
>   This is more required when filesystem(s) like
>   ext2 or ext3 don't expect their buffer heads to
>   disappear while the filesystem is mounted.
> 
>   Each open is now refcounted with the device being released
>   for re-use only when there are no "other users".
> 
>   A timedout or a disconnected device, if in use, can't
>   be used until it has been resetted. The reset happens
>   when all tasks having this bdev open closes this bdev.
> 
> Behavioral Change:
> 
> 1)NBD_DO_IT will not wait for the device to be reset. Hence
>   the nbd-client "may exit" while some other process is using
>   this device without actually doing the reset on this device
>   , hence thus making it unusable until all such user space
>   processes have stopped using this device.
> 
> 2)There's a window where the nbd-client will not be able to
>   change / issue ioctls to the nbd device. This is when there's
>   been a disconnect issued or a timeout has occured, however
>   there are "other" user processes which currently have this
>   nbd device opened.

Thanks for the updated patches. I applied all of your patches with some
smaller changes. I will test them later and push them to the git repo
then.

Best Regards,

Markus

> 
> Signed-off-by: Pranay Kr Srivastava 
> ---
>  drivers/block/nbd.c | 37 -
>  1 file changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 4919760..fe36280 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -74,6 +74,7 @@ struct nbd_device {
>*This is specifically for calling sock_shutdown, for now.
>*/
>   struct work_struct ws_shutdown;
> + atomic_t users; /* Users that opened the block device */
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -699,6 +700,9 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
>  static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>  unsigned int cmd, unsigned long arg)
>  {
> + if (nbd->disconnect || nbd->timedout)
> + return -EBUSY;
> +
>   switch (cmd) {
>   case NBD_DISCONNECT: {
>   struct request sreq;
> @@ -728,7 +732,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
>   return 0;
>  
>   case NBD_SET_SOCK: {
> @@ -804,16 +807,12 @@ static int __nbd_ioctl(struct block_device *bdev, 
> struct nbd_device *nbd,
>   mutex_lock(>tx_lock);
>   nbd->task_recv = NULL;
>   nbd_clear_que(nbd);
> - kill_bdev(bdev);
> - nbd_bdev_reset(bdev);
>  
>   if (nbd->disconnect) /* user requested, ignore socket errors */
>   error = 0;
>   if (nbd->timedout)
>   error = -ETIMEDOUT;
>  
> - nbd_reset(nbd);
> -
>   return error;
>   }
>  
> @@ -852,10 +851,38 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd = bdev->bd_disk->private_data;
> +
> + atomic_inc(>users);
> +
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd = disk->private_data;
> + struct block_device *bdev = bdget (part_devt(
> + dev_to_part(nbd_to_dev(nbd;
> +
> + WARN_ON(!bdev);
> + if (atomic_dec_and_test(>users)) {
> + if (bdev) {
> + nbd_bdev_reset(bdev);
> + kill_bdev(bdev);
> + bdput(bdev);
> + }
> + nbd_reset(nbd);
> + }
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release,
>  };
>  
>  static void nbd_ws_func_shutdown(struct work_struct *ws_nbd)
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v5 3/4] make nbd device wait for its users

[Markus Pargmann]

Hi,

On Saturday 16 July 2016 16:06:36 Pranay Kr Srivastava wrote:
>   When a timeout occurs or a recv fails, then
>   instead of abruptly killing nbd block device
>   wait for its users to finish.
> 
>   This is more required when filesystem(s) like
>   ext2 or ext3 don't expect their buffer heads to
>   disappear while the filesystem is mounted.
> 
>   Each open is now refcounted with the device being released
>   for re-use only when there are no "other users".
> 
>   A timedout or a disconnected device, if in use, can't
>   be used until it has been resetted. The reset happens
>   when all tasks having this bdev open closes this bdev.
> 
> Behavioral Change:
> 
> 1)NBD_DO_IT will not wait for the device to be reset. Hence
>   the nbd-client "may exit" while some other process is using
>   this device without actually doing the reset on this device
>   , hence thus making it unusable until all such user space
>   processes have stopped using this device.
> 
> 2)There's a window where the nbd-client will not be able to
>   change / issue ioctls to the nbd device. This is when there's
>   been a disconnect issued or a timeout has occured, however
>   there are "other" user processes which currently have this
>   nbd device opened.

Thanks for the updated patches. I applied all of your patches with some
smaller changes. I will test them later and push them to the git repo
then.

Best Regards,

Markus

> 
> Signed-off-by: Pranay Kr Srivastava 
> ---
>  drivers/block/nbd.c | 37 -
>  1 file changed, 32 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 4919760..fe36280 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -74,6 +74,7 @@ struct nbd_device {
>*This is specifically for calling sock_shutdown, for now.
>*/
>   struct work_struct ws_shutdown;
> + atomic_t users; /* Users that opened the block device */
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -699,6 +700,9 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
>  static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>  unsigned int cmd, unsigned long arg)
>  {
> + if (nbd->disconnect || nbd->timedout)
> + return -EBUSY;
> +
>   switch (cmd) {
>   case NBD_DISCONNECT: {
>   struct request sreq;
> @@ -728,7 +732,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
>   return 0;
>  
>   case NBD_SET_SOCK: {
> @@ -804,16 +807,12 @@ static int __nbd_ioctl(struct block_device *bdev, 
> struct nbd_device *nbd,
>   mutex_lock(>tx_lock);
>   nbd->task_recv = NULL;
>   nbd_clear_que(nbd);
> - kill_bdev(bdev);
> - nbd_bdev_reset(bdev);
>  
>   if (nbd->disconnect) /* user requested, ignore socket errors */
>   error = 0;
>   if (nbd->timedout)
>   error = -ETIMEDOUT;
>  
> - nbd_reset(nbd);
> -
>   return error;
>   }
>  
> @@ -852,10 +851,38 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd = bdev->bd_disk->private_data;
> +
> + atomic_inc(>users);
> +
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd = disk->private_data;
> + struct block_device *bdev = bdget (part_devt(
> + dev_to_part(nbd_to_dev(nbd;
> +
> + WARN_ON(!bdev);
> + if (atomic_dec_and_test(>users)) {
> + if (bdev) {
> + nbd_bdev_reset(bdev);
> + kill_bdev(bdev);
> + bdput(bdev);
> + }
> + nbd_reset(nbd);
> + }
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release,
>  };
>  
>  static void nbd_ws_func_shutdown(struct work_struct *ws_nbd)
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v4 3/5]nbd: make nbd device wait for its users

[Markus Pargmann]

On Sunday 10 July 2016 21:32:07 Pranay Srivastava wrote:
> On Sun, Jul 10, 2016 at 6:32 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> > On 2016 M06 30, Thu 14:02:03 CEST Pranay Kr. Srivastava wrote:
> >> When a timeout occurs or a recv fails, then
> >> instead of abruplty killing nbd block device
> >> wait for its users to finish.
> >>
> >> This is more required when filesystem(s) like
> >> ext2 or ext3 don't expect their buffer heads to
> >> disappear while the filesystem is mounted.
> >>
> >> Each open of a nbd device is refcounted, while
> >> the userland program [nbd-client] doing the
> >> NBD_DO_IT ioctl would now wait for any other users
> >> of this device before invalidating the nbd device.
> >>
> >> A timedout or a disconnected device, if in use, can't
> >> be used until it has been resetted. The reset happens
> >> when all tasks having this bdev open closes this bdev.
> >>
> >> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> >> ---
> >>  drivers/block/nbd.c | 106
> >> ++-- 1 file changed, 87
> >> insertions(+), 19 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index e362d44..fb56dd2 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -72,6 +72,8 @@ struct nbd_device {
> >>  #endif
> >>   /* This is specifically for calling sock_shutdown, for now. */
> >>   struct work_struct ws_shutdown;
> >> + struct kref users;
> >> + struct completion user_completion;
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -99,6 +101,8 @@ static int max_part;
> >>  static DEFINE_SPINLOCK(nbd_lock);
> >>
> >>  static void nbd_ws_func_shutdown(struct work_struct *);
> >> +static void nbd_kref_release(struct kref *);
> >> +static int nbd_size_clear(struct nbd_device *, struct block_device *);
> >
> > More function signatures. Why?
> 
> To avoid code move. But do let me know why is code signature(s)
> like this are bad , just asking to avoid such things.
> 
> >
> >>
> >>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> >>  {
> >> @@ -145,11 +149,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct
> >> block_device *bdev, int blocksize, int nr_blocks)
> >>  {
> >>   int ret;
> >> -
> >>   ret = set_blocksize(bdev, blocksize);
> >>   if (ret)
> >>   return ret;
> >> -
> >
> > Unrelated.
> >
> >>   nbd->blksize = blocksize;
> >>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
> >>
> >> @@ -197,6 +199,9 @@ static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >>
> >> + if (nbd->timedout)
> >> + return;
> >> +
> >
> > What does this have to do with the patch?
> 
> to avoid re-scheduling the work function. Apparently that did
> cause some trouble with ext4 and 10K dd processes.

Ah interesting. What was the timeout in this scenario?

> 
> >
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> @@ -472,8 +477,6 @@ static int nbd_thread_recv(struct nbd_device *nbd,
> >> struct block_device *bdev) nbd_end_request(nbd, req);
> >>   }
> >>
> >> - nbd_size_clear(nbd, bdev);
> >> -
> >>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
> >>
> >>   nbd->task_recv = NULL;
> >> @@ -650,12 +653,13 @@ static int nbd_set_socket(struct nbd_device *nbd,
> >> struct socket *sock) int ret = 0;
> >>
> >>   spin_lock(>sock_lock);
> >> - if (nbd->sock)
> >> +
> >> + if (nbd->sock || nbd->timedout)
> >>   ret = -EBUSY;
> >
> > nbd->timedout is already checked in __nbd_ioctl(), no need to check it 
> > twice.
> >
> >>   else
> >>   nbd->sock = sock;
> >> - spin_unlock(>sock_lock);
> >>
> >> + spin_unlock(>sock_lock);
> >
> > random modification.
> >
> >>   return ret;
> >>  }
> >>
> >> @@ -670,6 +674,7 @@ static void nbd_reset(struct nbd_device *

Re: [PATCH v4 3/5]nbd: make nbd device wait for its users

[Markus Pargmann]

On Sunday 10 July 2016 21:32:07 Pranay Srivastava wrote:
> On Sun, Jul 10, 2016 at 6:32 PM, Markus Pargmann  wrote:
> > On 2016 M06 30, Thu 14:02:03 CEST Pranay Kr. Srivastava wrote:
> >> When a timeout occurs or a recv fails, then
> >> instead of abruplty killing nbd block device
> >> wait for its users to finish.
> >>
> >> This is more required when filesystem(s) like
> >> ext2 or ext3 don't expect their buffer heads to
> >> disappear while the filesystem is mounted.
> >>
> >> Each open of a nbd device is refcounted, while
> >> the userland program [nbd-client] doing the
> >> NBD_DO_IT ioctl would now wait for any other users
> >> of this device before invalidating the nbd device.
> >>
> >> A timedout or a disconnected device, if in use, can't
> >> be used until it has been resetted. The reset happens
> >> when all tasks having this bdev open closes this bdev.
> >>
> >> Signed-off-by: Pranay Kr. Srivastava 
> >> ---
> >>  drivers/block/nbd.c | 106
> >> ++-- 1 file changed, 87
> >> insertions(+), 19 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index e362d44..fb56dd2 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -72,6 +72,8 @@ struct nbd_device {
> >>  #endif
> >>   /* This is specifically for calling sock_shutdown, for now. */
> >>   struct work_struct ws_shutdown;
> >> + struct kref users;
> >> + struct completion user_completion;
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -99,6 +101,8 @@ static int max_part;
> >>  static DEFINE_SPINLOCK(nbd_lock);
> >>
> >>  static void nbd_ws_func_shutdown(struct work_struct *);
> >> +static void nbd_kref_release(struct kref *);
> >> +static int nbd_size_clear(struct nbd_device *, struct block_device *);
> >
> > More function signatures. Why?
> 
> To avoid code move. But do let me know why is code signature(s)
> like this are bad , just asking to avoid such things.
> 
> >
> >>
> >>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> >>  {
> >> @@ -145,11 +149,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct
> >> block_device *bdev, int blocksize, int nr_blocks)
> >>  {
> >>   int ret;
> >> -
> >>   ret = set_blocksize(bdev, blocksize);
> >>   if (ret)
> >>   return ret;
> >> -
> >
> > Unrelated.
> >
> >>   nbd->blksize = blocksize;
> >>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
> >>
> >> @@ -197,6 +199,9 @@ static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >>
> >> + if (nbd->timedout)
> >> + return;
> >> +
> >
> > What does this have to do with the patch?
> 
> to avoid re-scheduling the work function. Apparently that did
> cause some trouble with ext4 and 10K dd processes.

Ah interesting. What was the timeout in this scenario?

> 
> >
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> @@ -472,8 +477,6 @@ static int nbd_thread_recv(struct nbd_device *nbd,
> >> struct block_device *bdev) nbd_end_request(nbd, req);
> >>   }
> >>
> >> - nbd_size_clear(nbd, bdev);
> >> -
> >>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
> >>
> >>   nbd->task_recv = NULL;
> >> @@ -650,12 +653,13 @@ static int nbd_set_socket(struct nbd_device *nbd,
> >> struct socket *sock) int ret = 0;
> >>
> >>   spin_lock(>sock_lock);
> >> - if (nbd->sock)
> >> +
> >> + if (nbd->sock || nbd->timedout)
> >>   ret = -EBUSY;
> >
> > nbd->timedout is already checked in __nbd_ioctl(), no need to check it 
> > twice.
> >
> >>   else
> >>   nbd->sock = sock;
> >> - spin_unlock(>sock_lock);
> >>
> >> + spin_unlock(>sock_lock);
> >
> > random modification.
> >
> >>   return ret;
> >>  }
> >>
> >> @@ -670,6 +674,7 @@ static void nbd_reset(struct nbd_device *nbd)
> >>   nbd->flags = 0;
>

Re: [PATCH v4 2/5]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

On Sunday 10 July 2016 21:03:05 Pranay Srivastava wrote:
> On Sunday, July 10, 2016, Markus Pargmann <m...@pengutronix.de> wrote:
> > Hi,
> >
> > On 2016 M06 30, Thu 14:02:02 CEST Pranay Kr. Srivastava wrote:
> >> spinlocked ranges should be small and not contain calls into huge
> >> subfunctions. Fix my mistake and just get the pointer to the socket
> >> instead of doing everything with spinlock held.
> >>
> >> Reported-by: Mikulas Patocka <miku...@twibright.com>
> >> Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> >>
> >> Changelog:
> >> Pranay Kr. Srivastava<pran...@gmail.com>:
> >>
> >> 1) Use spin_lock instead of irq version for sock_shutdown.
> >>
> >> 2) Use system work queue to actually trigger the shutdown of
> >> socket. This solves the issue when kernel_sendmsg is currently
> >> blocked while a timeout occurs.
> >>
> >> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> >> ---
> >>  drivers/block/nbd.c | 57
> >> + 1 file changed, 36
> >> insertions(+), 21 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 766c401..e362d44 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -39,6 +39,7 @@
> >>  #include 
> >>
> >>  #include 
> >> +#include 
> >>
> >>  struct nbd_device {
> >>   u32 flags;
> >> @@ -69,6 +70,8 @@ struct nbd_device {
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >>   struct dentry *dbg_dir;
> >>  #endif
> >> + /* This is specifically for calling sock_shutdown, for now. */
> >> + struct work_struct ws_shutdown;
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -95,6 +98,8 @@ static int max_part;
> >>   */
> >>  static DEFINE_SPINLOCK(nbd_lock);
> >>
> >> +static void nbd_ws_func_shutdown(struct work_struct *);
> >> +
> >
> > are you reading all the comments I had?...
> >
> > At least respond to my comments if you disagree. I still can't see the
> benefit
> > of a function signature here if we can avoid it.
> >
> 
> That would require some code to be moved. So to avoid those
> unnecessary changes it was better to have a prototype.
> 
> It would've pissed you off more if I had tried
> to get rid of protoype.

Ah I see, thanks.

> 
> >>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> >>  {
> >>   return disk_to_dev(nbd->disk);
> >> @@ -172,39 +177,36 @@ static void nbd_end_request(struct nbd_device *nbd,
> >> struct request *req) */
> >>  static void sock_shutdown(struct nbd_device *nbd)
> >>  {
> >> - spin_lock_irq(>sock_lock);
> >> -
> >> - if (!nbd->sock) {
> >> - spin_unlock_irq(>sock_lock);
> >> - return;
> >> - }
> >> + struct socket *sock;
> >>
> >> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> - sockfd_put(nbd->sock);
> >> + spin_lock(>sock_lock);
> >> + sock = nbd->sock;
> >>   nbd->sock = NULL;
> >> - spin_unlock_irq(>sock_lock);
> >> + spin_unlock(>sock_lock);
> >> +
> >> + if (!sock)
> >> + return;
> >>
> >>   del_timer(>timeout_timer);
> >> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> >> + kernel_sock_shutdown(sock, SHUT_RDWR);
> >> + sockfd_put(sock);
> >>  }
> >>
> >>  static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >> - unsigned long flags;
> >>
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> - spin_lock_irqsave(>sock_lock, flags);
> >> -
> >>   nbd->timedout = true;
> >> -
> >> - if (nbd->sock)
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> -
> >> - spin_unlock_irqrestore(>sock_lock, flags);
> >> -
> >> + /*
> >> +  * Make sure sender thread sees nbd->timedout.
> >>

Re: [PATCH v4 2/5]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

On Sunday 10 July 2016 21:03:05 Pranay Srivastava wrote:
> On Sunday, July 10, 2016, Markus Pargmann  wrote:
> > Hi,
> >
> > On 2016 M06 30, Thu 14:02:02 CEST Pranay Kr. Srivastava wrote:
> >> spinlocked ranges should be small and not contain calls into huge
> >> subfunctions. Fix my mistake and just get the pointer to the socket
> >> instead of doing everything with spinlock held.
> >>
> >> Reported-by: Mikulas Patocka 
> >> Signed-off-by: Markus Pargmann 
> >>
> >> Changelog:
> >> Pranay Kr. Srivastava:
> >>
> >> 1) Use spin_lock instead of irq version for sock_shutdown.
> >>
> >> 2) Use system work queue to actually trigger the shutdown of
> >> socket. This solves the issue when kernel_sendmsg is currently
> >> blocked while a timeout occurs.
> >>
> >> Signed-off-by: Pranay Kr. Srivastava 
> >> ---
> >>  drivers/block/nbd.c | 57
> >> + 1 file changed, 36
> >> insertions(+), 21 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 766c401..e362d44 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -39,6 +39,7 @@
> >>  #include 
> >>
> >>  #include 
> >> +#include 
> >>
> >>  struct nbd_device {
> >>   u32 flags;
> >> @@ -69,6 +70,8 @@ struct nbd_device {
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >>   struct dentry *dbg_dir;
> >>  #endif
> >> + /* This is specifically for calling sock_shutdown, for now. */
> >> + struct work_struct ws_shutdown;
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -95,6 +98,8 @@ static int max_part;
> >>   */
> >>  static DEFINE_SPINLOCK(nbd_lock);
> >>
> >> +static void nbd_ws_func_shutdown(struct work_struct *);
> >> +
> >
> > are you reading all the comments I had?...
> >
> > At least respond to my comments if you disagree. I still can't see the
> benefit
> > of a function signature here if we can avoid it.
> >
> 
> That would require some code to be moved. So to avoid those
> unnecessary changes it was better to have a prototype.
> 
> It would've pissed you off more if I had tried
> to get rid of protoype.

Ah I see, thanks.

> 
> >>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> >>  {
> >>   return disk_to_dev(nbd->disk);
> >> @@ -172,39 +177,36 @@ static void nbd_end_request(struct nbd_device *nbd,
> >> struct request *req) */
> >>  static void sock_shutdown(struct nbd_device *nbd)
> >>  {
> >> - spin_lock_irq(>sock_lock);
> >> -
> >> - if (!nbd->sock) {
> >> - spin_unlock_irq(>sock_lock);
> >> - return;
> >> - }
> >> + struct socket *sock;
> >>
> >> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> - sockfd_put(nbd->sock);
> >> + spin_lock(>sock_lock);
> >> + sock = nbd->sock;
> >>   nbd->sock = NULL;
> >> - spin_unlock_irq(>sock_lock);
> >> + spin_unlock(>sock_lock);
> >> +
> >> + if (!sock)
> >> + return;
> >>
> >>   del_timer(>timeout_timer);
> >> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> >> + kernel_sock_shutdown(sock, SHUT_RDWR);
> >> + sockfd_put(sock);
> >>  }
> >>
> >>  static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >> - unsigned long flags;
> >>
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> - spin_lock_irqsave(>sock_lock, flags);
> >> -
> >>   nbd->timedout = true;
> >> -
> >> - if (nbd->sock)
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> -
> >> - spin_unlock_irqrestore(>sock_lock, flags);
> >> -
> >> + /*
> >> +  * Make sure sender thread sees nbd->timedout.
> >> +  */
> >> + smp_wmb();
> >> + schedule_work(>ws_shutdown);
> >> + wake_up(>waiting

Re: [PATCH v4 3/5]nbd: make nbd device wait for its users

[Markus Pargmann]

On 2016 M06 30, Thu 14:02:03 CEST Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for its users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> A timedout or a disconnected device, if in use, can't
> be used until it has been resetted. The reset happens
> when all tasks having this bdev open closes this bdev.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 106
> ++-- 1 file changed, 87
> insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index e362d44..fb56dd2 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -72,6 +72,8 @@ struct nbd_device {
>  #endif
>   /* This is specifically for calling sock_shutdown, for now. */
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
> 
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -99,6 +101,8 @@ static int max_part;
>  static DEFINE_SPINLOCK(nbd_lock);
> 
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
> +static int nbd_size_clear(struct nbd_device *, struct block_device *);

More function signatures. Why?

> 
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -145,11 +149,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct
> block_device *bdev, int blocksize, int nr_blocks)
>  {
>   int ret;
> -
>   ret = set_blocksize(bdev, blocksize);
>   if (ret)
>   return ret;
> -

Unrelated.

>   nbd->blksize = blocksize;
>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
> 
> @@ -197,6 +199,9 @@ static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> 
> + if (nbd->timedout)
> + return;
> +

What does this have to do with the patch?

>   if (list_empty(>queue_head))
>   return;
> 
> @@ -472,8 +477,6 @@ static int nbd_thread_recv(struct nbd_device *nbd,
> struct block_device *bdev) nbd_end_request(nbd, req);
>   }
> 
> - nbd_size_clear(nbd, bdev);
> -
>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
> 
>   nbd->task_recv = NULL;
> @@ -650,12 +653,13 @@ static int nbd_set_socket(struct nbd_device *nbd,
> struct socket *sock) int ret = 0;
> 
>   spin_lock(>sock_lock);
> - if (nbd->sock)
> +
> + if (nbd->sock || nbd->timedout)
>   ret = -EBUSY;

nbd->timedout is already checked in __nbd_ioctl(), no need to check it twice.

>   else
>   nbd->sock = sock;
> - spin_unlock(>sock_lock);
> 
> + spin_unlock(>sock_lock);

random modification.

>   return ret;
>  }
> 
> @@ -670,6 +674,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> + init_completion(>user_completion);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -704,6 +709,9 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
>  static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>  unsigned int cmd, unsigned long arg)
>  {
> + if (nbd->timedout || nbd->disconnect)
> + return -EBUSY;
> +
>   switch (cmd) {
>   case NBD_DISCONNECT: {
>   struct request sreq;
> @@ -733,7 +741,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct
> nbd_device *nbd, nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
>   return 0;
> 
>   case NBD_SET_SOCK: {
> @@ -752,7 +759,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct
> nbd_device *nbd,
> 
>   case NBD_SET_BLKSIZE: {
>   loff_t bsize = div_s64(nbd->bytesize, arg);
> -

random modification.

>   return nbd_size_set(nbd, bdev, arg, bsize);
>   }
> 
> @@ -804,22 +810,29 @@ static int __nbd_ioctl(struct block_device *bdev,
> struct nbd_device *nbd, error = nbd_thread_recv(nbd, bdev);
>   nbd_dev_dbg_close(nbd);
>   kthread_stop(thread);
> - sock_shutdown(nbd);
> -
> - mutex_lock(>tx_lock);
> - nbd->task_recv = NULL;
> 
> - nbd_clear_que(nbd);
> - kill_bdev(bdev);
> - nbd_bdev_reset(bdev);
> + sock_shutdown(nbd);
> 
>   if (nbd->disconnect) /* user 

Re: [PATCH v4 3/5]nbd: make nbd device wait for its users

[Markus Pargmann]

On 2016 M06 30, Thu 14:02:03 CEST Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for its users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> A timedout or a disconnected device, if in use, can't
> be used until it has been resetted. The reset happens
> when all tasks having this bdev open closes this bdev.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 106
> ++-- 1 file changed, 87
> insertions(+), 19 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index e362d44..fb56dd2 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -72,6 +72,8 @@ struct nbd_device {
>  #endif
>   /* This is specifically for calling sock_shutdown, for now. */
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
> 
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -99,6 +101,8 @@ static int max_part;
>  static DEFINE_SPINLOCK(nbd_lock);
> 
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
> +static int nbd_size_clear(struct nbd_device *, struct block_device *);

More function signatures. Why?

> 
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -145,11 +149,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct
> block_device *bdev, int blocksize, int nr_blocks)
>  {
>   int ret;
> -
>   ret = set_blocksize(bdev, blocksize);
>   if (ret)
>   return ret;
> -

Unrelated.

>   nbd->blksize = blocksize;
>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
> 
> @@ -197,6 +199,9 @@ static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> 
> + if (nbd->timedout)
> + return;
> +

What does this have to do with the patch?

>   if (list_empty(>queue_head))
>   return;
> 
> @@ -472,8 +477,6 @@ static int nbd_thread_recv(struct nbd_device *nbd,
> struct block_device *bdev) nbd_end_request(nbd, req);
>   }
> 
> - nbd_size_clear(nbd, bdev);
> -
>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
> 
>   nbd->task_recv = NULL;
> @@ -650,12 +653,13 @@ static int nbd_set_socket(struct nbd_device *nbd,
> struct socket *sock) int ret = 0;
> 
>   spin_lock(>sock_lock);
> - if (nbd->sock)
> +
> + if (nbd->sock || nbd->timedout)
>   ret = -EBUSY;

nbd->timedout is already checked in __nbd_ioctl(), no need to check it twice.

>   else
>   nbd->sock = sock;
> - spin_unlock(>sock_lock);
> 
> + spin_unlock(>sock_lock);

random modification.

>   return ret;
>  }
> 
> @@ -670,6 +674,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> + init_completion(>user_completion);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -704,6 +709,9 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
>  static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>  unsigned int cmd, unsigned long arg)
>  {
> + if (nbd->timedout || nbd->disconnect)
> + return -EBUSY;
> +
>   switch (cmd) {
>   case NBD_DISCONNECT: {
>   struct request sreq;
> @@ -733,7 +741,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct
> nbd_device *nbd, nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
>   return 0;
> 
>   case NBD_SET_SOCK: {
> @@ -752,7 +759,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct
> nbd_device *nbd,
> 
>   case NBD_SET_BLKSIZE: {
>   loff_t bsize = div_s64(nbd->bytesize, arg);
> -

random modification.

>   return nbd_size_set(nbd, bdev, arg, bsize);
>   }
> 
> @@ -804,22 +810,29 @@ static int __nbd_ioctl(struct block_device *bdev,
> struct nbd_device *nbd, error = nbd_thread_recv(nbd, bdev);
>   nbd_dev_dbg_close(nbd);
>   kthread_stop(thread);
> - sock_shutdown(nbd);
> -
> - mutex_lock(>tx_lock);
> - nbd->task_recv = NULL;
> 
> - nbd_clear_que(nbd);
> - kill_bdev(bdev);
> - nbd_bdev_reset(bdev);
> + sock_shutdown(nbd);
> 
>   if (nbd->disconnect) /* user requested, ignore 

Re: [PATCH v4 2/5]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

Hi,

On 2016 M06 30, Thu 14:02:02 CEST Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka <miku...@twibright.com>
> Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> 
> Changelog:
> Pranay Kr. Srivastava<pran...@gmail.com>:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
> socket. This solves the issue when kernel_sendmsg is currently
> blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> ---
>  drivers/block/nbd.c | 57
> + 1 file changed, 36
> insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 766c401..e362d44 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
> 
>  #include 
> +#include 
> 
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,8 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /* This is specifically for calling sock_shutdown, for now. */
> + struct work_struct ws_shutdown;
>  };
> 
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +98,8 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
> 
> +static void nbd_ws_func_shutdown(struct work_struct *);
> +

are you reading all the comments I had?...

At least respond to my comments if you disagree. I still can't see the benefit
of a function signature here if we can avoid it.

>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +177,36 @@ static void nbd_end_request(struct nbd_device *nbd,
> struct request *req) */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
> 
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
> 
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
> 
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
> 
>   if (list_empty(>queue_head))
>   return;
> 
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();
> + schedule_work(>ws_shutdown);
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down
> connection\n"); }
> 
> @@ -588,7 +590,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
> 
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (nbd->timedout) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);

I already commented on this in the last patch. This is unrelated to the patch.
If you disagree then please tell me why instead of sending the same thing
again.

Also brackets on the else part would be preferred.

Regards,

Markus

>   }
> 
>   nbd->task_send = NULL;
> @@ -663,6 +669,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   set_capacity(nbd->disk, 0);
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
> + INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -797,11 +804,11 @@ static int __nbd_ioctl(struct block_d

Re: [PATCH v4 2/5]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

Hi,

On 2016 M06 30, Thu 14:02:02 CEST Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka 
> Signed-off-by: Markus Pargmann 
> 
> Changelog:
> Pranay Kr. Srivastava:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
> socket. This solves the issue when kernel_sendmsg is currently
> blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 57
> + 1 file changed, 36
> insertions(+), 21 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 766c401..e362d44 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
> 
>  #include 
> +#include 
> 
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,8 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /* This is specifically for calling sock_shutdown, for now. */
> + struct work_struct ws_shutdown;
>  };
> 
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +98,8 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
> 
> +static void nbd_ws_func_shutdown(struct work_struct *);
> +

are you reading all the comments I had?...

At least respond to my comments if you disagree. I still can't see the benefit
of a function signature here if we can avoid it.

>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +177,36 @@ static void nbd_end_request(struct nbd_device *nbd,
> struct request *req) */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
> 
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
> 
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
> 
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
> 
>   if (list_empty(>queue_head))
>   return;
> 
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();
> + schedule_work(>ws_shutdown);
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down
> connection\n"); }
> 
> @@ -588,7 +590,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
> 
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (nbd->timedout) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);

I already commented on this in the last patch. This is unrelated to the patch.
If you disagree then please tell me why instead of sending the same thing
again.

Also brackets on the else part would be preferred.

Regards,

Markus

>   }
> 
>   nbd->task_send = NULL;
> @@ -663,6 +669,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   set_capacity(nbd->disk, 0);
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
> + INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -797,11 +804,11 @@ static int __nbd_ioctl(struct block_device *bdev,
> struct nbd_device *nbd, error = nbd_thread_recv(nbd, bdev);
>   n

Re: [PATCH v3 1/3]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

On Friday 24 June 2016 13:09:34 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka <miku...@twibright.com>
> Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> 
> Changelog:
> Pranay Kr. Srivastava<pran...@gmail.com>:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> ---
>  drivers/block/nbd.c | 69 
> ++---
>  1 file changed, 44 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 56f7f5d..586d946 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,10 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /*
> + *This is specifically for calling sock_shutdown, for now.
> + */

Please fix the indentation of this comment.

> + struct work_struct ws_shutdown;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +100,11 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
>  
> +/*
> + * Shutdown function for nbd_dev work struct.
> + */
> +static void nbd_ws_func_shutdown(struct work_struct *);

You could as well put the function implementation here. No need for a
function signature.

> +
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +182,35 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + schedule_work(>ws_shutdown);
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -574,8 +580,8 @@ static int nbd_thread_send(void *data)
>   while (!kthread_should_stop() || !list_empty(>waiting_queue)) {
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
> -  kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> + kthread_should_stop() ||
> + !list_empty(>waiting_queue));

This is unrelated, please remove.

>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -583,12 +589,16 @@ static int nbd_thread_send(void *data)
>  
>   spin_lock_irq(>queue_lock);
>   req = list_entry(nbd->waiting_queue.next, struct request,
> -  queuelist);
> + queuelist);

Unrelated as well.

>   list_del_init(>queuelist);
>   spin_unloc

Re: [PATCH v3 1/3]nbd: fix might_sleep warning on socket shutdown

[Markus Pargmann]

On Friday 24 June 2016 13:09:34 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka 
> Signed-off-by: Markus Pargmann 
> 
> Changelog:
> Pranay Kr. Srivastava:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 69 
> ++---
>  1 file changed, 44 insertions(+), 25 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 56f7f5d..586d946 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,10 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /*
> + *This is specifically for calling sock_shutdown, for now.
> + */

Please fix the indentation of this comment.

> + struct work_struct ws_shutdown;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +100,11 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
>  
> +/*
> + * Shutdown function for nbd_dev work struct.
> + */
> +static void nbd_ws_func_shutdown(struct work_struct *);

You could as well put the function implementation here. No need for a
function signature.

> +
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +182,35 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + schedule_work(>ws_shutdown);
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -574,8 +580,8 @@ static int nbd_thread_send(void *data)
>   while (!kthread_should_stop() || !list_empty(>waiting_queue)) {
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
> -  kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> + kthread_should_stop() ||
> + !list_empty(>waiting_queue));

This is unrelated, please remove.

>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -583,12 +589,16 @@ static int nbd_thread_send(void *data)
>  
>   spin_lock_irq(>queue_lock);
>   req = list_entry(nbd->waiting_queue.next, struct request,
> -  queuelist);
> + queuelist);

Unrelated as well.

>   list_del_init(>queuelist);
>   spin_unlock_irq(>queue_lock);
>  
> - /* handle request */

Unrelated.

>   nbd_

Re: [PATCH 3/3]nbd: make nbd device wait for its users

[Markus Pargmann]

Hi,

On Friday 24 June 2016 13:09:36 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> A timedout or a disconnected device, if in use, can't
> be used until it has been resetted. The resetting happens
> when all tasks having this bdev open closes this bdev.

Sorry, but this patch is unreadable. You are changing so many unrelated
whitespaces, lines, comments (that you introduced yourself in a previous
patch) and unrelated code. Please keep only the things that are
necessary for this single patch. Everything else can go into different
patches. Also it would be good to run checkpatch sometimes.

Also using your own atomic implementation instead of kref would be good
perhaps. Although I thought kref would work at the beginning but it
seems not to.

Best Regards,

Markus

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 124 
> 
>  1 file changed, 96 insertions(+), 28 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 9223b09..0587bbd 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -70,10 +70,13 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> +
>   /*
> - *This is specifically for calling sock_shutdown, for now.
> - */
> +  *This is specifically for calling sock_shutdown, for now.
> +  */
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -104,6 +107,8 @@ static DEFINE_SPINLOCK(nbd_lock);
>   * Shutdown function for nbd_dev work struct.
>   */
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
> +static int nbd_size_clear(struct nbd_device *, struct block_device *);
>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -129,7 +134,7 @@ static const char *nbdcmd_to_ascii(int cmd)
>  
>  static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
>  {
> - bdev->bd_inode->i_size = 0;
> + i_size_write(bdev->bd_inode, 0);
>   set_capacity(nbd->disk, 0);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  
> @@ -141,7 +146,7 @@ static void nbd_size_update(struct nbd_device *nbd, 
> struct block_device *bdev)
>   if (!nbd_is_connected(nbd))
>   return;
>  
> - bdev->bd_inode->i_size = nbd->bytesize;
> + i_size_write(bdev->bd_inode, nbd->bytesize);
>   set_capacity(nbd->disk, nbd->bytesize >> 9);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  }
> @@ -150,11 +155,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct 
> block_device *bdev,
>   int blocksize, int nr_blocks)
>  {
>   int ret;
> -
>   ret = set_blocksize(bdev, blocksize);
>   if (ret)
>   return ret;
> -
>   nbd->blksize = blocksize;
>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
>  
> @@ -202,14 +205,19 @@ static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
>  
> + if (nbd->timedout)
> + return;
> +
>   if (list_empty(>queue_head))
>   return;
> +
>   nbd->timedout = true;
> - schedule_work(>ws_shutdown);
> +
>   /*
>* Make sure sender thread sees nbd->timedout.
>*/
>   smp_wmb();
> + schedule_work(>ws_shutdown);
>   wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -476,8 +484,6 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct 
> block_device *bdev)
>   nbd_end_request(nbd, req);
>   }
>  
> - nbd_size_clear(nbd, bdev);
> -
>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
>  
>   nbd->task_recv = NULL;
> @@ -580,8 +586,8 @@ static int nbd_thread_send(void *data)
>   while (!kthread_should_stop() || !list_empty(>waiting_queue)) {
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
> - kthread_should_stop() ||
> - !list_empty(>waiting_queue));
> +  kthread_should_stop() ||
> +  !list_empty(>waiting_queue));
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -589,11 +595,11 @@ static 

Re: [PATCH 3/3]nbd: make nbd device wait for its users

[Markus Pargmann]

Hi,

On Friday 24 June 2016 13:09:36 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> A timedout or a disconnected device, if in use, can't
> be used until it has been resetted. The resetting happens
> when all tasks having this bdev open closes this bdev.

Sorry, but this patch is unreadable. You are changing so many unrelated
whitespaces, lines, comments (that you introduced yourself in a previous
patch) and unrelated code. Please keep only the things that are
necessary for this single patch. Everything else can go into different
patches. Also it would be good to run checkpatch sometimes.

Also using your own atomic implementation instead of kref would be good
perhaps. Although I thought kref would work at the beginning but it
seems not to.

Best Regards,

Markus

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 124 
> 
>  1 file changed, 96 insertions(+), 28 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 9223b09..0587bbd 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -70,10 +70,13 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> +
>   /*
> - *This is specifically for calling sock_shutdown, for now.
> - */
> +  *This is specifically for calling sock_shutdown, for now.
> +  */
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -104,6 +107,8 @@ static DEFINE_SPINLOCK(nbd_lock);
>   * Shutdown function for nbd_dev work struct.
>   */
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
> +static int nbd_size_clear(struct nbd_device *, struct block_device *);
>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -129,7 +134,7 @@ static const char *nbdcmd_to_ascii(int cmd)
>  
>  static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
>  {
> - bdev->bd_inode->i_size = 0;
> + i_size_write(bdev->bd_inode, 0);
>   set_capacity(nbd->disk, 0);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  
> @@ -141,7 +146,7 @@ static void nbd_size_update(struct nbd_device *nbd, 
> struct block_device *bdev)
>   if (!nbd_is_connected(nbd))
>   return;
>  
> - bdev->bd_inode->i_size = nbd->bytesize;
> + i_size_write(bdev->bd_inode, nbd->bytesize);
>   set_capacity(nbd->disk, nbd->bytesize >> 9);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  }
> @@ -150,11 +155,9 @@ static int nbd_size_set(struct nbd_device *nbd, struct 
> block_device *bdev,
>   int blocksize, int nr_blocks)
>  {
>   int ret;
> -
>   ret = set_blocksize(bdev, blocksize);
>   if (ret)
>   return ret;
> -
>   nbd->blksize = blocksize;
>   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
>  
> @@ -202,14 +205,19 @@ static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
>  
> + if (nbd->timedout)
> + return;
> +
>   if (list_empty(>queue_head))
>   return;
> +
>   nbd->timedout = true;
> - schedule_work(>ws_shutdown);
> +
>   /*
>* Make sure sender thread sees nbd->timedout.
>*/
>   smp_wmb();
> + schedule_work(>ws_shutdown);
>   wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -476,8 +484,6 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct 
> block_device *bdev)
>   nbd_end_request(nbd, req);
>   }
>  
> - nbd_size_clear(nbd, bdev);
> -
>   device_remove_file(disk_to_dev(nbd->disk), _attr_pid);
>  
>   nbd->task_recv = NULL;
> @@ -580,8 +586,8 @@ static int nbd_thread_send(void *data)
>   while (!kthread_should_stop() || !list_empty(>waiting_queue)) {
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
> - kthread_should_stop() ||
> - !list_empty(>waiting_queue));
> +  kthread_should_stop() ||
> +  !list_empty(>waiting_queue));
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -589,11 +595,11 @@ static int 

Re: [PATCH 1/2] nbd: make nbd device wait for its users

[Markus Pargmann]

Hi,

On Saturday 25 June 2016 23:22:06 Pranay Srivastava wrote:
> On Fri, Jun 24, 2016 at 2:59 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> > From: "Pranay Kr. Srivastava" <pran...@gmail.com>
> >
> > When a timeout occurs or a recv fails, then instead of abruplty killing
> > nbd block device wait for it's users to finish.
> >
> > This is more required when filesystem(s) like ext2 or ext3 don't expect
> > their buffer heads to disappear while the filesystem is mounted.
> >
> > Each open is counted. The blockdevice is kept open until the last user
> > closes the block device. This offers the possibility as well to open a
> > new socket to be used while the filesystems are mounted.
> >
> > Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> >
> > [mpa: Keep the blockdevice open until all users left]
> > Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> > ---
> > Hi,
> >
> > I used your patch and changed it a bit based on my ideas. The general
> > difference is that this keeps the block device open. After all users left, 
> > the
> > device is reset.
> >
> > The followup patch then restricts access to ioctls after a disconnect. I 
> > wanted
> > to avoid that anyone sets up anything new without all the old users leaving.
> >
> > Please let me know what you think about this.
> >
> > Best Regards,
> >
> > Markus
> >
> >  drivers/block/nbd.c | 62 
> > ++---
> >  1 file changed, 45 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > index 1efc26bf1d21..620660f3ff0f 100644
> > --- a/drivers/block/nbd.c
> > +++ b/drivers/block/nbd.c
> > @@ -69,6 +69,8 @@ struct nbd_device {
> >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > struct dentry *dbg_dir;
> >  #endif
> > +   atomic_t users; /* Users that opened the block device */
> 
> We don't need to put bdev in nbd struct. We can get it via gendisk and
> bdget/bdput.

Indeed, thanks.

> > +   struct block_device *bdev;
> >  };
> >
> >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > @@ -655,9 +657,26 @@ static int nbd_set_socket(struct nbd_device *nbd, 
> > struct socket *sock)
> > return ret;
> >  }
> >
> > +static void nbd_bdev_reset(struct block_device *bdev)
> > +{
> > +   set_device_ro(bdev, false);
> > +   bdev->bd_inode->i_size = 0;
> 
> i_size_write should be better I guess.

Yes, but that is a separate patch. This code is just moved

> 
> > +   if (max_part > 0) {
> > +   blkdev_reread_part(bdev);
> > +   bdev->bd_invalidated = 1;
> > +   }
> > +}
> > +
> >  /* Reset all properties of an NBD device */
> >  static void nbd_reset(struct nbd_device *nbd)
> >  {
> > +   sock_shutdown(nbd);
> > +   nbd_clear_que(nbd);
> > +   if (nbd->bdev) {
> > +   kill_bdev(nbd->bdev);
> > +   nbd_bdev_reset(nbd->bdev);
> > +   }
> > +
> > nbd->disconnect = false;
> > nbd->timedout = false;
> > nbd->blksize = 1024;
> 
> I actually forgot to ask about this blksize. We do set_blocksize call
> for bdev, but
> shouldn't we instead do blkdev_logicial_blocksize?
> 
> Mount would set the blocksize depending on the filesystem's block size and the
> block device logical block size supported. Should we just get rid of this?

I think we should perhaps set the logical blocksize as well. But that's
a separate topic as well.

> 
> > @@ -669,16 +688,6 @@ static void nbd_reset(struct nbd_device *nbd)
> > del_timer_sync(>timeout_timer);
> >  }
> >
> > -static void nbd_bdev_reset(struct block_device *bdev)
> > -{
> > -   set_device_ro(bdev, false);
> > -   bdev->bd_inode->i_size = 0;
> > -   if (max_part > 0) {
> > -   blkdev_reread_part(bdev);
> > -   bdev->bd_invalidated = 1;
> > -   }
> > -}
> > -
> >  static void nbd_parse_flags(struct nbd_device *nbd, struct block_device 
> > *bdev)
> >  {
> > if (nbd->flags & NBD_FLAG_READ_ONLY)
> > @@ -803,18 +812,11 @@ static int __nbd_ioctl(struct block_device *bdev, 
> > struct nbd_device *nbd,
> 
> How about disallowing any ioctl until the nbd_device has been reset?
> Perhaps, throw error
> in open instead of checking here?

This is a comment to patch 2?!
The

Re: [PATCH 1/2] nbd: make nbd device wait for its users

[Markus Pargmann]

Hi,

On Saturday 25 June 2016 23:22:06 Pranay Srivastava wrote:
> On Fri, Jun 24, 2016 at 2:59 PM, Markus Pargmann  wrote:
> > From: "Pranay Kr. Srivastava" 
> >
> > When a timeout occurs or a recv fails, then instead of abruplty killing
> > nbd block device wait for it's users to finish.
> >
> > This is more required when filesystem(s) like ext2 or ext3 don't expect
> > their buffer heads to disappear while the filesystem is mounted.
> >
> > Each open is counted. The blockdevice is kept open until the last user
> > closes the block device. This offers the possibility as well to open a
> > new socket to be used while the filesystems are mounted.
> >
> > Signed-off-by: Pranay Kr. Srivastava 
> >
> > [mpa: Keep the blockdevice open until all users left]
> > Signed-off-by: Markus Pargmann 
> > ---
> > Hi,
> >
> > I used your patch and changed it a bit based on my ideas. The general
> > difference is that this keeps the block device open. After all users left, 
> > the
> > device is reset.
> >
> > The followup patch then restricts access to ioctls after a disconnect. I 
> > wanted
> > to avoid that anyone sets up anything new without all the old users leaving.
> >
> > Please let me know what you think about this.
> >
> > Best Regards,
> >
> > Markus
> >
> >  drivers/block/nbd.c | 62 
> > ++---
> >  1 file changed, 45 insertions(+), 17 deletions(-)
> >
> > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > index 1efc26bf1d21..620660f3ff0f 100644
> > --- a/drivers/block/nbd.c
> > +++ b/drivers/block/nbd.c
> > @@ -69,6 +69,8 @@ struct nbd_device {
> >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > struct dentry *dbg_dir;
> >  #endif
> > +   atomic_t users; /* Users that opened the block device */
> 
> We don't need to put bdev in nbd struct. We can get it via gendisk and
> bdget/bdput.

Indeed, thanks.

> > +   struct block_device *bdev;
> >  };
> >
> >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > @@ -655,9 +657,26 @@ static int nbd_set_socket(struct nbd_device *nbd, 
> > struct socket *sock)
> > return ret;
> >  }
> >
> > +static void nbd_bdev_reset(struct block_device *bdev)
> > +{
> > +   set_device_ro(bdev, false);
> > +   bdev->bd_inode->i_size = 0;
> 
> i_size_write should be better I guess.

Yes, but that is a separate patch. This code is just moved

> 
> > +   if (max_part > 0) {
> > +   blkdev_reread_part(bdev);
> > +   bdev->bd_invalidated = 1;
> > +   }
> > +}
> > +
> >  /* Reset all properties of an NBD device */
> >  static void nbd_reset(struct nbd_device *nbd)
> >  {
> > +   sock_shutdown(nbd);
> > +   nbd_clear_que(nbd);
> > +   if (nbd->bdev) {
> > +   kill_bdev(nbd->bdev);
> > +   nbd_bdev_reset(nbd->bdev);
> > +   }
> > +
> > nbd->disconnect = false;
> > nbd->timedout = false;
> > nbd->blksize = 1024;
> 
> I actually forgot to ask about this blksize. We do set_blocksize call
> for bdev, but
> shouldn't we instead do blkdev_logicial_blocksize?
> 
> Mount would set the blocksize depending on the filesystem's block size and the
> block device logical block size supported. Should we just get rid of this?

I think we should perhaps set the logical blocksize as well. But that's
a separate topic as well.

> 
> > @@ -669,16 +688,6 @@ static void nbd_reset(struct nbd_device *nbd)
> > del_timer_sync(>timeout_timer);
> >  }
> >
> > -static void nbd_bdev_reset(struct block_device *bdev)
> > -{
> > -   set_device_ro(bdev, false);
> > -   bdev->bd_inode->i_size = 0;
> > -   if (max_part > 0) {
> > -   blkdev_reread_part(bdev);
> > -   bdev->bd_invalidated = 1;
> > -   }
> > -}
> > -
> >  static void nbd_parse_flags(struct nbd_device *nbd, struct block_device 
> > *bdev)
> >  {
> > if (nbd->flags & NBD_FLAG_READ_ONLY)
> > @@ -803,18 +812,11 @@ static int __nbd_ioctl(struct block_device *bdev, 
> > struct nbd_device *nbd,
> 
> How about disallowing any ioctl until the nbd_device has been reset?
> Perhaps, throw error
> in open instead of checking here?

This is a comment to patch 2?!
The idea was to not allow any control over the nbd device unless some
clear instructions for a cle

[PATCH 2/2] nbd: Disallow ioctls on disconnected block device

[Markus Pargmann]

After NBD_DO_IT exited the block device may still be used. Make sure
that we handle intended disconnects differently and do not allow any
changed of the nbd device.

This patch should avoid that the nbd-client connects to a different server
and the users of the block device are suddenly reading/writing from a
different backend device.

For timeouts it is still possible to setup a new socket so that the
connection may be refreshed without creating problems for all users.

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 30 --
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 620660f3ff0f..39358efac73e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -708,6 +708,18 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
 static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
   unsigned int cmd, unsigned long arg)
 {
+   /*
+* After a disconnect was instructed, do not allow any further actions
+* on the block device that would lead to a new connected endpoint.
+* This condition stays until nbd_reset was called either because all
+* users closed the device or because of CLEAR_SOCK.
+*/
+   if (nbd->disconnect &&
+   cmd != NBD_CLEAR_SOCK && cmd != NBD_PRINT_DEBUG) {
+   dev_info(disk_to_dev(nbd->disk), "Device is still busy after 
instructing a disconnect\n");
+   return -EBUSY;
+   }
+
switch (cmd) {
case NBD_DISCONNECT: {
struct request sreq;
@@ -733,11 +745,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
}
 
case NBD_CLEAR_SOCK:
-   sock_shutdown(nbd);
-   nbd_clear_que(nbd);
-   BUG_ON(!list_empty(>queue_head));
-   BUG_ON(!list_empty(>waiting_queue));
-   kill_bdev(bdev);
+   if (nbd->disconnect) {
+   nbd_reset(nbd);
+   } else {
+   sock_shutdown(nbd);
+   nbd_clear_que(nbd);
+   BUG_ON(!list_empty(>queue_head));
+   BUG_ON(!list_empty(>waiting_queue));
+   kill_bdev(bdev);
+   }
return 0;
 
case NBD_SET_SOCK: {
@@ -812,8 +828,10 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
mutex_lock(>tx_lock);
nbd->task_recv = NULL;
 
-   if (nbd->disconnect) /* user requested, ignore socket errors */
+   if (nbd->disconnect) { /* user requested, ignore socket errors 
*/
+   sock_shutdown(nbd);
error = 0;
+   }
if (nbd->timedout)
error = -ETIMEDOUT;
 
-- 
2.1.4

[PATCH 1/2] nbd: make nbd device wait for its users

[Markus Pargmann]

From: "Pranay Kr. Srivastava" <pran...@gmail.com>

When a timeout occurs or a recv fails, then instead of abruplty killing
nbd block device wait for it's users to finish.

This is more required when filesystem(s) like ext2 or ext3 don't expect
their buffer heads to disappear while the filesystem is mounted.

Each open is counted. The blockdevice is kept open until the last user
closes the block device. This offers the possibility as well to open a
new socket to be used while the filesystems are mounted.

Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>

[mpa: Keep the blockdevice open until all users left]
Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
Hi,

I used your patch and changed it a bit based on my ideas. The general
difference is that this keeps the block device open. After all users left, the
device is reset.

The followup patch then restricts access to ioctls after a disconnect. I wanted
to avoid that anyone sets up anything new without all the old users leaving.

Please let me know what you think about this.

Best Regards,

Markus

 drivers/block/nbd.c | 62 ++---
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 1efc26bf1d21..620660f3ff0f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -69,6 +69,8 @@ struct nbd_device {
 #if IS_ENABLED(CONFIG_DEBUG_FS)
struct dentry *dbg_dir;
 #endif
+   atomic_t users; /* Users that opened the block device */
+   struct block_device *bdev;
 };
 
 #if IS_ENABLED(CONFIG_DEBUG_FS)
@@ -655,9 +657,26 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
socket *sock)
return ret;
 }
 
+static void nbd_bdev_reset(struct block_device *bdev)
+{
+   set_device_ro(bdev, false);
+   bdev->bd_inode->i_size = 0;
+   if (max_part > 0) {
+   blkdev_reread_part(bdev);
+   bdev->bd_invalidated = 1;
+   }
+}
+
 /* Reset all properties of an NBD device */
 static void nbd_reset(struct nbd_device *nbd)
 {
+   sock_shutdown(nbd);
+   nbd_clear_que(nbd);
+   if (nbd->bdev) {
+   kill_bdev(nbd->bdev);
+   nbd_bdev_reset(nbd->bdev);
+   }
+
nbd->disconnect = false;
nbd->timedout = false;
nbd->blksize = 1024;
@@ -669,16 +688,6 @@ static void nbd_reset(struct nbd_device *nbd)
del_timer_sync(>timeout_timer);
 }
 
-static void nbd_bdev_reset(struct block_device *bdev)
-{
-   set_device_ro(bdev, false);
-   bdev->bd_inode->i_size = 0;
-   if (max_part > 0) {
-   blkdev_reread_part(bdev);
-   bdev->bd_invalidated = 1;
-   }
-}
-
 static void nbd_parse_flags(struct nbd_device *nbd, struct block_device *bdev)
 {
if (nbd->flags & NBD_FLAG_READ_ONLY)
@@ -803,18 +812,11 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
mutex_lock(>tx_lock);
nbd->task_recv = NULL;
 
-   sock_shutdown(nbd);
-   nbd_clear_que(nbd);
-   kill_bdev(bdev);
-   nbd_bdev_reset(bdev);
-
if (nbd->disconnect) /* user requested, ignore socket errors */
error = 0;
if (nbd->timedout)
error = -ETIMEDOUT;
 
-   nbd_reset(nbd);
-
return error;
}
 
@@ -853,10 +855,35 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
mode,
return error;
 }
 
+static int nbd_open(struct block_device *bdev, fmode_t mode)
+{
+   struct nbd_device *nbd = bdev->bd_disk->private_data;
+
+   atomic_inc(>users);
+
+   if (!nbd->bdev)
+   nbd->bdev = bdev;
+
+   return 0;
+}
+
+static void nbd_release(struct gendisk *disk, fmode_t mode)
+{
+   struct nbd_device *nbd = disk->private_data;
+
+   if (atomic_dec_and_test(>users)) {
+   mutex_lock(>tx_lock);
+   nbd_reset(nbd);
+   mutex_unlock(>tx_lock);
+   }
+}
+
 static const struct block_device_operations nbd_fops = {
.owner =THIS_MODULE,
.ioctl =nbd_ioctl,
.compat_ioctl = nbd_ioctl,
+   .open = nbd_open,
+   .release =  nbd_release,
 };
 
 #if IS_ENABLED(CONFIG_DEBUG_FS)
@@ -1087,6 +1114,7 @@ static int __init nbd_init(void)
disk->private_data = _dev[i];
sprintf(disk->disk_name, "nbd%d", i);
nbd_reset(_dev[i]);
+   atomic_set(_dev[i].users, 0);
add_disk(disk);
}
 
-- 
2.1.4

[PATCH 2/2] nbd: Disallow ioctls on disconnected block device

[Markus Pargmann]

After NBD_DO_IT exited the block device may still be used. Make sure
that we handle intended disconnects differently and do not allow any
changed of the nbd device.

This patch should avoid that the nbd-client connects to a different server
and the users of the block device are suddenly reading/writing from a
different backend device.

For timeouts it is still possible to setup a new socket so that the
connection may be refreshed without creating problems for all users.

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 30 --
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 620660f3ff0f..39358efac73e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -708,6 +708,18 @@ static void nbd_dev_dbg_close(struct nbd_device *nbd);
 static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
   unsigned int cmd, unsigned long arg)
 {
+   /*
+* After a disconnect was instructed, do not allow any further actions
+* on the block device that would lead to a new connected endpoint.
+* This condition stays until nbd_reset was called either because all
+* users closed the device or because of CLEAR_SOCK.
+*/
+   if (nbd->disconnect &&
+   cmd != NBD_CLEAR_SOCK && cmd != NBD_PRINT_DEBUG) {
+   dev_info(disk_to_dev(nbd->disk), "Device is still busy after 
instructing a disconnect\n");
+   return -EBUSY;
+   }
+
switch (cmd) {
case NBD_DISCONNECT: {
struct request sreq;
@@ -733,11 +745,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
}
 
case NBD_CLEAR_SOCK:
-   sock_shutdown(nbd);
-   nbd_clear_que(nbd);
-   BUG_ON(!list_empty(>queue_head));
-   BUG_ON(!list_empty(>waiting_queue));
-   kill_bdev(bdev);
+   if (nbd->disconnect) {
+   nbd_reset(nbd);
+   } else {
+   sock_shutdown(nbd);
+   nbd_clear_que(nbd);
+   BUG_ON(!list_empty(>queue_head));
+   BUG_ON(!list_empty(>waiting_queue));
+   kill_bdev(bdev);
+   }
return 0;
 
case NBD_SET_SOCK: {
@@ -812,8 +828,10 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
mutex_lock(>tx_lock);
nbd->task_recv = NULL;
 
-   if (nbd->disconnect) /* user requested, ignore socket errors */
+   if (nbd->disconnect) { /* user requested, ignore socket errors 
*/
+   sock_shutdown(nbd);
error = 0;
+   }
if (nbd->timedout)
error = -ETIMEDOUT;
 
-- 
2.1.4

[PATCH 1/2] nbd: make nbd device wait for its users

[Markus Pargmann]

From: "Pranay Kr. Srivastava" 

When a timeout occurs or a recv fails, then instead of abruplty killing
nbd block device wait for it's users to finish.

This is more required when filesystem(s) like ext2 or ext3 don't expect
their buffer heads to disappear while the filesystem is mounted.

Each open is counted. The blockdevice is kept open until the last user
closes the block device. This offers the possibility as well to open a
new socket to be used while the filesystems are mounted.

Signed-off-by: Pranay Kr. Srivastava 

[mpa: Keep the blockdevice open until all users left]
Signed-off-by: Markus Pargmann 
---
Hi,

I used your patch and changed it a bit based on my ideas. The general
difference is that this keeps the block device open. After all users left, the
device is reset.

The followup patch then restricts access to ioctls after a disconnect. I wanted
to avoid that anyone sets up anything new without all the old users leaving.

Please let me know what you think about this.

Best Regards,

Markus

 drivers/block/nbd.c | 62 ++---
 1 file changed, 45 insertions(+), 17 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 1efc26bf1d21..620660f3ff0f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -69,6 +69,8 @@ struct nbd_device {
 #if IS_ENABLED(CONFIG_DEBUG_FS)
struct dentry *dbg_dir;
 #endif
+   atomic_t users; /* Users that opened the block device */
+   struct block_device *bdev;
 };
 
 #if IS_ENABLED(CONFIG_DEBUG_FS)
@@ -655,9 +657,26 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
socket *sock)
return ret;
 }
 
+static void nbd_bdev_reset(struct block_device *bdev)
+{
+   set_device_ro(bdev, false);
+   bdev->bd_inode->i_size = 0;
+   if (max_part > 0) {
+   blkdev_reread_part(bdev);
+   bdev->bd_invalidated = 1;
+   }
+}
+
 /* Reset all properties of an NBD device */
 static void nbd_reset(struct nbd_device *nbd)
 {
+   sock_shutdown(nbd);
+   nbd_clear_que(nbd);
+   if (nbd->bdev) {
+   kill_bdev(nbd->bdev);
+   nbd_bdev_reset(nbd->bdev);
+   }
+
nbd->disconnect = false;
nbd->timedout = false;
nbd->blksize = 1024;
@@ -669,16 +688,6 @@ static void nbd_reset(struct nbd_device *nbd)
del_timer_sync(>timeout_timer);
 }
 
-static void nbd_bdev_reset(struct block_device *bdev)
-{
-   set_device_ro(bdev, false);
-   bdev->bd_inode->i_size = 0;
-   if (max_part > 0) {
-   blkdev_reread_part(bdev);
-   bdev->bd_invalidated = 1;
-   }
-}
-
 static void nbd_parse_flags(struct nbd_device *nbd, struct block_device *bdev)
 {
if (nbd->flags & NBD_FLAG_READ_ONLY)
@@ -803,18 +812,11 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
mutex_lock(>tx_lock);
nbd->task_recv = NULL;
 
-   sock_shutdown(nbd);
-   nbd_clear_que(nbd);
-   kill_bdev(bdev);
-   nbd_bdev_reset(bdev);
-
if (nbd->disconnect) /* user requested, ignore socket errors */
error = 0;
if (nbd->timedout)
error = -ETIMEDOUT;
 
-   nbd_reset(nbd);
-
return error;
}
 
@@ -853,10 +855,35 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
mode,
return error;
 }
 
+static int nbd_open(struct block_device *bdev, fmode_t mode)
+{
+   struct nbd_device *nbd = bdev->bd_disk->private_data;
+
+   atomic_inc(>users);
+
+   if (!nbd->bdev)
+   nbd->bdev = bdev;
+
+   return 0;
+}
+
+static void nbd_release(struct gendisk *disk, fmode_t mode)
+{
+   struct nbd_device *nbd = disk->private_data;
+
+   if (atomic_dec_and_test(>users)) {
+   mutex_lock(>tx_lock);
+   nbd_reset(nbd);
+   mutex_unlock(>tx_lock);
+   }
+}
+
 static const struct block_device_operations nbd_fops = {
.owner =THIS_MODULE,
.ioctl =nbd_ioctl,
.compat_ioctl = nbd_ioctl,
+   .open = nbd_open,
+   .release =  nbd_release,
 };
 
 #if IS_ENABLED(CONFIG_DEBUG_FS)
@@ -1087,6 +1114,7 @@ static int __init nbd_init(void)
disk->private_data = _dev[i];
sprintf(disk->disk_name, "nbd%d", i);
nbd_reset(_dev[i]);
+   atomic_set(_dev[i].users, 0);
add_disk(disk);
}
 
-- 
2.1.4

Re: [PATCH v2 4/5]nbd: make nbd device wait for its users.

[Markus Pargmann]

Hi Pranay,

On Tuesday 14 June 2016 15:03:40 Pranay Srivastava wrote:
> Hi Markus,
> 
> On Tue, Jun 14, 2016 at 2:29 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> >
> > On Thursday 02 June 2016 13:25:00 Pranay Kr. Srivastava wrote:
> > > When a timeout occurs or a recv fails, then
> > > instead of abruplty killing nbd block device
> > > wait for it's users to finish.
> > >
> > > This is more required when filesystem(s) like
> > > ext2 or ext3 don't expect their buffer heads to
> > > disappear while the filesystem is mounted.
> > >
> > > Each open of a nbd device is refcounted, while
> > > the userland program [nbd-client] doing the
> > > NBD_DO_IT ioctl would now wait for any other users
> > > of this device before invalidating the nbd device.
> > >
> > > Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> > > ---
> > >  drivers/block/nbd.c | 58 
> > > +
> > >  1 file changed, 58 insertions(+)
> > >
> > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > > index d1d898d..4da40dc 100644
> > > --- a/drivers/block/nbd.c
> > > +++ b/drivers/block/nbd.c
> > > @@ -70,10 +70,13 @@ struct nbd_device {
> > >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > >   struct dentry *dbg_dir;
> > >  #endif
> > > + atomic_t inuse;
> > >   /*
> > >*This is specifically for calling sock_shutdown, for now.
> > >*/
> > >   struct work_struct ws_shutdown;
> > > + struct kref users;
> > > + struct completion user_completion;
> > >  };
> > >
> > >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > > @@ -104,6 +107,7 @@ static DEFINE_SPINLOCK(nbd_lock);
> > >   * Shutdown function for nbd_dev work struct.
> > >   */
> > >  static void nbd_ws_func_shutdown(struct work_struct *);
> > > +static void nbd_kref_release(struct kref *);
> > >
> > >  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> > >  {
> > > @@ -682,6 +686,8 @@ static void nbd_reset(struct nbd_device *nbd)
> > >   nbd->flags = 0;
> > >   nbd->xmit_timeout = 0;
> > >   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> > > + init_completion(>user_completion);
> > > + kref_init(>users);
> > >   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
> > >   del_timer_sync(>timeout_timer);
> > >  }
> > > @@ -815,6 +821,14 @@ static int __nbd_ioctl(struct block_device *bdev, 
> > > struct nbd_device *nbd,
> > >   kthread_stop(thread);
> > >
> > >   sock_shutdown(nbd);
> > > + /*
> > > +  * kref_init initializes with ref count as 1,
> > > +  * nbd_client, or the user-land program executing
> > > +  * this ioctl will make the refcount to 2[at least]
> > > +  * so subtracting 2 from refcount.
> > > +  */
> > > + kref_sub(>users, 2, nbd_kref_release);
> >
> > Why don't you use a kref_put?
> 
> Ok, so I'll try to explain as I've understood the problem.
> 
> When the module is loaded the kref is initialized to 1.
> 
> Suppose now, someone has started nbd-client [nbdC-1] , then this
> nbd-client will increase the ref count to 2. So far so good...
> 
> Now let's say this device is being shutdown via nbd-client[nbdC-2].
> 
> nbdC-1 will subtract the refcount by two, it has to do in NBD_DO_IT
> since device file will not
> be closed until after ioctl is over, and it'll wait_for_completion.
> 
> nbdC-2 now closes it's use of device file, this makes the refcount as
> zero and completion
> is triggered with nbdC-1 completed.
> 
> Now we don't want to trigger kref_put when nbdC-1 closes the device
> file so kref_put needs
> to be conditional in this regard so for that in_use is used.
> 
> 
> >
> > > + wait_for_completion(>user_completion);
> > >   mutex_lock(>tx_lock);
> > >   nbd_clear_que(nbd);
> > >   kill_bdev(bdev);
> > > @@ -865,13 +879,56 @@ static int nbd_ioctl(struct block_device *bdev, 
> > > fmode_t mode,
> > >
> > >   return error;
> > >  }
> > > +static void nbd_kref_release(struct kref *kref_users)
> > > +{
> > > + struct nbd_device *nbd 

Re: [PATCH v2 4/5]nbd: make nbd device wait for its users.

[Markus Pargmann]

Hi Pranay,

On Tuesday 14 June 2016 15:03:40 Pranay Srivastava wrote:
> Hi Markus,
> 
> On Tue, Jun 14, 2016 at 2:29 PM, Markus Pargmann  wrote:
> >
> > On Thursday 02 June 2016 13:25:00 Pranay Kr. Srivastava wrote:
> > > When a timeout occurs or a recv fails, then
> > > instead of abruplty killing nbd block device
> > > wait for it's users to finish.
> > >
> > > This is more required when filesystem(s) like
> > > ext2 or ext3 don't expect their buffer heads to
> > > disappear while the filesystem is mounted.
> > >
> > > Each open of a nbd device is refcounted, while
> > > the userland program [nbd-client] doing the
> > > NBD_DO_IT ioctl would now wait for any other users
> > > of this device before invalidating the nbd device.
> > >
> > > Signed-off-by: Pranay Kr. Srivastava 
> > > ---
> > >  drivers/block/nbd.c | 58 
> > > +
> > >  1 file changed, 58 insertions(+)
> > >
> > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > > index d1d898d..4da40dc 100644
> > > --- a/drivers/block/nbd.c
> > > +++ b/drivers/block/nbd.c
> > > @@ -70,10 +70,13 @@ struct nbd_device {
> > >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > >   struct dentry *dbg_dir;
> > >  #endif
> > > + atomic_t inuse;
> > >   /*
> > >*This is specifically for calling sock_shutdown, for now.
> > >*/
> > >   struct work_struct ws_shutdown;
> > > + struct kref users;
> > > + struct completion user_completion;
> > >  };
> > >
> > >  #if IS_ENABLED(CONFIG_DEBUG_FS)
> > > @@ -104,6 +107,7 @@ static DEFINE_SPINLOCK(nbd_lock);
> > >   * Shutdown function for nbd_dev work struct.
> > >   */
> > >  static void nbd_ws_func_shutdown(struct work_struct *);
> > > +static void nbd_kref_release(struct kref *);
> > >
> > >  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
> > >  {
> > > @@ -682,6 +686,8 @@ static void nbd_reset(struct nbd_device *nbd)
> > >   nbd->flags = 0;
> > >   nbd->xmit_timeout = 0;
> > >   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> > > + init_completion(>user_completion);
> > > + kref_init(>users);
> > >   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
> > >   del_timer_sync(>timeout_timer);
> > >  }
> > > @@ -815,6 +821,14 @@ static int __nbd_ioctl(struct block_device *bdev, 
> > > struct nbd_device *nbd,
> > >   kthread_stop(thread);
> > >
> > >   sock_shutdown(nbd);
> > > + /*
> > > +  * kref_init initializes with ref count as 1,
> > > +  * nbd_client, or the user-land program executing
> > > +  * this ioctl will make the refcount to 2[at least]
> > > +  * so subtracting 2 from refcount.
> > > +  */
> > > + kref_sub(>users, 2, nbd_kref_release);
> >
> > Why don't you use a kref_put?
> 
> Ok, so I'll try to explain as I've understood the problem.
> 
> When the module is loaded the kref is initialized to 1.
> 
> Suppose now, someone has started nbd-client [nbdC-1] , then this
> nbd-client will increase the ref count to 2. So far so good...
> 
> Now let's say this device is being shutdown via nbd-client[nbdC-2].
> 
> nbdC-1 will subtract the refcount by two, it has to do in NBD_DO_IT
> since device file will not
> be closed until after ioctl is over, and it'll wait_for_completion.
> 
> nbdC-2 now closes it's use of device file, this makes the refcount as
> zero and completion
> is triggered with nbdC-1 completed.
> 
> Now we don't want to trigger kref_put when nbdC-1 closes the device
> file so kref_put needs
> to be conditional in this regard so for that in_use is used.
> 
> 
> >
> > > + wait_for_completion(>user_completion);
> > >   mutex_lock(>tx_lock);
> > >   nbd_clear_que(nbd);
> > >   kill_bdev(bdev);
> > > @@ -865,13 +879,56 @@ static int nbd_ioctl(struct block_device *bdev, 
> > > fmode_t mode,
> > >
> > >   return error;
> > >  }
> > > +static void nbd_kref_release(struct kref *kref_users)
> > > +{
> > > + struct nbd_device *nbd = container_of(kref_users, struc

Re: [PATCH v2 4/5]nbd: make nbd device wait for its users.

[Markus Pargmann]

On Thursday 02 June 2016 13:25:00 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 58 
> +
>  1 file changed, 58 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index d1d898d..4da40dc 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -70,10 +70,13 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + atomic_t inuse;
>   /*
>*This is specifically for calling sock_shutdown, for now.
>*/
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -104,6 +107,7 @@ static DEFINE_SPINLOCK(nbd_lock);
>   * Shutdown function for nbd_dev work struct.
>   */
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -682,6 +686,8 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> + init_completion(>user_completion);
> + kref_init(>users);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -815,6 +821,14 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   sock_shutdown(nbd);
> + /*
> +  * kref_init initializes with ref count as 1,
> +  * nbd_client, or the user-land program executing
> +  * this ioctl will make the refcount to 2[at least]
> +  * so subtracting 2 from refcount.
> +  */
> + kref_sub(>users, 2, nbd_kref_release);

Why don't you use a kref_put?

> + wait_for_completion(>user_completion);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
>   kill_bdev(bdev);
> @@ -865,13 +879,56 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>  
>   return error;
>  }
> +static void nbd_kref_release(struct kref *kref_users)
> +{
> + struct nbd_device *nbd = container_of(kref_users, struct nbd_device,
> + users);

Not indented to opening bracket.

> + pr_debug("Releasing kref [%s]\n", __func__);
> + atomic_set(>inuse, 0);
> + complete(>user_completion);
> +
> +}
> +
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> +
> + if (kref_get_unless_zero(_dev->users))
> + atomic_set(_dev->inuse, 1);
> +
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);

Indent to opening bracket.

> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + /*
> + *kref_init initializes ref count to 1, so we
> + *we check for refcount to be 2 for a final put.
> + *
> + *kref needs to be re-initialized just here as the
> + *other process holding it must see the ref count as 2.
> + */
> + if (atomic_read(_dev->inuse))
> + kref_put(_dev->users,  nbd_kref_release);

What is this inuse atomic for? Everyone that releases the nbd device
will need to execute a kref_put().

Best Regards,

Markus

> +
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> +}
>  
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
> +
>  static void nbd_ws_func_shutdown(struct work_struct *ws_nbd)
>  {
>   struct nbd_device *nbd_dev = container_of(ws_nbd, struct nbd_device,
> @@ -1107,6 +1164,7 @@ static int __init nbd_init(void)
>   disk->fops = _fops;
>   disk->private_data = _dev[i];
>   sprintf(disk->disk_name, "nbd%d", i);
> + atomic_set(_dev[i].inuse, 0);
>   

Re: [PATCH v2 4/5]nbd: make nbd device wait for its users.

[Markus Pargmann]

On Thursday 02 June 2016 13:25:00 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Each open of a nbd device is refcounted, while
> the userland program [nbd-client] doing the
> NBD_DO_IT ioctl would now wait for any other users
> of this device before invalidating the nbd device.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 58 
> +
>  1 file changed, 58 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index d1d898d..4da40dc 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -70,10 +70,13 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + atomic_t inuse;
>   /*
>*This is specifically for calling sock_shutdown, for now.
>*/
>   struct work_struct ws_shutdown;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -104,6 +107,7 @@ static DEFINE_SPINLOCK(nbd_lock);
>   * Shutdown function for nbd_dev work struct.
>   */
>  static void nbd_ws_func_shutdown(struct work_struct *);
> +static void nbd_kref_release(struct kref *);
>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -682,6 +686,8 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
> + init_completion(>user_completion);
> + kref_init(>users);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -815,6 +821,14 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   sock_shutdown(nbd);
> + /*
> +  * kref_init initializes with ref count as 1,
> +  * nbd_client, or the user-land program executing
> +  * this ioctl will make the refcount to 2[at least]
> +  * so subtracting 2 from refcount.
> +  */
> + kref_sub(>users, 2, nbd_kref_release);

Why don't you use a kref_put?

> + wait_for_completion(>user_completion);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
>   kill_bdev(bdev);
> @@ -865,13 +879,56 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>  
>   return error;
>  }
> +static void nbd_kref_release(struct kref *kref_users)
> +{
> + struct nbd_device *nbd = container_of(kref_users, struct nbd_device,
> + users);

Not indented to opening bracket.

> + pr_debug("Releasing kref [%s]\n", __func__);
> + atomic_set(>inuse, 0);
> + complete(>user_completion);
> +
> +}
> +
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> +
> + if (kref_get_unless_zero(_dev->users))
> + atomic_set(_dev->inuse, 1);
> +
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);

Indent to opening bracket.

> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + /*
> + *kref_init initializes ref count to 1, so we
> + *we check for refcount to be 2 for a final put.
> + *
> + *kref needs to be re-initialized just here as the
> + *other process holding it must see the ref count as 2.
> + */
> + if (atomic_read(_dev->inuse))
> + kref_put(_dev->users,  nbd_kref_release);

What is this inuse atomic for? Everyone that releases the nbd device
will need to execute a kref_put().

Best Regards,

Markus

> +
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> +}
>  
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
> +
>  static void nbd_ws_func_shutdown(struct work_struct *ws_nbd)
>  {
>   struct nbd_device *nbd_dev = container_of(ws_nbd, struct nbd_device,
> @@ -1107,6 +1164,7 @@ static int __init nbd_init(void)
>   disk->fops = _fops;
>   disk->private_data = _dev[i];
>   sprintf(disk->disk_name, "nbd%d", i);
> + atomic_set(_dev[i].inuse, 0);
>   

Re: [PATCH v2 1/5] nbd: fix might_sleep warning on socket shutdown.

[Markus Pargmann]

Hi,

On Thursday 02 June 2016 13:24:57 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka <miku...@twibright.com>
> Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> 
> Changelog:
> Pranay Kr. Srivastava<pran...@gmail.com>:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>

This looks better. Some smaller things inline. Also this patch does not
apply on my tree anymore. Can you please rebase it onto:

http://git.pengutronix.de/?p=mpa/linux-nbd.git;a=shortlog;h=refs/heads/master

> ---
>  drivers/block/nbd.c | 65 
> ++---
>  1 file changed, 42 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..0339d40 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,10 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /*
> +  *This is specifically for calling sock_shutdown, for now.
> +  */
> + struct work_struct ws_shutdown;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +100,11 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
>  
> +/*
> + * Shutdown function for nbd_dev work struct.
> + */
> +static void nbd_ws_func_shutdown(struct work_struct *);
> +
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +182,35 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + schedule_work(>ws_shutdown);
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();

I am not sure that we need this memory barrier here. But as it is just
the timeout path it probably won't hurt.

> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -592,7 +598,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (nbd->timedout) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -672,6 +682,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   set_capacity(nbd->disk, 0);
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
> + INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }

Re: [PATCH v2 1/5] nbd: fix might_sleep warning on socket shutdown.

[Markus Pargmann]

Hi,

On Thursday 02 June 2016 13:24:57 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka 
> Signed-off-by: Markus Pargmann 
> 
> Changelog:
> Pranay Kr. Srivastava:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> 
> Signed-off-by: Pranay Kr. Srivastava 

This looks better. Some smaller things inline. Also this patch does not
apply on my tree anymore. Can you please rebase it onto:

http://git.pengutronix.de/?p=mpa/linux-nbd.git;a=shortlog;h=refs/heads/master

> ---
>  drivers/block/nbd.c | 65 
> ++---
>  1 file changed, 42 insertions(+), 23 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..0339d40 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -69,6 +70,10 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + /*
> +  *This is specifically for calling sock_shutdown, for now.
> +  */
> + struct work_struct ws_shutdown;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -95,6 +100,11 @@ static int max_part;
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
>  
> +/*
> + * Shutdown function for nbd_dev work struct.
> + */
> +static void nbd_ws_func_shutdown(struct work_struct *);
> +
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
>   return disk_to_dev(nbd->disk);
> @@ -172,39 +182,35 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
>   nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + schedule_work(>ws_shutdown);
> + /*
> +  * Make sure sender thread sees nbd->timedout.
> +  */
> + smp_wmb();

I am not sure that we need this memory barrier here. But as it is just
the timeout path it probably won't hurt.

> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -592,7 +598,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (nbd->timedout) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -672,6 +682,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   set_capacity(nbd->disk, 0);
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
> + INIT_WORK(>ws_shutdown, nbd_ws_func_shutdown);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -804,15 +815,15 @@ static int __nbd_ioctl(struct block_device *bdev, 
> struct nbd_devic

Re: [PATCH] nbd: fix race in ioctl

[Markus Pargmann]

Hi,

On Friday 27 May 2016 12:59:35 Vegard Nossum wrote:
> Quentin ran into this bug:
> 
> WARNING: CPU: 64 PID: 10085 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x65/0x80
> sysfs: cannot create duplicate filename '/devices/virtual/block/nbd3/pid'
> Modules linked in: nbd
> CPU: 64 PID: 10085 Comm: qemu-nbd Tainted: G  D 4.6.0+ #7
>   8820330bba68 814b8791 8820330bbac8
>   8820330bbab8 810d04ab 8820330bbaa8
>  001f0296 00017681 8810380bf000 a0001790
> Call Trace:
>  [] dump_stack+0x4d/0x6c
>  [] __warn+0xdb/0x100
>  [] warn_slowpath_fmt+0x44/0x50
>  [] sysfs_warn_dup+0x65/0x80
>  [] sysfs_add_file_mode_ns+0x172/0x180
>  [] sysfs_create_file_ns+0x25/0x30
>  [] device_create_file+0x36/0x90
>  [] __nbd_ioctl+0x32d/0x9b0 [nbd]
>  [] ? find_next_bit+0x18/0x20
>  [] ? select_idle_sibling+0xe9/0x120
>  [] ? __enqueue_entity+0x67/0x70
>  [] ? enqueue_task_fair+0x630/0xe20
>  [] ? resched_curr+0x36/0x70
>  [] ? check_preempt_curr+0x78/0x90
>  [] ? ttwu_do_wakeup+0x12/0x80
>  [] ? ttwu_do_activate.constprop.86+0x61/0x70
>  [] ? try_to_wake_up+0x185/0x2d0
>  [] ? default_wake_function+0xd/0x10
>  [] ? autoremove_wake_function+0x11/0x40
>  [] nbd_ioctl+0x67/0x94 [nbd]
>  [] blkdev_ioctl+0x14d/0x940
>  [] ? put_pipe_info+0x22/0x60
>  [] block_ioctl+0x3c/0x40
>  [] do_vfs_ioctl+0x8d/0x5e0
>  [] ? fput+0x9/0x10
>  [] ? task_work_run+0x72/0x90
>  [] SyS_ioctl+0x47/0x80
>  [] entry_SYSCALL_64_fastpath+0x17/0x93
> ---[ end trace 7899b295e4f850c8 ]---
> 
> It seems fairly obvious that device_create_file() is not being protected
> from being run concurrently on the same nbd.
> 
> Quentin found the following relevant commits:
> 
> 1a2ad21 nbd: add locking to nbd_ioctl
> 90b8f28 [PATCH] end of methods switch: remove the old ones
> d4430d6 [PATCH] beginning of methods conversion
> 08f8585 [PATCH] move block_device_operations to blkdev.h
> 
> It would seem that the race was introduced in the process of moving nbd
> from BKL to unlocked ioctls.
> 
> By setting nbd->task_recv while the mutex is held, we can prevent other
> processes from running concurrently (since nbd->task_recv is also checked
> while the mutex is held).
> 
> Reported-and-tested-by: Quentin Casasnovas <quentin.casasno...@oracle.com>
> Cc: Markus Pargmann <m...@pengutronix.de>
> Cc: Paul Clements <paul.cleme...@steeleye.com>
> Cc: Pavel Machek <pa...@suse.cz>
> Cc: Jens Axboe <ax...@fb.com>
> Cc: Al Viro <v...@zeniv.linux.org.uk>
> Signed-off-by: Vegard Nossum <vegard.nos...@oracle.com>

Thanks, applied.

Best Regards,

Markus

> ---
>  drivers/block/nbd.c | 12 
>  1 file changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..a831f2b 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -451,14 +451,9 @@ static int nbd_thread_recv(struct nbd_device *nbd, 
> struct block_device *bdev)
>  
>   sk_set_memalloc(nbd->sock->sk);
>  
> - nbd->task_recv = current;
> -
>   ret = device_create_file(disk_to_dev(nbd->disk), _attr);
>   if (ret) {
>   dev_err(disk_to_dev(nbd->disk), "device_create_file failed!\n");
> -
> - nbd->task_recv = NULL;
> -
>   return ret;
>   }
>  
> @@ -477,9 +472,6 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct 
> block_device *bdev)
>   nbd_size_clear(nbd, bdev);
>  
>   device_remove_file(disk_to_dev(nbd->disk), _attr);
> -
> - nbd->task_recv = NULL;
> -
>   return ret;
>  }
>  
> @@ -788,6 +780,8 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   if (!nbd->sock)
>   return -EINVAL;
>  
> + /* We have to claim the device under the lock */
> + nbd->task_recv = current;
>   mutex_unlock(>tx_lock);
>  
>   nbd_parse_flags(nbd, bdev);
> @@ -796,6 +790,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>nbd_name(nbd));
>   if (IS_ERR(thread)) {
>   mutex_lock(>tx_lock);
> + nbd->task_recv = NULL;
>   return PTR_ERR(thread);
>   }
>  
> @@ -805,6 +800,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   mutex_lock(>tx_lock);
> + nbd->task_recv = NULL;
>  
>   sock_shutdown(nbd);
>   nbd_clear_que(nbd);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] nbd: fix race in ioctl

[Markus Pargmann]

Hi,

On Friday 27 May 2016 12:59:35 Vegard Nossum wrote:
> Quentin ran into this bug:
> 
> WARNING: CPU: 64 PID: 10085 at fs/sysfs/dir.c:31 sysfs_warn_dup+0x65/0x80
> sysfs: cannot create duplicate filename '/devices/virtual/block/nbd3/pid'
> Modules linked in: nbd
> CPU: 64 PID: 10085 Comm: qemu-nbd Tainted: G  D 4.6.0+ #7
>   8820330bba68 814b8791 8820330bbac8
>   8820330bbab8 810d04ab 8820330bbaa8
>  001f0296 00017681 8810380bf000 a0001790
> Call Trace:
>  [] dump_stack+0x4d/0x6c
>  [] __warn+0xdb/0x100
>  [] warn_slowpath_fmt+0x44/0x50
>  [] sysfs_warn_dup+0x65/0x80
>  [] sysfs_add_file_mode_ns+0x172/0x180
>  [] sysfs_create_file_ns+0x25/0x30
>  [] device_create_file+0x36/0x90
>  [] __nbd_ioctl+0x32d/0x9b0 [nbd]
>  [] ? find_next_bit+0x18/0x20
>  [] ? select_idle_sibling+0xe9/0x120
>  [] ? __enqueue_entity+0x67/0x70
>  [] ? enqueue_task_fair+0x630/0xe20
>  [] ? resched_curr+0x36/0x70
>  [] ? check_preempt_curr+0x78/0x90
>  [] ? ttwu_do_wakeup+0x12/0x80
>  [] ? ttwu_do_activate.constprop.86+0x61/0x70
>  [] ? try_to_wake_up+0x185/0x2d0
>  [] ? default_wake_function+0xd/0x10
>  [] ? autoremove_wake_function+0x11/0x40
>  [] nbd_ioctl+0x67/0x94 [nbd]
>  [] blkdev_ioctl+0x14d/0x940
>  [] ? put_pipe_info+0x22/0x60
>  [] block_ioctl+0x3c/0x40
>  [] do_vfs_ioctl+0x8d/0x5e0
>  [] ? fput+0x9/0x10
>  [] ? task_work_run+0x72/0x90
>  [] SyS_ioctl+0x47/0x80
>  [] entry_SYSCALL_64_fastpath+0x17/0x93
> ---[ end trace 7899b295e4f850c8 ]---
> 
> It seems fairly obvious that device_create_file() is not being protected
> from being run concurrently on the same nbd.
> 
> Quentin found the following relevant commits:
> 
> 1a2ad21 nbd: add locking to nbd_ioctl
> 90b8f28 [PATCH] end of methods switch: remove the old ones
> d4430d6 [PATCH] beginning of methods conversion
> 08f8585 [PATCH] move block_device_operations to blkdev.h
> 
> It would seem that the race was introduced in the process of moving nbd
> from BKL to unlocked ioctls.
> 
> By setting nbd->task_recv while the mutex is held, we can prevent other
> processes from running concurrently (since nbd->task_recv is also checked
> while the mutex is held).
> 
> Reported-and-tested-by: Quentin Casasnovas 
> Cc: Markus Pargmann 
> Cc: Paul Clements 
> Cc: Pavel Machek 
> Cc: Jens Axboe 
> Cc: Al Viro 
> Signed-off-by: Vegard Nossum 

Thanks, applied.

Best Regards,

Markus

> ---
>  drivers/block/nbd.c | 12 
>  1 file changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..a831f2b 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -451,14 +451,9 @@ static int nbd_thread_recv(struct nbd_device *nbd, 
> struct block_device *bdev)
>  
>   sk_set_memalloc(nbd->sock->sk);
>  
> - nbd->task_recv = current;
> -
>   ret = device_create_file(disk_to_dev(nbd->disk), _attr);
>   if (ret) {
>   dev_err(disk_to_dev(nbd->disk), "device_create_file failed!\n");
> -
> - nbd->task_recv = NULL;
> -
>   return ret;
>   }
>  
> @@ -477,9 +472,6 @@ static int nbd_thread_recv(struct nbd_device *nbd, struct 
> block_device *bdev)
>   nbd_size_clear(nbd, bdev);
>  
>   device_remove_file(disk_to_dev(nbd->disk), _attr);
> -
> - nbd->task_recv = NULL;
> -
>   return ret;
>  }
>  
> @@ -788,6 +780,8 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   if (!nbd->sock)
>   return -EINVAL;
>  
> + /* We have to claim the device under the lock */
> + nbd->task_recv = current;
>   mutex_unlock(>tx_lock);
>  
>   nbd_parse_flags(nbd, bdev);
> @@ -796,6 +790,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>nbd_name(nbd));
>   if (IS_ERR(thread)) {
>   mutex_lock(>tx_lock);
> + nbd->task_recv = NULL;
>   return PTR_ERR(thread);
>   }
>  
> @@ -805,6 +800,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   mutex_lock(>tx_lock);
> + nbd->task_recv = NULL;
>  
>   sock_shutdown(nbd);
>   nbd_clear_que(nbd);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 0/4]nbd: fixes for nbd

[Markus Pargmann]

Hi,

On Tuesday 24 May 2016 14:26:24 Pranay Kr. Srivastava wrote:
> This patch series fixes the following
> 
> 1) fix might_sleep warning on socket shutdown:
>Fix sock_shutdown to avoid calling kernel_sock_shutdown
>while holding spin_lock.
> 
> 2)fix various coding standard warnings
>Make shutdown get called in a process context instead, using
>system_wq.
> 
> 3) make nbd device wait for its users.
>When a timeout or error occurs then nbd driver simply kills
>the block device. Many filesystem(s) example ext2/ext3 don't
>expect their buffer heads to disappear like that. Fix this
>by making nbd device wait for its users.
> 
> 4) use device_attr macros for sysfs attribute
>use DEVICE_ATTR_RO for sysfs pid attribute.

Thanks, I applied patches 2 and 4. For the next version please add a
"nbd: " prefix to the other patches.

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 0/4]nbd: fixes for nbd

[Markus Pargmann]

Hi,

On Tuesday 24 May 2016 14:26:24 Pranay Kr. Srivastava wrote:
> This patch series fixes the following
> 
> 1) fix might_sleep warning on socket shutdown:
>Fix sock_shutdown to avoid calling kernel_sock_shutdown
>while holding spin_lock.
> 
> 2)fix various coding standard warnings
>Make shutdown get called in a process context instead, using
>system_wq.
> 
> 3) make nbd device wait for its users.
>When a timeout or error occurs then nbd driver simply kills
>the block device. Many filesystem(s) example ext2/ext3 don't
>expect their buffer heads to disappear like that. Fix this
>by making nbd device wait for its users.
> 
> 4) use device_attr macros for sysfs attribute
>use DEVICE_ATTR_RO for sysfs pid attribute.

Thanks, I applied patches 2 and 4. For the next version please add a
"nbd: " prefix to the other patches.

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 1/4] fix might_sleep warning on socket shutdown.

[Markus Pargmann]

Hi,

On Tuesday 24 May 2016 14:26:25 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka <miku...@twibright.com>
> Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> 
> Changelog:
> Pranay Kr. Srivastava<pran...@gmail.com>:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> ---
>  drivers/block/nbd.c |   62 
> +--
>  1 file changed, 35 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 08afbc7..a5dab48 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -57,7 +58,7 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;

Why are you using atomic here? It seems you are just counting the
occurences of timeouts with this but never actually use that number. 

>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
> @@ -69,6 +70,7 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + struct work_struct ws_nbd;

Can you rename this so that it is clear that this is a worker struct
specifically for the socket shutdown?

>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -94,6 +96,7 @@ static int max_part;
>   * Thanks go to Jens Axboe and Al Viro for their LKML emails explaining this!
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
> +static void nbd_work_func(struct work_struct *);

Same here. nbd_work_func() doesn't give any hint about the real purpose
of the function.

>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -172,39 +175,31 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + atomic_inc(>timedout);
> + schedule_work(>ws_nbd);
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -592,7 +587,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -666,12 +665,13 @@ out:
>  static void nbd_reset(struct nbd_device *nbd)
>  {
>   nbd->disconnect = false;
> - nbd->timedout = false;
> + atomic_set(>timedout, 0);
>   nbd->blksize = 1024;
>   nbd->bytesize = 0;

Re: [PATCH 1/4] fix might_sleep warning on socket shutdown.

[Markus Pargmann]

Hi,

On Tuesday 24 May 2016 14:26:25 Pranay Kr. Srivastava wrote:
> spinlocked ranges should be small and not contain calls into huge
> subfunctions. Fix my mistake and just get the pointer to the socket
> instead of doing everything with spinlock held.
> 
> Reported-by: Mikulas Patocka 
> Signed-off-by: Markus Pargmann 
> 
> Changelog:
> Pranay Kr. Srivastava:
> 
> 1) Use spin_lock instead of irq version for sock_shutdown.
> 
> 2) Use system work queue to actually trigger the shutdown of
>socket. This solves the issue when kernel_sendmsg is currently
>blocked while a timeout occurs.
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c |   62 
> +--
>  1 file changed, 35 insertions(+), 27 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 08afbc7..a5dab48 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -39,6 +39,7 @@
>  #include 
>  
>  #include 
> +#include 
>  
>  struct nbd_device {
>   u32 flags;
> @@ -57,7 +58,7 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;

Why are you using atomic here? It seems you are just counting the
occurences of timeouts with this but never actually use that number. 

>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
> @@ -69,6 +70,7 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + struct work_struct ws_nbd;

Can you rename this so that it is clear that this is a worker struct
specifically for the socket shutdown?

>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -94,6 +96,7 @@ static int max_part;
>   * Thanks go to Jens Axboe and Al Viro for their LKML emails explaining this!
>   */
>  static DEFINE_SPINLOCK(nbd_lock);
> +static void nbd_work_func(struct work_struct *);

Same here. nbd_work_func() doesn't give any hint about the real purpose
of the function.

>  
>  static inline struct device *nbd_to_dev(struct nbd_device *nbd)
>  {
> @@ -172,39 +175,31 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> - if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> - return;
> - }
> + struct socket *sock;
>  
> - dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> - sockfd_put(nbd->sock);
> + spin_lock(>sock_lock);
> + sock = nbd->sock;
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> + spin_unlock(>sock_lock);
> +
> + if (!sock)
> + return;
>  
>   del_timer(>timeout_timer);
> + dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> + kernel_sock_shutdown(sock, SHUT_RDWR);
> + sockfd_put(sock);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
> -
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> -
> + atomic_inc(>timedout);
> + schedule_work(>ws_nbd);
> + wake_up(>waiting_wq);
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
>  
> @@ -592,7 +587,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -666,12 +665,13 @@ out:
>  static void nbd_reset(struct nbd_device *nbd)
>  {
>   nbd->disconnect = false;
> - nbd->timedout = false;
> + atomic_set(>timedout, 0);
>   nbd->blksize = 1024;
>   nbd->bytesize = 0;
>   set_capacity(nbd->disk, 0);
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
> + INIT_WORK(

Re: [PATCH 3/4] make nbd device wait for its users.

[Markus Pargmann]

Hi,

sorry I couldn't fit the review into last week.

On Tuesday 24 May 2016 14:26:27 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Use a kref for users using this. The device will
> be released for kref count of 2, not less or more.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c |   51 
> +++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index af86c9b..59db890 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -71,6 +71,8 @@ struct nbd_device {
>   struct dentry *dbg_dir;
>  #endif
>   struct work_struct ws_nbd;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -674,6 +676,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_nbd, nbd_work_func);
> + init_completion(>user_completion);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -807,6 +810,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   sock_shutdown(nbd);
> + wait_for_completion(>user_completion);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
>   kill_bdev(bdev);
> @@ -858,12 +862,58 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static void nbd_kref_release(struct kref *kref_users)
> +{
> + struct nbd_device *nbd = container_of(kref_users, struct nbd_device,
> + users);
> + pr_debug("Releasing kref [%s]\n", __FUNCTION__);
> + complete(>user_completion);
> +
> +}
> +
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> +
> + kref_get(_dev->users);
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + /*
> + *kref_init initializes ref count to 1, so we
> + *we check for refcount to be 2 for a final put.
> + *
> + *kref needs to be re-initialized just here as the
> + *other process holding it must see the ref count as 2.
> + */
> + kref_put(_dev->users,  nbd_kref_release);
> +
> + if (atomic_read(_dev->users.refcount) == 2) {
> + kref_sub(_dev->users, 2, nbd_kref_release);
> + kref_init(_dev->users);
> + kref_get(_dev->users);

Reading the refcount directly seems not to be as it supposed to be.

Why don't you put a kref_init() and kref_put() call into NBD_DO_IT? This
way you don't have to work around the property that kref_init() starts
with a refcount of 1 but you can use it.

For example:
NBD_DO_IT:
kref_init()
...
kref_put()

nbd_thread_recv() and nbd_thread_send():
kref_get()
...
kref_put()

In nbd_open() you could use kref_get_unless_zero() to avoid
opening a not connected device.

nbd_release() would then be a very simple kref_put() without
checking for 2 and so on.

Also there are some checkpatch issues with this patch.

Best Regards,

Markus

> + }
> +
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
> +
>  static void nbd_work_func(struct work_struct *ws_nbd)
>  {
>   struct nbd_device *nbd_dev = container_of(ws_nbd, struct nbd_device,
> @@ -1098,6 +1148,7 @@ static int __init nbd_init(void)
>   disk->first_minor = i << part_shift;
>   disk->fops = _fops;
>   disk->private_data = _dev[i];
> + kref_init(_dev[i].users);
>   sprintf(disk->disk_name, "nbd%d", i);
>   nbd_reset(_dev[i]);
>   add_disk(disk);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | 

Re: [PATCH 3/4] make nbd device wait for its users.

[Markus Pargmann]

Hi,

sorry I couldn't fit the review into last week.

On Tuesday 24 May 2016 14:26:27 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> Use a kref for users using this. The device will
> be released for kref count of 2, not less or more.
> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c |   51 
> +++
>  1 file changed, 51 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index af86c9b..59db890 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -71,6 +71,8 @@ struct nbd_device {
>   struct dentry *dbg_dir;
>  #endif
>   struct work_struct ws_nbd;
> + struct kref users;
> + struct completion user_completion;
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -674,6 +676,7 @@ static void nbd_reset(struct nbd_device *nbd)
>   nbd->flags = 0;
>   nbd->xmit_timeout = 0;
>   INIT_WORK(>ws_nbd, nbd_work_func);
> + init_completion(>user_completion);
>   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
>   del_timer_sync(>timeout_timer);
>  }
> @@ -807,6 +810,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   kthread_stop(thread);
>  
>   sock_shutdown(nbd);
> + wait_for_completion(>user_completion);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
>   kill_bdev(bdev);
> @@ -858,12 +862,58 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static void nbd_kref_release(struct kref *kref_users)
> +{
> + struct nbd_device *nbd = container_of(kref_users, struct nbd_device,
> + users);
> + pr_debug("Releasing kref [%s]\n", __FUNCTION__);
> + complete(>user_completion);
> +
> +}
> +
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> +
> + kref_get(_dev->users);
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + /*
> + *kref_init initializes ref count to 1, so we
> + *we check for refcount to be 2 for a final put.
> + *
> + *kref needs to be re-initialized just here as the
> + *other process holding it must see the ref count as 2.
> + */
> + kref_put(_dev->users,  nbd_kref_release);
> +
> + if (atomic_read(_dev->users.refcount) == 2) {
> + kref_sub(_dev->users, 2, nbd_kref_release);
> + kref_init(_dev->users);
> + kref_get(_dev->users);

Reading the refcount directly seems not to be as it supposed to be.

Why don't you put a kref_init() and kref_put() call into NBD_DO_IT? This
way you don't have to work around the property that kref_init() starts
with a refcount of 1 but you can use it.

For example:
NBD_DO_IT:
kref_init()
...
kref_put()

nbd_thread_recv() and nbd_thread_send():
kref_get()
...
kref_put()

In nbd_open() you could use kref_get_unless_zero() to avoid
opening a not connected device.

nbd_release() would then be a very simple kref_put() without
checking for 2 and so on.

Also there are some checkpatch issues with this patch.

Best Regards,

Markus

> + }
> +
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name,
> + atomic_read(_dev->users.refcount) - 1);
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
> +
>  static void nbd_work_func(struct work_struct *ws_nbd)
>  {
>   struct nbd_device *nbd_dev = container_of(ws_nbd, struct nbd_device,
> @@ -1098,6 +1148,7 @@ static int __init nbd_init(void)
>   disk->first_minor = i << part_shift;
>   disk->fops = _fops;
>   disk->private_data = _dev[i];
> + kref_init(_dev[i].users);
>   sprintf(disk->disk_name, "nbd%d", i);
>   nbd_reset(_dev[i]);
>   add_disk(disk);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

On Friday 20 May 2016 02:05:36 Pranay Srivastava wrote:
> On Thu, May 19, 2016 at 11:52 AM, Markus Pargmann <m...@pengutronix.de> wrote:
> > Hi,
> >
> > On Wed, May 11, 2016 at 11:18:29AM +0300, Pranay Kr. Srivastava wrote:
> >> This patch fixes the warning generated when a timeout occurs
> >> on the request and socket is closed from a non-sleep context
> >> by
> >>
> >> 1. Moving the socket closing on a timeout to nbd_thread_send
> >
> > What happens if a send blocks?
> 
> socket closing needs to be moved to a non-atomic context and,
> sender thread seemed to be a good place to do this. If you mean
> send blocks just before calling sock_shutdown[?] in nbd_thread_send
> then yes I think that should be removed. I need to re-check how nbd-server
> behaves in that case.

No that's not what I meant. Your approach uses the sender thread as a
worker to close the socket. You are using waiting_wq to notify the
sender thread. That's fine so far.

But what happens if the sender thread is at this point trying to send on
the socket which blocks? Then the timeout triggers and waiting_wq will
notify the sending thread as soon as it left the sending routine. But it
will not interrupt the thread that is waiting in kernel_sendmsg() and
the sending thread will be stuck much longer than specified in the
timeout. 

> 
> >
> >>
> >> 2. Make sock lock to be a mutex instead of a spin lock, since
> >>nbd_xmit_timeout doesn't need to hold it anymore.
> >
> > I can't see why we need a mutex instead of a spinlock?
> 
> you are right, with your earlier patch we don't need it to be a mutex.
> 
> >
> >>
> >> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> >> ---
> >>  drivers/block/nbd.c | 65 
> >> -
> >>  1 file changed, 39 insertions(+), 26 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 31e73a7..c79bcd7 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -57,12 +57,12 @@ struct nbd_device {
> >>   int blksize;
> >>   loff_t bytesize;
> >>   int xmit_timeout;
> >> - bool timedout;
> >> + atomic_t timedout;
> >>   bool disconnect; /* a disconnect has been requested by user */
> >>
> >>   struct timer_list timeout_timer;
> >>   /* protects initialization and shutdown of the socket */
> >> - spinlock_t sock_lock;
> >> + struct mutex sock_lock;
> >>   struct task_struct *task_recv;
> >>   struct task_struct *task_send;
> >>
> >> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> >> struct request *req)
> >>   */
> >>  static void sock_shutdown(struct nbd_device *nbd)
> >>  {
> >> - spin_lock_irq(>sock_lock);
> >> -
> >> + mutex_lock(>sock_lock);
> >>   if (!nbd->sock) {
> >> - spin_unlock_irq(>sock_lock);
> >> + mutex_unlock(>sock_lock);
> >>   return;
> >>   }
> >>
> >> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
> >>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >>   sockfd_put(nbd->sock);
> >>   nbd->sock = NULL;
> >> - spin_unlock_irq(>sock_lock);
> >> -
> >> + mutex_unlock(>sock_lock);
> >>   del_timer(>timeout_timer);
> >>  }
> >>
> >>  static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >> - unsigned long flags;
> >>
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> - spin_lock_irqsave(>sock_lock, flags);
> >> -
> >> - nbd->timedout = true;
> >> -
> >> - if (nbd->sock)
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> -
> >> - spin_unlock_irqrestore(>sock_lock, flags);
> >> + atomic_inc(>timedout);
> >> + wake_up(>waiting_wq);
> >>
> >>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> >> connection\n");
> >>  }
> >> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
> >>   /* wait for something to do */
> >>

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

On Friday 20 May 2016 02:05:36 Pranay Srivastava wrote:
> On Thu, May 19, 2016 at 11:52 AM, Markus Pargmann  wrote:
> > Hi,
> >
> > On Wed, May 11, 2016 at 11:18:29AM +0300, Pranay Kr. Srivastava wrote:
> >> This patch fixes the warning generated when a timeout occurs
> >> on the request and socket is closed from a non-sleep context
> >> by
> >>
> >> 1. Moving the socket closing on a timeout to nbd_thread_send
> >
> > What happens if a send blocks?
> 
> socket closing needs to be moved to a non-atomic context and,
> sender thread seemed to be a good place to do this. If you mean
> send blocks just before calling sock_shutdown[?] in nbd_thread_send
> then yes I think that should be removed. I need to re-check how nbd-server
> behaves in that case.

No that's not what I meant. Your approach uses the sender thread as a
worker to close the socket. You are using waiting_wq to notify the
sender thread. That's fine so far.

But what happens if the sender thread is at this point trying to send on
the socket which blocks? Then the timeout triggers and waiting_wq will
notify the sending thread as soon as it left the sending routine. But it
will not interrupt the thread that is waiting in kernel_sendmsg() and
the sending thread will be stuck much longer than specified in the
timeout. 

> 
> >
> >>
> >> 2. Make sock lock to be a mutex instead of a spin lock, since
> >>nbd_xmit_timeout doesn't need to hold it anymore.
> >
> > I can't see why we need a mutex instead of a spinlock?
> 
> you are right, with your earlier patch we don't need it to be a mutex.
> 
> >
> >>
> >> Signed-off-by: Pranay Kr. Srivastava 
> >> ---
> >>  drivers/block/nbd.c | 65 
> >> -
> >>  1 file changed, 39 insertions(+), 26 deletions(-)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 31e73a7..c79bcd7 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -57,12 +57,12 @@ struct nbd_device {
> >>   int blksize;
> >>   loff_t bytesize;
> >>   int xmit_timeout;
> >> - bool timedout;
> >> + atomic_t timedout;
> >>   bool disconnect; /* a disconnect has been requested by user */
> >>
> >>   struct timer_list timeout_timer;
> >>   /* protects initialization and shutdown of the socket */
> >> - spinlock_t sock_lock;
> >> + struct mutex sock_lock;
> >>   struct task_struct *task_recv;
> >>   struct task_struct *task_send;
> >>
> >> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> >> struct request *req)
> >>   */
> >>  static void sock_shutdown(struct nbd_device *nbd)
> >>  {
> >> - spin_lock_irq(>sock_lock);
> >> -
> >> + mutex_lock(>sock_lock);
> >>   if (!nbd->sock) {
> >> - spin_unlock_irq(>sock_lock);
> >> + mutex_unlock(>sock_lock);
> >>   return;
> >>   }
> >>
> >> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
> >>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >>   sockfd_put(nbd->sock);
> >>   nbd->sock = NULL;
> >> - spin_unlock_irq(>sock_lock);
> >> -
> >> + mutex_unlock(>sock_lock);
> >>   del_timer(>timeout_timer);
> >>  }
> >>
> >>  static void nbd_xmit_timeout(unsigned long arg)
> >>  {
> >>   struct nbd_device *nbd = (struct nbd_device *)arg;
> >> - unsigned long flags;
> >>
> >>   if (list_empty(>queue_head))
> >>   return;
> >>
> >> - spin_lock_irqsave(>sock_lock, flags);
> >> -
> >> - nbd->timedout = true;
> >> -
> >> - if (nbd->sock)
> >> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> >> -
> >> - spin_unlock_irqrestore(>sock_lock, flags);
> >> + atomic_inc(>timedout);
> >> + wake_up(>waiting_wq);
> >>
> >>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> >> connection\n");
> >>  }
> >> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
> >>   /* wait for something to do */
> >>   wait_event_interruptible(nbd->waiting_wq,
> >> 

Re: [PATCH v4 18/18] make nbd device wait for its users in case of timeout

[Markus Pargmann]

Hi,

On Thu, May 12, 2016 at 08:49:00PM +0530, Pranay Srivastava wrote:
> On Thu, May 12, 2016 at 2:49 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> > Hi,
> >
> > On Wednesday 11 May 2016 11:18:46 Pranay Kr. Srivastava wrote:
> >> When a timeout occurs or a recv fails, then
> >> instead of abruplty killing nbd block device
> >> wait for it's users to finish.
> >>
> >> This is more required when filesystem(s) like
> >> ext2 or ext3 don't expect their buffer heads to
> >> disappear while the filesystem is mounted.
> >>
> >> The change is described below:
> >> a) Add a users count to nbd_device structure.
> >> b) Add a bit flag to nbd_device structure of unsigned long.
> >>
> >> If the current user count is not 1 then make nbd-client wait
> >> for the in_use bit to be cleared.
> >
> > Thanks, I like this approach much more.
> >
> >>
> >> Signed-off-by: Pranay Kr. Srivastava <pran...@gmail.com>
> >> ---
> >>  drivers/block/nbd.c  | 40 
> >>  include/uapi/linux/nbd.h |  1 +
> >>  2 files changed, 41 insertions(+)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 482a3c0..9b024d8 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -59,6 +59,7 @@ struct nbd_device {
> >>   int xmit_timeout;
> >>   atomic_t timedout;
> >>   bool disconnect; /* a disconnect has been requested by user */
> >> + u32 users;
> >
> > Perhaps it is better to use kref for this?
> 
> Yup will do.
> 
> >
> >>
> >>   struct timer_list timeout_timer;
> >>   /* protects initialization and shutdown of the socket */
> >> @@ -69,6 +70,7 @@ struct nbd_device {
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >>   struct dentry *dbg_dir;
> >>  #endif
> >> + unsigned long bflags;   /* word size bit flags for use. */
> >
> > Maybe it is better to use a completion instead of a bitfield.
> 
> Ok.
> 
> >
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -822,6 +824,15 @@ static int __nbd_ioctl(struct block_device *bdev, 
> >> struct nbd_device *nbd,
> >>   sock_shutdown(nbd);
> >>   mutex_lock(>tx_lock);
> >>   nbd_clear_que(nbd);
> >> + /*
> >> +  * Wait for any users currently using
> >> +  * this block device.
> >> +  */
> >> + mutex_unlock(>tx_lock);
> >> + pr_info("Waiting for users to release device %s ...\n",
> >> + bdev->bd_disk->disk_name);
> >> + wait_on_bit(>bflags, NBD_BFLAG_INUSE_BIT, 
> >> TASK_INTERRUPTIBLE);
> >> + mutex_lock(>tx_lock);
> >>   kill_bdev(bdev);
> >>   nbd_bdev_reset(bdev);
> >>
> >> @@ -870,10 +881,39 @@ static int nbd_ioctl(struct block_device *bdev, 
> >> fmode_t mode,
> >>   return error;
> >>  }
> >>
> >> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> >> +{
> >> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> >
> > Here is a new line missing otherwise checkpatch will probably warn about
> > this?
> >
> > Should we check here if we are connected here? And check whether the
> > connection is about to be closed?
> 
> I don't think it should matter if we are connected or not. We already
> handle that case
> correctly. Requests would fail and those failures will be reported to
> userland. The idea
> here is not to enforce "harsh measures" on the user on failure but to
> report them instead.

I see. Yes that's fine.

Thanks,

Markus

> 
> >
> > Best Regards,
> >
> > Markus
> 
> I'll send in a new patch after making changes as per your review.
> 
> Thanks alot!.
> 
> >
> >> + nbd_dev->users++;
> >> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> >> + bdev->bd_disk->disk_name, nbd_dev->users);
> >> + if (nbd_dev->users > 1)
> >> + {
> >> + set_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> >> + }
> >> + return 0;
> >> +}
> >> +
> >>

Re: [PATCH v4 18/18] make nbd device wait for its users in case of timeout

[Markus Pargmann]

Hi,

On Thu, May 12, 2016 at 08:49:00PM +0530, Pranay Srivastava wrote:
> On Thu, May 12, 2016 at 2:49 PM, Markus Pargmann  wrote:
> > Hi,
> >
> > On Wednesday 11 May 2016 11:18:46 Pranay Kr. Srivastava wrote:
> >> When a timeout occurs or a recv fails, then
> >> instead of abruplty killing nbd block device
> >> wait for it's users to finish.
> >>
> >> This is more required when filesystem(s) like
> >> ext2 or ext3 don't expect their buffer heads to
> >> disappear while the filesystem is mounted.
> >>
> >> The change is described below:
> >> a) Add a users count to nbd_device structure.
> >> b) Add a bit flag to nbd_device structure of unsigned long.
> >>
> >> If the current user count is not 1 then make nbd-client wait
> >> for the in_use bit to be cleared.
> >
> > Thanks, I like this approach much more.
> >
> >>
> >> Signed-off-by: Pranay Kr. Srivastava 
> >> ---
> >>  drivers/block/nbd.c  | 40 
> >>  include/uapi/linux/nbd.h |  1 +
> >>  2 files changed, 41 insertions(+)
> >>
> >> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> >> index 482a3c0..9b024d8 100644
> >> --- a/drivers/block/nbd.c
> >> +++ b/drivers/block/nbd.c
> >> @@ -59,6 +59,7 @@ struct nbd_device {
> >>   int xmit_timeout;
> >>   atomic_t timedout;
> >>   bool disconnect; /* a disconnect has been requested by user */
> >> + u32 users;
> >
> > Perhaps it is better to use kref for this?
> 
> Yup will do.
> 
> >
> >>
> >>   struct timer_list timeout_timer;
> >>   /* protects initialization and shutdown of the socket */
> >> @@ -69,6 +70,7 @@ struct nbd_device {
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >>   struct dentry *dbg_dir;
> >>  #endif
> >> + unsigned long bflags;   /* word size bit flags for use. */
> >
> > Maybe it is better to use a completion instead of a bitfield.
> 
> Ok.
> 
> >
> >>  };
> >>
> >>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> >> @@ -822,6 +824,15 @@ static int __nbd_ioctl(struct block_device *bdev, 
> >> struct nbd_device *nbd,
> >>   sock_shutdown(nbd);
> >>   mutex_lock(>tx_lock);
> >>   nbd_clear_que(nbd);
> >> + /*
> >> +  * Wait for any users currently using
> >> +  * this block device.
> >> +  */
> >> + mutex_unlock(>tx_lock);
> >> + pr_info("Waiting for users to release device %s ...\n",
> >> + bdev->bd_disk->disk_name);
> >> + wait_on_bit(>bflags, NBD_BFLAG_INUSE_BIT, 
> >> TASK_INTERRUPTIBLE);
> >> + mutex_lock(>tx_lock);
> >>   kill_bdev(bdev);
> >>   nbd_bdev_reset(bdev);
> >>
> >> @@ -870,10 +881,39 @@ static int nbd_ioctl(struct block_device *bdev, 
> >> fmode_t mode,
> >>   return error;
> >>  }
> >>
> >> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> >> +{
> >> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;
> >
> > Here is a new line missing otherwise checkpatch will probably warn about
> > this?
> >
> > Should we check here if we are connected here? And check whether the
> > connection is about to be closed?
> 
> I don't think it should matter if we are connected or not. We already
> handle that case
> correctly. Requests would fail and those failures will be reported to
> userland. The idea
> here is not to enforce "harsh measures" on the user on failure but to
> report them instead.

I see. Yes that's fine.

Thanks,

Markus

> 
> >
> > Best Regards,
> >
> > Markus
> 
> I'll send in a new patch after making changes as per your review.
> 
> Thanks alot!.
> 
> >
> >> + nbd_dev->users++;
> >> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> >> + bdev->bd_disk->disk_name, nbd_dev->users);
> >> + if (nbd_dev->users > 1)
> >> + {
> >> + set_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> >> + }
> >> + return 0;
> >> +}
> >> +
> >> +static void nbd_release(struct gendisk *disk, f

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi Wouter,

On Sun, May 15, 2016 at 02:55:39PM +0200, Wouter Verhelst wrote:
> Hi Markus,
> 
> On Thu, May 12, 2016 at 11:53:01AM +0200, Markus Pargmann wrote:
> > On Thursday 28 April 2016 18:27:34 Wouter Verhelst wrote:
> > > However, at some point I agreed with Paul (your predecessor) that when
> > > this happens due to an error condition (as opposed to it being due to an
> > > explicit disconnect), the kernel would block all reads from or writes to
> > > the device, and the client may try to reconnect *from the same
> > > PID* (i.e., it may not fork()). If that succeeds, the next NBD_DO_IT is
> > > assumed to be connected to the same server; if instead the process
> > > exits, then the block device is assumed to be dead, will be reset, and
> > > all pending reads or writes would error.
> > > 
> > > In principle, this allows for a proper reconnect from userspace if it
> > > can be done. However, I'm not sure whether this ever worked well or
> > > whether it was documented, so it's probably fine if you think it should
> > > be replaced with something else.
> > 
> > At least I was not aware of this possibility. As far as I know the
> > previous code even had issues with the signals used to kill on timeouts
> > which also killed the userspace program sometimes.
> > 
> > Currently I can't see a code path that supports reconnects. But I may
> > have removed that accidently in the past.
> 
> Right. Like I said, I'm not sure if it ever worked well. The user space
> client has a -persist option that tries to implement it, but I've been
> getting some bug reports from people who've tried it (although that may
> have been my fault rather than the kernel's).
> 
> > > (obviously, userspace reconnecting the device to a different device is
> > > wrong and should not be done, but that's a case of "if you break it, you
> > > get to keep both pieces)
> > > 
> > > At any rate, I think it makes sense for userspace to be given a chance
> > > to *attempt* to reconnect a device when the connection drops
> > > unexpectedly.
> > 
> > Perhaps it would be better to setup the kernel driver explicitly for
> > that. Perhaps some flag to let the kernel driver know that the client
> > would like to keep the block device open? In that case the client could
> > excplicitly use NBD_CLEAR_SOCK to cleanup everything.
> 
> I'm not sure what you mean by this. Can you clarify?

I meant that it might be better to have a separate way for NBD_DO_IT.
Something where the client software can directly instruct the kernel to
keep everything opened in case of an error so that the client may
reconnect afterwards.

This could be a new ioctl that sets it up, for example 'NBD_PERSISTENT'.
The NBD_DO_IT afterwards would keep everything up and running in case of
a connection error so that the client could set a new socket using
NBD_SET_SOCK and reenter using NBD_DO_IT.

For all clients that are not capable of this mechanism or don't use it,
NBD_DO_IT would clean up everything properly on any error.


> 
> > Or perhaps a completely new ioctl that can transmit back some more
> > information about what failures were seen and whether the blockdevice
> > was closed or not?
> 
> The intent was that ioctl(NBD_DO_IT) would return an error when the
> disconnect was not requested, and would return 0 when the connection
> dropped due to userspace doing ioctl(NBD_DISCONNECT), since dropping the
> connection when userspace explicitly asks for it is not an error.
> 
> drivers/block/nbd.c contains the following:
> 
> static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>unsigned int cmd, unsigned long arg)
> {
> [...]
>   case NBD_DO_IT: {
> [...]
> if (nbd->disconnect) /* user requested, ignore socket errors 
> */
> return 0;
> return error;
>   }
> [...]
> 
> so the signalling part of it is at least still there. Whether it works,
> I haven't tested.

I just looked up the kernel code from 4.0. This code was there as
well. But the socket and blockdevice were both destroyed before leaving
the NBD_DO_IT ioctl. So it seems to have never been really persistent.
Filesystems would have still been killed.

So for a persistent nbd device there is some more code necessary to do
it.

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: PGP signature

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi Wouter,

On Sun, May 15, 2016 at 02:55:39PM +0200, Wouter Verhelst wrote:
> Hi Markus,
> 
> On Thu, May 12, 2016 at 11:53:01AM +0200, Markus Pargmann wrote:
> > On Thursday 28 April 2016 18:27:34 Wouter Verhelst wrote:
> > > However, at some point I agreed with Paul (your predecessor) that when
> > > this happens due to an error condition (as opposed to it being due to an
> > > explicit disconnect), the kernel would block all reads from or writes to
> > > the device, and the client may try to reconnect *from the same
> > > PID* (i.e., it may not fork()). If that succeeds, the next NBD_DO_IT is
> > > assumed to be connected to the same server; if instead the process
> > > exits, then the block device is assumed to be dead, will be reset, and
> > > all pending reads or writes would error.
> > > 
> > > In principle, this allows for a proper reconnect from userspace if it
> > > can be done. However, I'm not sure whether this ever worked well or
> > > whether it was documented, so it's probably fine if you think it should
> > > be replaced with something else.
> > 
> > At least I was not aware of this possibility. As far as I know the
> > previous code even had issues with the signals used to kill on timeouts
> > which also killed the userspace program sometimes.
> > 
> > Currently I can't see a code path that supports reconnects. But I may
> > have removed that accidently in the past.
> 
> Right. Like I said, I'm not sure if it ever worked well. The user space
> client has a -persist option that tries to implement it, but I've been
> getting some bug reports from people who've tried it (although that may
> have been my fault rather than the kernel's).
> 
> > > (obviously, userspace reconnecting the device to a different device is
> > > wrong and should not be done, but that's a case of "if you break it, you
> > > get to keep both pieces)
> > > 
> > > At any rate, I think it makes sense for userspace to be given a chance
> > > to *attempt* to reconnect a device when the connection drops
> > > unexpectedly.
> > 
> > Perhaps it would be better to setup the kernel driver explicitly for
> > that. Perhaps some flag to let the kernel driver know that the client
> > would like to keep the block device open? In that case the client could
> > excplicitly use NBD_CLEAR_SOCK to cleanup everything.
> 
> I'm not sure what you mean by this. Can you clarify?

I meant that it might be better to have a separate way for NBD_DO_IT.
Something where the client software can directly instruct the kernel to
keep everything opened in case of an error so that the client may
reconnect afterwards.

This could be a new ioctl that sets it up, for example 'NBD_PERSISTENT'.
The NBD_DO_IT afterwards would keep everything up and running in case of
a connection error so that the client could set a new socket using
NBD_SET_SOCK and reenter using NBD_DO_IT.

For all clients that are not capable of this mechanism or don't use it,
NBD_DO_IT would clean up everything properly on any error.


> 
> > Or perhaps a completely new ioctl that can transmit back some more
> > information about what failures were seen and whether the blockdevice
> > was closed or not?
> 
> The intent was that ioctl(NBD_DO_IT) would return an error when the
> disconnect was not requested, and would return 0 when the connection
> dropped due to userspace doing ioctl(NBD_DISCONNECT), since dropping the
> connection when userspace explicitly asks for it is not an error.
> 
> drivers/block/nbd.c contains the following:
> 
> static int __nbd_ioctl(struct block_device *bdev, struct nbd_device *nbd,
>unsigned int cmd, unsigned long arg)
> {
> [...]
>   case NBD_DO_IT: {
> [...]
> if (nbd->disconnect) /* user requested, ignore socket errors 
> */
> return 0;
> return error;
>   }
> [...]
> 
> so the signalling part of it is at least still there. Whether it works,
> I haven't tested.

I just looked up the kernel code from 4.0. This code was there as
well. But the socket and blockdevice were both destroyed before leaving
the NBD_DO_IT ioctl. So it seems to have never been really persistent.
Filesystems would have still been killed.

So for a persistent nbd device there is some more code necessary to do
it.

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: PGP signature

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

Hi,

On Wed, May 11, 2016 at 11:18:29AM +0300, Pranay Kr. Srivastava wrote:
> This patch fixes the warning generated when a timeout occurs
> on the request and socket is closed from a non-sleep context
> by
> 
> 1. Moving the socket closing on a timeout to nbd_thread_send

What happens if a send blocks?

> 
> 2. Make sock lock to be a mutex instead of a spin lock, since
>nbd_xmit_timeout doesn't need to hold it anymore.

I can't see why we need a mutex instead of a spinlock?

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 65 
> -
>  1 file changed, 39 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..c79bcd7 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -57,12 +57,12 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> - spinlock_t sock_lock;
> + struct mutex sock_lock;
>   struct task_struct *task_recv;
>   struct task_struct *task_send;
>  
> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> + mutex_lock(>sock_lock);
>   if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> + mutex_unlock(>sock_lock);
>   return;
>   }
>  
> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
>   sockfd_put(nbd->sock);
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> -
> + mutex_unlock(>sock_lock);
>   del_timer(>timeout_timer);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
>  
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> + atomic_inc(>timedout);
> + wake_up(>waiting_wq);
>  
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
>kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> +  !list_empty(>waiting_queue) ||
> +  atomic_read(>timedout));
> +
> + if (atomic_read(>timedout)) {
> + mutex_lock(>sock_lock);
> + if (nbd->sock) {
> + struct request sreq;
> +
> + blk_rq_init(NULL, );
> + sreq.cmd_type = REQ_TYPE_DRV_PRIV;
> + mutex_lock(>tx_lock);
> + nbd->disconnect = true;
> + nbd_send_req(nbd, );
> + mutex_unlock(>tx_lock);
> + dev_err(disk_to_dev(nbd->disk),
> + "Device Timeout occured.Shutting down"
> + " socket.");
> + }
> + mutex_unlock(>sock_lock);
> + sock_shutdown(nbd);

Why are you trying to send something on a connection that timed out
(nbd_send_req())? And afterwards you execute a socket shutdown so in most
timeout cases this won't reach the server and we risk a blocking send on
a timedout connection.

Regards,

Markus

> + }
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -592,7 +603,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -647,7 +662,7 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
> socket *sock)
>  {
>   int ret = 0;
>  
> - spin_lock_irq(>sock_lock);
> + mutex_lock(>sock_lock);
>  
>   if (nbd->sock) {
>   ret = -EBUSY;
> @@ -657,7 +672,7 @@ static int 

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

Hi,

On Wed, May 11, 2016 at 11:18:29AM +0300, Pranay Kr. Srivastava wrote:
> This patch fixes the warning generated when a timeout occurs
> on the request and socket is closed from a non-sleep context
> by
> 
> 1. Moving the socket closing on a timeout to nbd_thread_send

What happens if a send blocks?

> 
> 2. Make sock lock to be a mutex instead of a spin lock, since
>nbd_xmit_timeout doesn't need to hold it anymore.

I can't see why we need a mutex instead of a spinlock?

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 65 
> -
>  1 file changed, 39 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..c79bcd7 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -57,12 +57,12 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> - spinlock_t sock_lock;
> + struct mutex sock_lock;
>   struct task_struct *task_recv;
>   struct task_struct *task_send;
>  
> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> + mutex_lock(>sock_lock);
>   if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> + mutex_unlock(>sock_lock);
>   return;
>   }
>  
> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
>   sockfd_put(nbd->sock);
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> -
> + mutex_unlock(>sock_lock);
>   del_timer(>timeout_timer);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
>  
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> + atomic_inc(>timedout);
> + wake_up(>waiting_wq);
>  
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
>kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> +  !list_empty(>waiting_queue) ||
> +  atomic_read(>timedout));
> +
> + if (atomic_read(>timedout)) {
> + mutex_lock(>sock_lock);
> + if (nbd->sock) {
> + struct request sreq;
> +
> + blk_rq_init(NULL, );
> + sreq.cmd_type = REQ_TYPE_DRV_PRIV;
> + mutex_lock(>tx_lock);
> + nbd->disconnect = true;
> + nbd_send_req(nbd, );
> + mutex_unlock(>tx_lock);
> + dev_err(disk_to_dev(nbd->disk),
> + "Device Timeout occured.Shutting down"
> + " socket.");
> + }
> + mutex_unlock(>sock_lock);
> + sock_shutdown(nbd);

Why are you trying to send something on a connection that timed out
(nbd_send_req())? And afterwards you execute a socket shutdown so in most
timeout cases this won't reach the server and we risk a blocking send on
a timedout connection.

Regards,

Markus

> + }
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -592,7 +603,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -647,7 +662,7 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
> socket *sock)
>  {
>   int ret = 0;
>  
> - spin_lock_irq(>sock_lock);
> + mutex_lock(>sock_lock);
>  
>   if (nbd->sock) {
>   ret = -EBUSY;
> @@ -657,7 +672,7 @@ static int 

Re: [PATCH] nbd: Move socket shutdown out of spinlock

[Markus Pargmann]

Hi,

On Thursday 12 May 2016 16:42:31 Pranay Srivastava wrote:
> Hi Markus,
> 
> 
> On Thu, May 12, 2016 at 3:13 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> > spinlocked ranges should be small and not contain calls into huge
> > subfunctions. Fix my mistake and just get the pointer to the socket
> > instead of doing everything with spinlock held.
> >
> > Reported-by: Mikulas Patocka <miku...@twibright.com>
> > Signed-off-by: Markus Pargmann <m...@pengutronix.de>
> > ---
> >  drivers/block/nbd.c | 18 ++
> >  1 file changed, 10 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > index 0b892eed06a0..157bf3da876e 100644
> > --- a/drivers/block/nbd.c
> > +++ b/drivers/block/nbd.c
> > @@ -173,20 +173,22 @@ static void nbd_end_request(struct nbd_device *nbd, 
> > struct request *req)
> >   */
> >  static void sock_shutdown(struct nbd_device *nbd)
> >  {
> > +   struct socket *sock;
> > +
> > spin_lock_irq(>sock_lock);
> > +   sock = nbd->sock;
> > +   nbd->sock = NULL;
> > +   spin_unlock_irq(>sock_lock);
> >
> > -   if (!nbd->sock) {
> > -   spin_unlock_irq(>sock_lock);
> > +   if (!sock)
> > return;
> > -   }
> > +
> > +   del_timer(>timeout_timer);
> >
> > dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> > -   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> > -   sockfd_put(nbd->sock);
> > -   nbd->sock = NULL;
> > -   spin_unlock_irq(>sock_lock);
> > +   kernel_sock_shutdown(sock, SHUT_RDWR);
> > +   sockfd_put(sock);
> >
> > -   del_timer(>timeout_timer);
> >  }
> >
> >  static void nbd_xmit_timeout(unsigned long arg)
> 
> I was concerned about nbd_xmit_timeout as well. There's also a call to
> kernel_sock_shutdown,
> while holding the spin_lock in the timeout. The above is ok for
> sock_shutdown but some kind of change
> is also required in nbd_xmit_timeout as well. My patch addressed both these.

Oh I see thanks. Seems there is some duplicate code in
nbd_xmit_timeout and sock_shutdown. I think nbd_xmit_timeout could
perhaps be simplified?

static void nbd_xmit_timeout(unsigned long arg)
{
struct nbd_device *nbd = (struct nbd_device *)arg;

if (list_empty(>queue_head))
return;

nbd->timedout = true;
dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
connection\n");

sock_shutdown(nbd);
}

Thanks,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] nbd: Move socket shutdown out of spinlock

[Markus Pargmann]

Hi,

On Thursday 12 May 2016 16:42:31 Pranay Srivastava wrote:
> Hi Markus,
> 
> 
> On Thu, May 12, 2016 at 3:13 PM, Markus Pargmann  wrote:
> > spinlocked ranges should be small and not contain calls into huge
> > subfunctions. Fix my mistake and just get the pointer to the socket
> > instead of doing everything with spinlock held.
> >
> > Reported-by: Mikulas Patocka 
> > Signed-off-by: Markus Pargmann 
> > ---
> >  drivers/block/nbd.c | 18 ++
> >  1 file changed, 10 insertions(+), 8 deletions(-)
> >
> > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > index 0b892eed06a0..157bf3da876e 100644
> > --- a/drivers/block/nbd.c
> > +++ b/drivers/block/nbd.c
> > @@ -173,20 +173,22 @@ static void nbd_end_request(struct nbd_device *nbd, 
> > struct request *req)
> >   */
> >  static void sock_shutdown(struct nbd_device *nbd)
> >  {
> > +   struct socket *sock;
> > +
> > spin_lock_irq(>sock_lock);
> > +   sock = nbd->sock;
> > +   nbd->sock = NULL;
> > +   spin_unlock_irq(>sock_lock);
> >
> > -   if (!nbd->sock) {
> > -   spin_unlock_irq(>sock_lock);
> > +   if (!sock)
> > return;
> > -   }
> > +
> > +   del_timer(>timeout_timer);
> >
> > dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
> > -   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> > -   sockfd_put(nbd->sock);
> > -   nbd->sock = NULL;
> > -   spin_unlock_irq(>sock_lock);
> > +   kernel_sock_shutdown(sock, SHUT_RDWR);
> > +   sockfd_put(sock);
> >
> > -   del_timer(>timeout_timer);
> >  }
> >
> >  static void nbd_xmit_timeout(unsigned long arg)
> 
> I was concerned about nbd_xmit_timeout as well. There's also a call to
> kernel_sock_shutdown,
> while holding the spin_lock in the timeout. The above is ok for
> sock_shutdown but some kind of change
> is also required in nbd_xmit_timeout as well. My patch addressed both these.

Oh I see thanks. Seems there is some duplicate code in
nbd_xmit_timeout and sock_shutdown. I think nbd_xmit_timeout could
perhaps be simplified?

static void nbd_xmit_timeout(unsigned long arg)
{
struct nbd_device *nbd = (struct nbd_device *)arg;

if (list_empty(>queue_head))
return;

nbd->timedout = true;
dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
connection\n");

sock_shutdown(nbd);
}

Thanks,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Thursday 28 April 2016 18:27:34 Wouter Verhelst wrote:
> On Thu, Apr 28, 2016 at 11:00:20AM +0200, Markus Pargmann wrote:
> > Hi,
> > 
> > On Saturday 23 April 2016 07:47:21 Ratna Manoj wrote:
> > > Thanks for the review. 
> > > 
> > > Atleast for ext4 this crash happens on a sys_umount() call, timing of
> > > which is not in control of block driver. Block driver cannot force the
> > > filesystems to be unmounted, and the file system does not expect 
> > > buffers to get unmapped under it.
> > 
> > Yes the block driver can't force a clean umount.
> > 
> > > 
> > > Ext4 can be fixed with the this patch:
> > > http://www.spinics.net/lists/linux-ext4/msg51112.html 
> > > It did not make to the kernel. It checks the state of the buffer head
> > > before committing.
> > > 
> > > When we consider diskett/CD as user space thread that called NBD_DO_IT,
> > > this problem is analogous to changing disk with another or the same
> > > disk suddenly when the file system is still mounted. 
> > > 
> > > If we completely kill the block device we would loss some writes when
> > > same thread is reconnected.
> > 
> > I am not so sure about your exact use-case here.
> > 
> > If the NBD_DO_IT thread returns I am considering the connection and
> > block device as dead and disconnected. Securing any data afterwards with
> > a new connection is potentially dangerous as it may be a different
> > server.
> 
> Yes, from the kernel POV you're right.
> 
> However, at some point I agreed with Paul (your predecessor) that when
> this happens due to an error condition (as opposed to it being due to an
> explicit disconnect), the kernel would block all reads from or writes to
> the device, and the client may try to reconnect *from the same
> PID* (i.e., it may not fork()). If that succeeds, the next NBD_DO_IT is
> assumed to be connected to the same server; if instead the process
> exits, then the block device is assumed to be dead, will be reset, and
> all pending reads or writes would error.
> 
> In principle, this allows for a proper reconnect from userspace if it
> can be done. However, I'm not sure whether this ever worked well or
> whether it was documented, so it's probably fine if you think it should
> be replaced with something else.

At least I was not aware of this possibility. As far as I know the
previous code even had issues with the signals used to kill on timeouts
which also killed the userspace program sometimes.

Currently I can't see a code path that supports reconnects. But I may
have removed that accidently in the past.

> 
> (obviously, userspace reconnecting the device to a different device is
> wrong and should not be done, but that's a case of "if you break it, you
> get to keep both pieces)
> 
> At any rate, I think it makes sense for userspace to be given a chance
> to *attempt* to reconnect a device when the connection drops
> unexpectedly.

Perhaps it would be better to setup the kernel driver explicitly for
that. Perhaps some flag to let the kernel driver know that the client
would like to keep the block device open? In that case the client could
excplicitly use NBD_CLEAR_SOCK to cleanup everything.

Or perhaps a completely new ioctl that can transmit back some more
information about what failures were seen and whether the blockdevice
was closed or not?

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [Nbd] [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Thursday 28 April 2016 18:27:34 Wouter Verhelst wrote:
> On Thu, Apr 28, 2016 at 11:00:20AM +0200, Markus Pargmann wrote:
> > Hi,
> > 
> > On Saturday 23 April 2016 07:47:21 Ratna Manoj wrote:
> > > Thanks for the review. 
> > > 
> > > Atleast for ext4 this crash happens on a sys_umount() call, timing of
> > > which is not in control of block driver. Block driver cannot force the
> > > filesystems to be unmounted, and the file system does not expect 
> > > buffers to get unmapped under it.
> > 
> > Yes the block driver can't force a clean umount.
> > 
> > > 
> > > Ext4 can be fixed with the this patch:
> > > http://www.spinics.net/lists/linux-ext4/msg51112.html 
> > > It did not make to the kernel. It checks the state of the buffer head
> > > before committing.
> > > 
> > > When we consider diskett/CD as user space thread that called NBD_DO_IT,
> > > this problem is analogous to changing disk with another or the same
> > > disk suddenly when the file system is still mounted. 
> > > 
> > > If we completely kill the block device we would loss some writes when
> > > same thread is reconnected.
> > 
> > I am not so sure about your exact use-case here.
> > 
> > If the NBD_DO_IT thread returns I am considering the connection and
> > block device as dead and disconnected. Securing any data afterwards with
> > a new connection is potentially dangerous as it may be a different
> > server.
> 
> Yes, from the kernel POV you're right.
> 
> However, at some point I agreed with Paul (your predecessor) that when
> this happens due to an error condition (as opposed to it being due to an
> explicit disconnect), the kernel would block all reads from or writes to
> the device, and the client may try to reconnect *from the same
> PID* (i.e., it may not fork()). If that succeeds, the next NBD_DO_IT is
> assumed to be connected to the same server; if instead the process
> exits, then the block device is assumed to be dead, will be reset, and
> all pending reads or writes would error.
> 
> In principle, this allows for a proper reconnect from userspace if it
> can be done. However, I'm not sure whether this ever worked well or
> whether it was documented, so it's probably fine if you think it should
> be replaced with something else.

At least I was not aware of this possibility. As far as I know the
previous code even had issues with the signals used to kill on timeouts
which also killed the userspace program sometimes.

Currently I can't see a code path that supports reconnects. But I may
have removed that accidently in the past.

> 
> (obviously, userspace reconnecting the device to a different device is
> wrong and should not be done, but that's a case of "if you break it, you
> get to keep both pieces)
> 
> At any rate, I think it makes sense for userspace to be given a chance
> to *attempt* to reconnect a device when the connection drops
> unexpectedly.

Perhaps it would be better to setup the kernel driver explicitly for
that. Perhaps some flag to let the kernel driver know that the client
would like to keep the block device open? In that case the client could
excplicitly use NBD_CLEAR_SOCK to cleanup everything.

Or perhaps a completely new ioctl that can transmit back some more
information about what failures were seen and whether the blockdevice
was closed or not?

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

[PATCH] nbd: Move socket shutdown out of spinlock

[Markus Pargmann]

spinlocked ranges should be small and not contain calls into huge
subfunctions. Fix my mistake and just get the pointer to the socket
instead of doing everything with spinlock held.

Reported-by: Mikulas Patocka <miku...@twibright.com>
Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 0b892eed06a0..157bf3da876e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -173,20 +173,22 @@ static void nbd_end_request(struct nbd_device *nbd, 
struct request *req)
  */
 static void sock_shutdown(struct nbd_device *nbd)
 {
+   struct socket *sock;
+
spin_lock_irq(>sock_lock);
+   sock = nbd->sock;
+   nbd->sock = NULL;
+   spin_unlock_irq(>sock_lock);
 
-   if (!nbd->sock) {
-   spin_unlock_irq(>sock_lock);
+   if (!sock)
return;
-   }
+
+   del_timer(>timeout_timer);
 
dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
-   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
-   sockfd_put(nbd->sock);
-   nbd->sock = NULL;
-   spin_unlock_irq(>sock_lock);
+   kernel_sock_shutdown(sock, SHUT_RDWR);
+   sockfd_put(sock);
 
-   del_timer(>timeout_timer);
 }
 
 static void nbd_xmit_timeout(unsigned long arg)
-- 
2.8.0.rc3

[PATCH] nbd: Move socket shutdown out of spinlock

[Markus Pargmann]

spinlocked ranges should be small and not contain calls into huge
subfunctions. Fix my mistake and just get the pointer to the socket
instead of doing everything with spinlock held.

Reported-by: Mikulas Patocka 
Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 0b892eed06a0..157bf3da876e 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -173,20 +173,22 @@ static void nbd_end_request(struct nbd_device *nbd, 
struct request *req)
  */
 static void sock_shutdown(struct nbd_device *nbd)
 {
+   struct socket *sock;
+
spin_lock_irq(>sock_lock);
+   sock = nbd->sock;
+   nbd->sock = NULL;
+   spin_unlock_irq(>sock_lock);
 
-   if (!nbd->sock) {
-   spin_unlock_irq(>sock_lock);
+   if (!sock)
return;
-   }
+
+   del_timer(>timeout_timer);
 
dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
-   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
-   sockfd_put(nbd->sock);
-   nbd->sock = NULL;
-   spin_unlock_irq(>sock_lock);
+   kernel_sock_shutdown(sock, SHUT_RDWR);
+   sockfd_put(sock);
 
-   del_timer(>timeout_timer);
 }
 
 static void nbd_xmit_timeout(unsigned long arg)
-- 
2.8.0.rc3

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:29 Pranay Kr. Srivastava wrote:
> This patch fixes the warning generated when a timeout occurs
> on the request and socket is closed from a non-sleep context
> by
> 
> 1. Moving the socket closing on a timeout to nbd_thread_send
> 
> 2. Make sock lock to be a mutex instead of a spin lock, since
>nbd_xmit_timeout doesn't need to hold it anymore.

This patch seems quite big and complicated. Isn't the main issue that a
socket shutdown is called from within a spinlock?

When the issue got reported by Mikulas Patocka I created a patch but
forgot to send it, sorry. It is a bit simpler by simpling moving the
socket shutdown out of the spinlock. I will send it as reply. Please
have a look.

Thanks,

Markus

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 65 
> -
>  1 file changed, 39 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..c79bcd7 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -57,12 +57,12 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> - spinlock_t sock_lock;
> + struct mutex sock_lock;
>   struct task_struct *task_recv;
>   struct task_struct *task_send;
>  
> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> + mutex_lock(>sock_lock);
>   if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> + mutex_unlock(>sock_lock);
>   return;
>   }
>  
> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
>   sockfd_put(nbd->sock);
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> -
> + mutex_unlock(>sock_lock);
>   del_timer(>timeout_timer);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
>  
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> + atomic_inc(>timedout);
> + wake_up(>waiting_wq);
>  
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
>kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> +  !list_empty(>waiting_queue) ||
> +  atomic_read(>timedout));
> +
> + if (atomic_read(>timedout)) {
> + mutex_lock(>sock_lock);
> + if (nbd->sock) {
> + struct request sreq;
> +
> + blk_rq_init(NULL, );
> + sreq.cmd_type = REQ_TYPE_DRV_PRIV;
> + mutex_lock(>tx_lock);
> + nbd->disconnect = true;
> + nbd_send_req(nbd, );
> + mutex_unlock(>tx_lock);
> + dev_err(disk_to_dev(nbd->disk),
> + "Device Timeout occured.Shutting down"
> + " socket.");
> + }
> + mutex_unlock(>sock_lock);
> + sock_shutdown(nbd);
> + }
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -592,7 +603,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -647,7 +662,7 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
> socket *sock)
>  {
>   int ret = 0;
>  
> - spin_lock_irq(>sock_lock);
> + mutex_lock(>sock_lock);
>  
>   if (nbd->sock) {
>   ret = -EBUSY;
> @@ -657,7 +672,7 @@ 

Re: [PATCH v4 01/18] nbd: Fix might_sleep warning on xmit timeout

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:29 Pranay Kr. Srivastava wrote:
> This patch fixes the warning generated when a timeout occurs
> on the request and socket is closed from a non-sleep context
> by
> 
> 1. Moving the socket closing on a timeout to nbd_thread_send
> 
> 2. Make sock lock to be a mutex instead of a spin lock, since
>nbd_xmit_timeout doesn't need to hold it anymore.

This patch seems quite big and complicated. Isn't the main issue that a
socket shutdown is called from within a spinlock?

When the issue got reported by Mikulas Patocka I created a patch but
forgot to send it, sorry. It is a bit simpler by simpling moving the
socket shutdown out of the spinlock. I will send it as reply. Please
have a look.

Thanks,

Markus

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 65 
> -
>  1 file changed, 39 insertions(+), 26 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 31e73a7..c79bcd7 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -57,12 +57,12 @@ struct nbd_device {
>   int blksize;
>   loff_t bytesize;
>   int xmit_timeout;
> - bool timedout;
> + atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> - spinlock_t sock_lock;
> + struct mutex sock_lock;
>   struct task_struct *task_recv;
>   struct task_struct *task_send;
>  
> @@ -172,10 +172,9 @@ static void nbd_end_request(struct nbd_device *nbd, 
> struct request *req)
>   */
>  static void sock_shutdown(struct nbd_device *nbd)
>  {
> - spin_lock_irq(>sock_lock);
> -
> + mutex_lock(>sock_lock);
>   if (!nbd->sock) {
> - spin_unlock_irq(>sock_lock);
> + mutex_unlock(>sock_lock);
>   return;
>   }
>  
> @@ -183,27 +182,19 @@ static void sock_shutdown(struct nbd_device *nbd)
>   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
>   sockfd_put(nbd->sock);
>   nbd->sock = NULL;
> - spin_unlock_irq(>sock_lock);
> -
> + mutex_unlock(>sock_lock);
>   del_timer(>timeout_timer);
>  }
>  
>  static void nbd_xmit_timeout(unsigned long arg)
>  {
>   struct nbd_device *nbd = (struct nbd_device *)arg;
> - unsigned long flags;
>  
>   if (list_empty(>queue_head))
>   return;
>  
> - spin_lock_irqsave(>sock_lock, flags);
> -
> - nbd->timedout = true;
> -
> - if (nbd->sock)
> - kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
> -
> - spin_unlock_irqrestore(>sock_lock, flags);
> + atomic_inc(>timedout);
> + wake_up(>waiting_wq);
>  
>   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
> connection\n");
>  }
> @@ -579,7 +570,27 @@ static int nbd_thread_send(void *data)
>   /* wait for something to do */
>   wait_event_interruptible(nbd->waiting_wq,
>kthread_should_stop() ||
> -  !list_empty(>waiting_queue));
> +  !list_empty(>waiting_queue) ||
> +  atomic_read(>timedout));
> +
> + if (atomic_read(>timedout)) {
> + mutex_lock(>sock_lock);
> + if (nbd->sock) {
> + struct request sreq;
> +
> + blk_rq_init(NULL, );
> + sreq.cmd_type = REQ_TYPE_DRV_PRIV;
> + mutex_lock(>tx_lock);
> + nbd->disconnect = true;
> + nbd_send_req(nbd, );
> + mutex_unlock(>tx_lock);
> + dev_err(disk_to_dev(nbd->disk),
> + "Device Timeout occured.Shutting down"
> + " socket.");
> + }
> + mutex_unlock(>sock_lock);
> + sock_shutdown(nbd);
> + }
>  
>   /* extract request */
>   if (list_empty(>waiting_queue))
> @@ -592,7 +603,11 @@ static int nbd_thread_send(void *data)
>   spin_unlock_irq(>queue_lock);
>  
>   /* handle request */
> - nbd_handle_req(nbd, req);
> + if (atomic_read(>timedout)) {
> + req->errors++;
> + nbd_end_request(nbd, req);
> + } else
> + nbd_handle_req(nbd, req);
>   }
>  
>   nbd->task_send = NULL;
> @@ -647,7 +662,7 @@ static int nbd_set_socket(struct nbd_device *nbd, struct 
> socket *sock)
>  {
>   int ret = 0;
>  
> - spin_lock_irq(>sock_lock);
> + mutex_lock(>sock_lock);
>  
>   if (nbd->sock) {
>   ret = -EBUSY;
> @@ -657,7 +672,7 @@ static int 

Re: [Nbd] Fwd: [PATCH v4 02/18] nbd: fix checkpatch trailing space warning.

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 07:46:25 Eric Blake wrote:
> On 05/11/2016 03:38 AM, Pranay Srivastava wrote:
> 
> > 
> > The series contained some checkpatch changes so I had included you as well.
> > 
> >> know why you are sending them to me), but I know I do not accept patches
> >> without any changelog text at all in them, as that's just lazy.
> > 
> > That should be per patch or can appear in a cover letter for the patches?
> 
> Per patch.  However, if it were me, I would not have split into quite so
> many patches.  The mantra is one patch per one fix, but I think it is
> reasonable to state that "silence all checkpatch warnings" counts as one
> fix, rather than 16 separate fixes.  If you DO consolidate the
> checkpatch changes into a single patch, then the commit message body
> should call out a bulleted list of all the changes you are making, as
> well as a justification why it is worth churning the entire file rather
> than just making smaller checkpatch fixes in just the areas that your
> other patches touch.

Yes, I think about this similarly. There is simply no benefit in having
the history full of patches that partly even have the exact same
subject. So please squash them into one and describe your changes in the
commit message.

Thanks,

Markus

> 
> > 
> > Actually I've made more patches in this series after I had sent the
> > earlier ones,
> > but the earlier ones are not changed at all. It's only the addition of
> > newer patches
> > to the series.
> 
> The cover letter is a great place to point out how v4 differs from v3,
> but also to point out which patches are unchanged from v3, to save
> reviewer's time.  So if all you did was add new patches, a cover-letter
> mention of which patches remain unchanged might be helpful.
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [Nbd] Fwd: [PATCH v4 02/18] nbd: fix checkpatch trailing space warning.

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 07:46:25 Eric Blake wrote:
> On 05/11/2016 03:38 AM, Pranay Srivastava wrote:
> 
> > 
> > The series contained some checkpatch changes so I had included you as well.
> > 
> >> know why you are sending them to me), but I know I do not accept patches
> >> without any changelog text at all in them, as that's just lazy.
> > 
> > That should be per patch or can appear in a cover letter for the patches?
> 
> Per patch.  However, if it were me, I would not have split into quite so
> many patches.  The mantra is one patch per one fix, but I think it is
> reasonable to state that "silence all checkpatch warnings" counts as one
> fix, rather than 16 separate fixes.  If you DO consolidate the
> checkpatch changes into a single patch, then the commit message body
> should call out a bulleted list of all the changes you are making, as
> well as a justification why it is worth churning the entire file rather
> than just making smaller checkpatch fixes in just the areas that your
> other patches touch.

Yes, I think about this similarly. There is simply no benefit in having
the history full of patches that partly even have the exact same
subject. So please squash them into one and describe your changes in the
commit message.

Thanks,

Markus

> 
> > 
> > Actually I've made more patches in this series after I had sent the
> > earlier ones,
> > but the earlier ones are not changed at all. It's only the addition of
> > newer patches
> > to the series.
> 
> The cover letter is a great place to point out how v4 differs from v3,
> but also to point out which patches are unchanged from v3, to save
> reviewer's time.  So if all you did was add new patches, a cover-letter
> mention of which patches remain unchanged might be helpful.
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v4 18/18] make nbd device wait for its users in case of timeout

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:46 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> The change is described below:
> a) Add a users count to nbd_device structure.
> b) Add a bit flag to nbd_device structure of unsigned long.
> 
> If the current user count is not 1 then make nbd-client wait
> for the in_use bit to be cleared.

Thanks, I like this approach much more.

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c  | 40 
>  include/uapi/linux/nbd.h |  1 +
>  2 files changed, 41 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 482a3c0..9b024d8 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -59,6 +59,7 @@ struct nbd_device {
>   int xmit_timeout;
>   atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
> + u32 users;

Perhaps it is better to use kref for this?

>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> @@ -69,6 +70,7 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + unsigned long bflags;   /* word size bit flags for use. */

Maybe it is better to use a completion instead of a bitfield.

>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -822,6 +824,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   sock_shutdown(nbd);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
> + /*
> +  * Wait for any users currently using
> +  * this block device.
> +  */
> + mutex_unlock(>tx_lock);
> + pr_info("Waiting for users to release device %s ...\n",
> + bdev->bd_disk->disk_name);
> + wait_on_bit(>bflags, NBD_BFLAG_INUSE_BIT, 
> TASK_INTERRUPTIBLE);
> + mutex_lock(>tx_lock);
>   kill_bdev(bdev);
>   nbd_bdev_reset(bdev);
>  
> @@ -870,10 +881,39 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;

Here is a new line missing otherwise checkpatch will probably warn about
this?

Should we check here if we are connected here? And check whether the
connection is about to be closed?

Best Regards,

Markus

> + nbd_dev->users++;
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name, nbd_dev->users);
> + if (nbd_dev->users > 1)
> + {
> + set_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> + }
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + nbd_dev->users--;
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name, nbd_dev->users);
> + if (nbd_dev->users == 1)
> + {
> + clear_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> + smp_mb();
> + wake_up_bit(_dev->bflags, NBD_BFLAG_INUSE_BIT);
> + }
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> diff --git a/include/uapi/linux/nbd.h b/include/uapi/linux/nbd.h
> index e08e413..8f3d3f0 100644
> --- a/include/uapi/linux/nbd.h
> +++ b/include/uapi/linux/nbd.h
> @@ -44,6 +44,7 @@ enum {
>  /* there is a gap here to match userspace */
>  #define NBD_FLAG_SEND_TRIM(1 << 5) /* send trim/discard */
>  
> +#define NBD_BFLAG_INUSE_BIT  (1) /* bit number for bflags */
>  /* userspace doesn't need the nbd_device structure */
>  
>  /* These are sent over the network in the request/reply magic fields */
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v4 18/18] make nbd device wait for its users in case of timeout

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:46 Pranay Kr. Srivastava wrote:
> When a timeout occurs or a recv fails, then
> instead of abruplty killing nbd block device
> wait for it's users to finish.
> 
> This is more required when filesystem(s) like
> ext2 or ext3 don't expect their buffer heads to
> disappear while the filesystem is mounted.
> 
> The change is described below:
> a) Add a users count to nbd_device structure.
> b) Add a bit flag to nbd_device structure of unsigned long.
> 
> If the current user count is not 1 then make nbd-client wait
> for the in_use bit to be cleared.

Thanks, I like this approach much more.

> 
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c  | 40 
>  include/uapi/linux/nbd.h |  1 +
>  2 files changed, 41 insertions(+)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 482a3c0..9b024d8 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -59,6 +59,7 @@ struct nbd_device {
>   int xmit_timeout;
>   atomic_t timedout;
>   bool disconnect; /* a disconnect has been requested by user */
> + u32 users;

Perhaps it is better to use kref for this?

>  
>   struct timer_list timeout_timer;
>   /* protects initialization and shutdown of the socket */
> @@ -69,6 +70,7 @@ struct nbd_device {
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
>   struct dentry *dbg_dir;
>  #endif
> + unsigned long bflags;   /* word size bit flags for use. */

Maybe it is better to use a completion instead of a bitfield.

>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> @@ -822,6 +824,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   sock_shutdown(nbd);
>   mutex_lock(>tx_lock);
>   nbd_clear_que(nbd);
> + /*
> +  * Wait for any users currently using
> +  * this block device.
> +  */
> + mutex_unlock(>tx_lock);
> + pr_info("Waiting for users to release device %s ...\n",
> + bdev->bd_disk->disk_name);
> + wait_on_bit(>bflags, NBD_BFLAG_INUSE_BIT, 
> TASK_INTERRUPTIBLE);
> + mutex_lock(>tx_lock);
>   kill_bdev(bdev);
>   nbd_bdev_reset(bdev);
>  
> @@ -870,10 +881,39 @@ static int nbd_ioctl(struct block_device *bdev, fmode_t 
> mode,
>   return error;
>  }
>  
> +static int nbd_open(struct block_device *bdev, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = bdev->bd_disk->private_data;

Here is a new line missing otherwise checkpatch will probably warn about
this?

Should we check here if we are connected here? And check whether the
connection is about to be closed?

Best Regards,

Markus

> + nbd_dev->users++;
> + pr_debug("Opening nbd_dev %s. Active users = %u\n",
> + bdev->bd_disk->disk_name, nbd_dev->users);
> + if (nbd_dev->users > 1)
> + {
> + set_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> + }
> + return 0;
> +}
> +
> +static void nbd_release(struct gendisk *disk, fmode_t mode)
> +{
> + struct nbd_device *nbd_dev = disk->private_data;
> + nbd_dev->users--;
> + pr_debug("Closing nbd_dev %s. Active users = %u\n",
> + disk->disk_name, nbd_dev->users);
> + if (nbd_dev->users == 1)
> + {
> + clear_bit(NBD_BFLAG_INUSE_BIT, _dev->bflags);
> + smp_mb();
> + wake_up_bit(_dev->bflags, NBD_BFLAG_INUSE_BIT);
> + }
> +}
> +
>  static const struct block_device_operations nbd_fops = {
>   .owner =THIS_MODULE,
>   .ioctl =nbd_ioctl,
>   .compat_ioctl = nbd_ioctl,
> + .open = nbd_open,
> + .release =  nbd_release
>  };
>  
>  #if IS_ENABLED(CONFIG_DEBUG_FS)
> diff --git a/include/uapi/linux/nbd.h b/include/uapi/linux/nbd.h
> index e08e413..8f3d3f0 100644
> --- a/include/uapi/linux/nbd.h
> +++ b/include/uapi/linux/nbd.h
> @@ -44,6 +44,7 @@ enum {
>  /* there is a gap here to match userspace */
>  #define NBD_FLAG_SEND_TRIM(1 << 5) /* send trim/discard */
>  
> +#define NBD_BFLAG_INUSE_BIT  (1) /* bit number for bflags */
>  /* userspace doesn't need the nbd_device structure */
>  
>  /* These are sent over the network in the request/reply magic fields */
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v4 07/18] nbd: fix checkpatch split string warning.

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:35 Pranay Kr. Srivastava wrote:
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 6a4dc3a..7a5b8ef 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -587,8 +587,7 @@ static int nbd_thread_send(void *data)
>   nbd_send_req(nbd, );
>   mutex_unlock(>tx_lock);
>   dev_err(disk_to_dev(nbd->disk),
> - "Device Timeout occured.Shutting down"
> - " socket.");
> + "Device Timeout occured.Shutting down socket.");

Please keep the indentation to the opening bracket here. Shouldn't this
create a checkpatch warning as well?

Regards,

Markus

>   }
>   mutex_unlock(>sock_lock);
>   sock_shutdown(nbd);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH v4 07/18] nbd: fix checkpatch split string warning.

[Markus Pargmann]

Hi,

On Wednesday 11 May 2016 11:18:35 Pranay Kr. Srivastava wrote:
> Signed-off-by: Pranay Kr. Srivastava 
> ---
>  drivers/block/nbd.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index 6a4dc3a..7a5b8ef 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -587,8 +587,7 @@ static int nbd_thread_send(void *data)
>   nbd_send_req(nbd, );
>   mutex_unlock(>tx_lock);
>   dev_err(disk_to_dev(nbd->disk),
> - "Device Timeout occured.Shutting down"
> - " socket.");
> + "Device Timeout occured.Shutting down socket.");

Please keep the indentation to the opening bracket here. Shouldn't this
create a checkpatch warning as well?

Regards,

Markus

>   }
>   mutex_unlock(>sock_lock);
>   sock_shutdown(nbd);
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Saturday 23 April 2016 07:47:21 Ratna Manoj wrote:
> Thanks for the review. 
> 
> Atleast for ext4 this crash happens on a sys_umount() call, timing of
> which is not in control of block driver. Block driver cannot force the
> filesystems to be unmounted, and the file system does not expect 
> buffers to get unmapped under it.

Yes the block driver can't force a clean umount.

> 
> Ext4 can be fixed with the this patch:
> http://www.spinics.net/lists/linux-ext4/msg51112.html 
> It did not make to the kernel. It checks the state of the buffer head
> before committing.
> 
> When we consider diskett/CD as user space thread that called NBD_DO_IT,
> this problem is analogous to changing disk with another or the same
> disk suddenly when the file system is still mounted. 
> 
> If we completely kill the block device we would loss some writes when
> same thread is reconnected.

I am not so sure about your exact use-case here.

If the NBD_DO_IT thread returns I am considering the connection and
block device as dead and disconnected. Securing any data afterwards with
a new connection is potentially dangerous as it may be a different
server.

> 
> if we do not completely kill or if we only invalidate clean buffers, 
> we will have inconsistency on re-attach with a different thread
> (analogous to replacing disk with different disk suddenly). 

Yes exactly. That's why I suggested that NBD_DO_IT waits until all
blockdevice users are gone. This would avoid any issues with
writing/reading data to a wrong server.

Best Regards,

Markus

> 
> Ratna.
>
> 
> On Wed, Apr 20, 2016 at 4:36 PM, Markus Pargmann <m...@pengutronix.de> wrote:
> 
> > Hi,
> >
> > On Thursday 24 March 2016 07:04:10 Ratna Manoj wrote:
> > > From: Ratna Manoj Bolla <manoj...@gmail.com>
> > >
> > > When a filesystem is mounted on a nbd device and on a disconnect, because
> > > of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
> > > getting destroyed under mounted filesystem.
> > >
> > > After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> > > followed by a sys_umount(),
> > > generic_shutdown_super()->...
> > > ->__sync_blockdev()->...
> > > -> blkdev_writepages()->...
> > > ->do_invalidatepage()->...
> > > -> discard_buffer()   is discarding superblock buffer_head 
> > assumed
> > > to be in mapped state by ext4_commit_super().
> > >
> > >
> > >
> > > Signed-off-by: Ratna Manoj Bolla <manoj...@gmail.com>
> > > ---
> > > This script reproduces both the kernel panic scenarios:
> > >
> > > $ qemu-img create -f qcow2 f.img 1G
> > > $ mkfs.ext4 f.img
> > > $ qemu-nbd -c /dev/nbd0 f.img
> > > $ mount /dev/nbd0 dir
> > > $ killall -KILL qemu-nbd
> > > $ sleep 1
> > > $ ls dir
> > > $ umount dir
> > >
> > > Bug reports:
> > > http://www.kernelhub.org/?p=2=361407
> > > 
> > https://www.mail-archive.com/nbd-general@lists.sourceforge.net/msg02388.html
> >
> > Thanks, please CC nbd-gene...@lists.sourceforge.net,
> > linux-kernel@vger.kernel.org as well.
> >
> > So this patch simply does not cleanup the blockdevice to avoid any
> > errors on the filesystem side. The userspace thread that called
> > NBD_DO_IT will exit immediately before the filesystem decided to release
> > the blockdevice. The nbd driver assumes that the shutdown was done and
> > accepts new clients setting up sockets and so on. Couldn't this lead to
> > a lot of problems?
> >
> > Currently NBD_DO_IT returns when it is save to use the NBD device again.
> > This patch changes this as the blockdevice may still be in use when
> > NBD_DO_IT returns. I think it would be better to delay NBD_DO_IT until
> > everything is cleaned up and all filesystems are closed.
> >
> > Best Regards,
> >
> > Markus
> >
> > >
> > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > > index f6b51d7..6e77b3a 100644
> > > --- a/drivers/block/nbd.c
> > > +++ b/drivers/block/nbd.c
> > > @@ -119,7 +119,8 @@ static const char *nbdcmd_to_ascii(int cmd)
> > >
> > >  static int nbd_size_clear(struct nbd_device *nbd, struct block_device 
> > *bdev)
> > >  {
> > > - bdev->bd_inode->i_size = 0;
> > > + if (bdev->bd_openers <= 1)
> > > + bdev->bd_inode->i_size = 0;
> > >  

Re: [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Saturday 23 April 2016 07:47:21 Ratna Manoj wrote:
> Thanks for the review. 
> 
> Atleast for ext4 this crash happens on a sys_umount() call, timing of
> which is not in control of block driver. Block driver cannot force the
> filesystems to be unmounted, and the file system does not expect 
> buffers to get unmapped under it.

Yes the block driver can't force a clean umount.

> 
> Ext4 can be fixed with the this patch:
> http://www.spinics.net/lists/linux-ext4/msg51112.html 
> It did not make to the kernel. It checks the state of the buffer head
> before committing.
> 
> When we consider diskett/CD as user space thread that called NBD_DO_IT,
> this problem is analogous to changing disk with another or the same
> disk suddenly when the file system is still mounted. 
> 
> If we completely kill the block device we would loss some writes when
> same thread is reconnected.

I am not so sure about your exact use-case here.

If the NBD_DO_IT thread returns I am considering the connection and
block device as dead and disconnected. Securing any data afterwards with
a new connection is potentially dangerous as it may be a different
server.

> 
> if we do not completely kill or if we only invalidate clean buffers, 
> we will have inconsistency on re-attach with a different thread
> (analogous to replacing disk with different disk suddenly). 

Yes exactly. That's why I suggested that NBD_DO_IT waits until all
blockdevice users are gone. This would avoid any issues with
writing/reading data to a wrong server.

Best Regards,

Markus

> 
> Ratna.
>
> 
> On Wed, Apr 20, 2016 at 4:36 PM, Markus Pargmann  wrote:
> 
> > Hi,
> >
> > On Thursday 24 March 2016 07:04:10 Ratna Manoj wrote:
> > > From: Ratna Manoj Bolla 
> > >
> > > When a filesystem is mounted on a nbd device and on a disconnect, because
> > > of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are
> > > getting destroyed under mounted filesystem.
> > >
> > > After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> > > followed by a sys_umount(),
> > > generic_shutdown_super()->...
> > > ->__sync_blockdev()->...
> > > -> blkdev_writepages()->...
> > > ->do_invalidatepage()->...
> > > -> discard_buffer()   is discarding superblock buffer_head 
> > assumed
> > > to be in mapped state by ext4_commit_super().
> > >
> > >
> > >
> > > Signed-off-by: Ratna Manoj Bolla 
> > > ---
> > > This script reproduces both the kernel panic scenarios:
> > >
> > > $ qemu-img create -f qcow2 f.img 1G
> > > $ mkfs.ext4 f.img
> > > $ qemu-nbd -c /dev/nbd0 f.img
> > > $ mount /dev/nbd0 dir
> > > $ killall -KILL qemu-nbd
> > > $ sleep 1
> > > $ ls dir
> > > $ umount dir
> > >
> > > Bug reports:
> > > http://www.kernelhub.org/?p=2=361407
> > > 
> > https://www.mail-archive.com/nbd-general@lists.sourceforge.net/msg02388.html
> >
> > Thanks, please CC nbd-gene...@lists.sourceforge.net,
> > linux-kernel@vger.kernel.org as well.
> >
> > So this patch simply does not cleanup the blockdevice to avoid any
> > errors on the filesystem side. The userspace thread that called
> > NBD_DO_IT will exit immediately before the filesystem decided to release
> > the blockdevice. The nbd driver assumes that the shutdown was done and
> > accepts new clients setting up sockets and so on. Couldn't this lead to
> > a lot of problems?
> >
> > Currently NBD_DO_IT returns when it is save to use the NBD device again.
> > This patch changes this as the blockdevice may still be in use when
> > NBD_DO_IT returns. I think it would be better to delay NBD_DO_IT until
> > everything is cleaned up and all filesystems are closed.
> >
> > Best Regards,
> >
> > Markus
> >
> > >
> > > diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> > > index f6b51d7..6e77b3a 100644
> > > --- a/drivers/block/nbd.c
> > > +++ b/drivers/block/nbd.c
> > > @@ -119,7 +119,8 @@ static const char *nbdcmd_to_ascii(int cmd)
> > >
> > >  static int nbd_size_clear(struct nbd_device *nbd, struct block_device 
> > *bdev)
> > >  {
> > > - bdev->bd_inode->i_size = 0;
> > > + if (bdev->bd_openers <= 1)
> > > + bdev->bd_inode->i_size = 0;
> > >   set_capacity(nbd->disk, 0);
> > >   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE

Re: [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Thursday 24 March 2016 07:04:10 Ratna Manoj wrote:
> From: Ratna Manoj Bolla 
> 
> When a filesystem is mounted on a nbd device and on a disconnect, because 
> of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are 
> getting destroyed under mounted filesystem.
> 
> After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> followed by a sys_umount(),
> generic_shutdown_super()->...
> ->__sync_blockdev()->...
> -> blkdev_writepages()->...
> ->do_invalidatepage()->...
> -> discard_buffer()   is discarding superblock buffer_head assumed
> to be in mapped state by ext4_commit_super().
> 
> 
> 
> Signed-off-by: Ratna Manoj Bolla 
> ---
> This script reproduces both the kernel panic scenarios:
>  
> $ qemu-img create -f qcow2 f.img 1G
> $ mkfs.ext4 f.img
> $ qemu-nbd -c /dev/nbd0 f.img
> $ mount /dev/nbd0 dir
> $ killall -KILL qemu-nbd
> $ sleep 1
> $ ls dir
> $ umount dir
> 
> Bug reports:
> http://www.kernelhub.org/?p=2=361407
> https://www.mail-archive.com/nbd-general@lists.sourceforge.net/msg02388.html

Thanks, please CC nbd-gene...@lists.sourceforge.net,
linux-kernel@vger.kernel.org as well.

So this patch simply does not cleanup the blockdevice to avoid any
errors on the filesystem side. The userspace thread that called
NBD_DO_IT will exit immediately before the filesystem decided to release
the blockdevice. The nbd driver assumes that the shutdown was done and
accepts new clients setting up sockets and so on. Couldn't this lead to
a lot of problems?

Currently NBD_DO_IT returns when it is save to use the NBD device again.
This patch changes this as the blockdevice may still be in use when
NBD_DO_IT returns. I think it would be better to delay NBD_DO_IT until
everything is cleaned up and all filesystems are closed.

Best Regards,

Markus

> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index f6b51d7..6e77b3a 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -119,7 +119,8 @@ static const char *nbdcmd_to_ascii(int cmd)
>  
>  static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
>  {
> - bdev->bd_inode->i_size = 0;
> + if (bdev->bd_openers <= 1)
> + bdev->bd_inode->i_size = 0;
>   set_capacity(nbd->disk, 0);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  
> @@ -678,6 +679,9 @@ static void nbd_reset(struct nbd_device *nbd)
>  
>  static void nbd_bdev_reset(struct block_device *bdev)
>  {
> + if (bdev->bd_openers > 1)
> + return;
> +
>   set_device_ro(bdev, false);
>   bdev->bd_inode->i_size = 0;
>   if (max_part > 0) {
> @@ -735,7 +739,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
> + __invalidate_device(bdev, true);
>   return 0;
>  
>   case NBD_SET_SOCK: {
> @@ -809,7 +813,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>  
>   sock_shutdown(nbd);
>   nbd_clear_que(nbd);
> - kill_bdev(bdev);
> + __invalidate_device(bdev, true);
>   nbd_bdev_reset(bdev);
>  
>   if (nbd->disconnect) /* user requested, ignore socket errors */
> 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] NBD: replace kill_bdev() with __invalidate_device()

[Markus Pargmann]

Hi,

On Thursday 24 March 2016 07:04:10 Ratna Manoj wrote:
> From: Ratna Manoj Bolla 
> 
> When a filesystem is mounted on a nbd device and on a disconnect, because 
> of kill_bdev(), and resetting bdev size to zero, buffer_head mappings are 
> getting destroyed under mounted filesystem.
> 
> After a bdev size reset(i.e bdev->bd_inode->i_size = 0) on a disconnect,
> followed by a sys_umount(),
> generic_shutdown_super()->...
> ->__sync_blockdev()->...
> -> blkdev_writepages()->...
> ->do_invalidatepage()->...
> -> discard_buffer()   is discarding superblock buffer_head assumed
> to be in mapped state by ext4_commit_super().
> 
> 
> 
> Signed-off-by: Ratna Manoj Bolla 
> ---
> This script reproduces both the kernel panic scenarios:
>  
> $ qemu-img create -f qcow2 f.img 1G
> $ mkfs.ext4 f.img
> $ qemu-nbd -c /dev/nbd0 f.img
> $ mount /dev/nbd0 dir
> $ killall -KILL qemu-nbd
> $ sleep 1
> $ ls dir
> $ umount dir
> 
> Bug reports:
> http://www.kernelhub.org/?p=2=361407
> https://www.mail-archive.com/nbd-general@lists.sourceforge.net/msg02388.html

Thanks, please CC nbd-gene...@lists.sourceforge.net,
linux-kernel@vger.kernel.org as well.

So this patch simply does not cleanup the blockdevice to avoid any
errors on the filesystem side. The userspace thread that called
NBD_DO_IT will exit immediately before the filesystem decided to release
the blockdevice. The nbd driver assumes that the shutdown was done and
accepts new clients setting up sockets and so on. Couldn't this lead to
a lot of problems?

Currently NBD_DO_IT returns when it is save to use the NBD device again.
This patch changes this as the blockdevice may still be in use when
NBD_DO_IT returns. I think it would be better to delay NBD_DO_IT until
everything is cleaned up and all filesystems are closed.

Best Regards,

Markus

> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index f6b51d7..6e77b3a 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -119,7 +119,8 @@ static const char *nbdcmd_to_ascii(int cmd)
>  
>  static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
>  {
> - bdev->bd_inode->i_size = 0;
> + if (bdev->bd_openers <= 1)
> + bdev->bd_inode->i_size = 0;
>   set_capacity(nbd->disk, 0);
>   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
>  
> @@ -678,6 +679,9 @@ static void nbd_reset(struct nbd_device *nbd)
>  
>  static void nbd_bdev_reset(struct block_device *bdev)
>  {
> + if (bdev->bd_openers > 1)
> + return;
> +
>   set_device_ro(bdev, false);
>   bdev->bd_inode->i_size = 0;
>   if (max_part > 0) {
> @@ -735,7 +739,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>   nbd_clear_que(nbd);
>   BUG_ON(!list_empty(>queue_head));
>   BUG_ON(!list_empty(>waiting_queue));
> - kill_bdev(bdev);
> + __invalidate_device(bdev, true);
>   return 0;
>  
>   case NBD_SET_SOCK: {
> @@ -809,7 +813,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
> nbd_device *nbd,
>  
>   sock_shutdown(nbd);
>   nbd_clear_que(nbd);
> - kill_bdev(bdev);
> + __invalidate_device(bdev, true);
>   nbd_bdev_reset(bdev);
>  
>   if (nbd->disconnect) /* user requested, ignore socket errors */
> 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: Warning when unloading network block device

[Markus Pargmann]

Hi,

On Friday 15 April 2016 18:30:32 Mikulas Patocka wrote:
> Hi
> 
> The patch 23272a6754b81ff6503e09c743bb4ceeeab39997 (nbd: Remove signal 
> usage) triggers a warning in nbd when the network block device is unloaded 
> - WARN_ON_ONCE(in_irq() || irqs_disabled()) in the function 
> __local_bh_enable_ip. The reason for the warning is that the function 
> kernel_sock_shutdown is called with interrupts disabled.

Thanks for reporting. Will post a patch soon.

Best Regards,

Markus

> 
> block nbd0: NBD_DISCONNECT
> block nbd0: shutting down socket
> [ cut here ]
> WARNING: CPU: 10 PID: 11939 at kernel/softirq.c:150 
> __local_bh_enable_ip+0x66/0x90
> Call Trace:
>  [] ? dump_stack+0x46/0x63
>  [] ? __warn+0xe7/0x100
>  [] ? __local_bh_enable_ip+0x66/0x90
>  [] ? inet_shutdown+0x3f/0x120
>  [] ? sock_shutdown+0x40/0x80 [nbd]
>  [] ? __nbd_ioctl+0x5b8/0xa40 [nbd]
>  [] ? soft_cursor+0x169/0x1e6 [softcursor]
>  [] ? bit_cursor+0x59d/0x5d0 [bitblit]
>  [] ? cursor_timer_handler+0x11/0x40 [fbcon]
>  [] ? up+0xd/0x50
>  [] ? console_unlock+0x216/0x580
>  [] ? remove_wait_queue+0x18/0x60
>  [] ? n_tty_write+0x24a/0x460
>  [] ? security_capable+0x3c/0x50
>  [] ? nbd_ioctl+0x67/0x99 [nbd]
>  [] ? blkdev_ioctl+0x649/0xa70
>  [] ? __vfs_write+0x1e/0xe0
>  [] ? dput+0x42/0x2b0
>  [] ? block_ioctl+0x34/0x40
>  [] ? do_vfs_ioctl+0x96/0x5b0
>  [] ? vfs_write+0x131/0x190
>  [] ? SyS_ioctl+0x91/0xa0
>  [] ? entry_SYSCALL_64_fastpath+0x17/0x93
> ---[ end trace 1956420c2e5f102b ]---
> block nbd0: Receive control failed (result -32)
> 
> Mikulas
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: Warning when unloading network block device

[Markus Pargmann]

Hi,

On Friday 15 April 2016 18:30:32 Mikulas Patocka wrote:
> Hi
> 
> The patch 23272a6754b81ff6503e09c743bb4ceeeab39997 (nbd: Remove signal 
> usage) triggers a warning in nbd when the network block device is unloaded 
> - WARN_ON_ONCE(in_irq() || irqs_disabled()) in the function 
> __local_bh_enable_ip. The reason for the warning is that the function 
> kernel_sock_shutdown is called with interrupts disabled.

Thanks for reporting. Will post a patch soon.

Best Regards,

Markus

> 
> block nbd0: NBD_DISCONNECT
> block nbd0: shutting down socket
> [ cut here ]
> WARNING: CPU: 10 PID: 11939 at kernel/softirq.c:150 
> __local_bh_enable_ip+0x66/0x90
> Call Trace:
>  [] ? dump_stack+0x46/0x63
>  [] ? __warn+0xe7/0x100
>  [] ? __local_bh_enable_ip+0x66/0x90
>  [] ? inet_shutdown+0x3f/0x120
>  [] ? sock_shutdown+0x40/0x80 [nbd]
>  [] ? __nbd_ioctl+0x5b8/0xa40 [nbd]
>  [] ? soft_cursor+0x169/0x1e6 [softcursor]
>  [] ? bit_cursor+0x59d/0x5d0 [bitblit]
>  [] ? cursor_timer_handler+0x11/0x40 [fbcon]
>  [] ? up+0xd/0x50
>  [] ? console_unlock+0x216/0x580
>  [] ? remove_wait_queue+0x18/0x60
>  [] ? n_tty_write+0x24a/0x460
>  [] ? security_capable+0x3c/0x50
>  [] ? nbd_ioctl+0x67/0x99 [nbd]
>  [] ? blkdev_ioctl+0x649/0xa70
>  [] ? __vfs_write+0x1e/0xe0
>  [] ? dput+0x42/0x2b0
>  [] ? block_ioctl+0x34/0x40
>  [] ? do_vfs_ioctl+0x96/0x5b0
>  [] ? vfs_write+0x131/0x190
>  [] ? SyS_ioctl+0x91/0xa0
>  [] ? entry_SYSCALL_64_fastpath+0x17/0x93
> ---[ end trace 1956420c2e5f102b ]---
> block nbd0: Receive control failed (result -32)
> 
> Mikulas
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 1/1] iio: gyro: bmg160: fix buffer read values

[Markus Pargmann]

Hi,

On Monday 28 March 2016 20:15:46 Irina Tirdea wrote:
> When reading gyroscope axes using iio buffers, the values
> returned are always 0. In the interrupt handler, the return
> value of the read operation is returned to the user instead
> of the value read. Return the value read to the user.
> 
> This is also fixed in commit 82d8e5da1a33 ("iio:
> accel: bmg160: optimize transfers in trigger handler").
> 
> Signed-off-by: Irina Tirdea 

Thanks a lot for fixing all the bugs I introduced.

Best Regards,

Markus

> ---
>  drivers/iio/gyro/bmg160_core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/iio/gyro/bmg160_core.c b/drivers/iio/gyro/bmg160_core.c
> index 295cf1d..e165ce9 100644
> --- a/drivers/iio/gyro/bmg160_core.c
> +++ b/drivers/iio/gyro/bmg160_core.c
> @@ -781,7 +781,7 @@ static irqreturn_t bmg160_trigger_handler(int irq, void 
> *p)
>   mutex_unlock(>mutex);
>   goto err;
>   }
> - data->buffer[i++] = ret;
> + data->buffer[i++] = val;
>   }
>   mutex_unlock(>mutex);
>  
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 1/1] iio: gyro: bmg160: fix buffer read values

[Markus Pargmann]

Hi,

On Monday 28 March 2016 20:15:46 Irina Tirdea wrote:
> When reading gyroscope axes using iio buffers, the values
> returned are always 0. In the interrupt handler, the return
> value of the read operation is returned to the user instead
> of the value read. Return the value read to the user.
> 
> This is also fixed in commit 82d8e5da1a33 ("iio:
> accel: bmg160: optimize transfers in trigger handler").
> 
> Signed-off-by: Irina Tirdea 

Thanks a lot for fixing all the bugs I introduced.

Best Regards,

Markus

> ---
>  drivers/iio/gyro/bmg160_core.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/iio/gyro/bmg160_core.c b/drivers/iio/gyro/bmg160_core.c
> index 295cf1d..e165ce9 100644
> --- a/drivers/iio/gyro/bmg160_core.c
> +++ b/drivers/iio/gyro/bmg160_core.c
> @@ -781,7 +781,7 @@ static irqreturn_t bmg160_trigger_handler(int irq, void 
> *p)
>   mutex_unlock(>mutex);
>   goto err;
>   }
> - data->buffer[i++] = ret;
> + data->buffer[i++] = val;
>   }
>   mutex_unlock(>mutex);
>  
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 4/5] gpio: of: Add support to have multiple gpios in gpio-hog

[Markus Pargmann]

On Thursday 10 March 2016 12:37:32 Laxman Dewangan wrote:
> 
> On Wednesday 09 March 2016 10:47 PM, Stephen Warren wrote:
> > On 03/09/2016 06:20 AM, Laxman Dewangan wrote:
> >>
> >> On Wednesday 09 March 2016 11:58 AM, Markus Pargmann wrote:
> >>> * PGP Signed by an unknown key
> >>>
> >>> Hi,
> >>>
> >>> On Tue, Mar 08, 2016 at 05:32:07PM +0530, Laxman Dewangan wrote:
> >>>> The child node for gpio hogs under gpio controller's node
> >>>> provide the mechanism to automatic GPIO request and
> >>>> configuration as part of the gpio-controller's driver
> >>>> probe function.
> >>>>
> >>>> Currently, property "gpio" takes one gpios for such
> >>>> configuration. Add support to have multiple GPIOs in
> >>>> this property so that multiple GPIOs of gpio-controller
> >>>> can be configured by this mechanism with one child node.
> >>> So if I read this correctly you want to have multiple GPIOs with the
> >>> same line name? Why don't you use multiple child nodes with individual
> >>> line names?
> >>>
> >> There is cases on which particular functional configuration needs sets
> >> of GPIO to set. On this case, making sub node for each GPIOs creates
> >> lots of sub-nodes and  add complexity on readability, usability and
> >> maintainability.
> >> Example: for my board, I wanted to set GPIO H2 to input and H0 and H1 to
> >> be output high.
> >> Instead of three nodes, I can have two here:
> >> gpio@0,6000d000 {
> >> wlan_input {
> >> gpio-hog;
> >> gpios = <TEGRA_GPIO(H, 2) 0>;
> >> input;
> >> };
> >>
> >> wlan_output {
> >> gpio-hog;
> >> gpios = <TEGRA_GPIO(H, 0) 0 TEGRA_GPIO(H, 1) 0>;
> >> output-high;
> >> };
> >> };
> > >
> >> So here I am grouping the multiple output GPIO together.
> >>
> >> This looks much similar if we have many GPIOs for one type of
> >> configurations.
> >>
> >> Even it looks better if we have something:
> >> gpio@0,6000d000 {
> >> wlan_control {
> >> gpio-hog;
> >> gpios-input = <TEGRA_GPIO(H, 2) 0>;
> >> gpios-output-high = <TEGRA_GPIO(H, 0) 0
> >> TEGRA_GPIO(H, 1) 0>;
> >> };
> >> };
> >
> > The problem with that is the description used when acquiring the GPIO 
> > is just "wlan_input", "wlan_output", or "wlan_control". There's 
> > nothing to indicate what those individual pins do (perhaps one is a 
> > reset signal, one is a regulator enable, etc.?) By requiring separate 
> > nodes for each GPIO, then the node name can provide a meaningful 
> > semantic name/description for each GPIO, which provides much more 
> > information.
> >
> 
> On this case, we have already property "line-name" and passed the name 
> of the gpio via this property.
> The property names is "line-name" which is good for one string. We can 
> support other property "line-names" with multiple string per GPIO index.
> 
> line-names = "wlan-reset", "wlan-enable";

There is currently a discussion about the future bindings for subnodes in GPIO
controller nodes. Please have a look at these two mail threads:

"Device tree binding documentation for gpio-switch"
"gpio: of: Add support to have multiple gpios in gpio-hog"

Best Regards,

Markus

> 
> 
> > If the approach in this patch is acceptable though, I think you want 
> > to update the description of "gpios" (in the GPIO hog definition 
> > section) in Documentation/devicetree/bindings/gpio/gpio.txt to mention 
> > that multiple GPIO entries are legal. Right now it says that property 
> > much contain exactly #gpio-cells, not a multiple of #gpio-cells.
> 
> I have 5th patch for this and will rearrange series as you suggested on 
> 5th patch.
> 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

Re: [PATCH 4/5] gpio: of: Add support to have multiple gpios in gpio-hog

[Markus Pargmann]

On Thursday 10 March 2016 12:37:32 Laxman Dewangan wrote:
> 
> On Wednesday 09 March 2016 10:47 PM, Stephen Warren wrote:
> > On 03/09/2016 06:20 AM, Laxman Dewangan wrote:
> >>
> >> On Wednesday 09 March 2016 11:58 AM, Markus Pargmann wrote:
> >>> * PGP Signed by an unknown key
> >>>
> >>> Hi,
> >>>
> >>> On Tue, Mar 08, 2016 at 05:32:07PM +0530, Laxman Dewangan wrote:
> >>>> The child node for gpio hogs under gpio controller's node
> >>>> provide the mechanism to automatic GPIO request and
> >>>> configuration as part of the gpio-controller's driver
> >>>> probe function.
> >>>>
> >>>> Currently, property "gpio" takes one gpios for such
> >>>> configuration. Add support to have multiple GPIOs in
> >>>> this property so that multiple GPIOs of gpio-controller
> >>>> can be configured by this mechanism with one child node.
> >>> So if I read this correctly you want to have multiple GPIOs with the
> >>> same line name? Why don't you use multiple child nodes with individual
> >>> line names?
> >>>
> >> There is cases on which particular functional configuration needs sets
> >> of GPIO to set. On this case, making sub node for each GPIOs creates
> >> lots of sub-nodes and  add complexity on readability, usability and
> >> maintainability.
> >> Example: for my board, I wanted to set GPIO H2 to input and H0 and H1 to
> >> be output high.
> >> Instead of three nodes, I can have two here:
> >> gpio@0,6000d000 {
> >> wlan_input {
> >> gpio-hog;
> >> gpios = ;
> >> input;
> >> };
> >>
> >> wlan_output {
> >> gpio-hog;
> >> gpios = ;
> >> output-high;
> >> };
> >> };
> > >
> >> So here I am grouping the multiple output GPIO together.
> >>
> >> This looks much similar if we have many GPIOs for one type of
> >> configurations.
> >>
> >> Even it looks better if we have something:
> >> gpio@0,6000d000 {
> >> wlan_control {
> >> gpio-hog;
> >> gpios-input = ;
> >> gpios-output-high =  >> TEGRA_GPIO(H, 1) 0>;
> >> };
> >> };
> >
> > The problem with that is the description used when acquiring the GPIO 
> > is just "wlan_input", "wlan_output", or "wlan_control". There's 
> > nothing to indicate what those individual pins do (perhaps one is a 
> > reset signal, one is a regulator enable, etc.?) By requiring separate 
> > nodes for each GPIO, then the node name can provide a meaningful 
> > semantic name/description for each GPIO, which provides much more 
> > information.
> >
> 
> On this case, we have already property "line-name" and passed the name 
> of the gpio via this property.
> The property names is "line-name" which is good for one string. We can 
> support other property "line-names" with multiple string per GPIO index.
> 
> line-names = "wlan-reset", "wlan-enable";

There is currently a discussion about the future bindings for subnodes in GPIO
controller nodes. Please have a look at these two mail threads:

"Device tree binding documentation for gpio-switch"
"gpio: of: Add support to have multiple gpios in gpio-hog"

Best Regards,

Markus

> 
> 
> > If the approach in this patch is acceptable though, I think you want 
> > to update the description of "gpios" (in the GPIO hog definition 
> > section) in Documentation/devicetree/bindings/gpio/gpio.txt to mention 
> > that multiple GPIO entries are legal. Right now it says that property 
> > much contain exactly #gpio-cells, not a multiple of #gpio-cells.
> 
> I have 5th patch for this and will rearrange series as you suggested on 
> 5th patch.
> 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |

Re: [PATCH 4/5] gpio: of: Add support to have multiple gpios in gpio-hog

[Markus Pargmann]

Hi,

On Tue, Mar 08, 2016 at 05:32:07PM +0530, Laxman Dewangan wrote:
> The child node for gpio hogs under gpio controller's node
> provide the mechanism to automatic GPIO request and
> configuration as part of the gpio-controller's driver
> probe function.
> 
> Currently, property "gpio" takes one gpios for such
> configuration. Add support to have multiple GPIOs in
> this property so that multiple GPIOs of gpio-controller
> can be configured by this mechanism with one child node.

So if I read this correctly you want to have multiple GPIOs with the
same line name? Why don't you use multiple child nodes with individual
line names?

Best Regards,

Markus

> 
> Signed-off-by: Laxman Dewangan 
> Cc: Benoit Parrot 
> Cc: Alexandre Courbot 
> ---
>  drivers/gpio/gpiolib-of.c | 64 
> ---
>  1 file changed, 49 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
> index d81dbd8..0e4e8fd 100644
> --- a/drivers/gpio/gpiolib-of.c
> +++ b/drivers/gpio/gpiolib-of.c
> @@ -118,6 +118,21 @@ int of_get_named_gpio_flags(struct device_node *np, 
> const char *list_name,
>  }
>  EXPORT_SYMBOL(of_get_named_gpio_flags);
>  
> +static int of_gpio_get_gpio_cells_size(struct device_node *chip_np)
> +{
> + u32 ncells;
> + int ret;
> +
> + ret = of_property_read_u32(chip_np, "#gpio-cells", );
> + if (ret)
> + return ret;
> +
> + if (ncells > MAX_PHANDLE_ARGS)
> + return -EINVAL;
> +
> + return ncells;
> +}
> +
>  /**
>   * of_parse_own_gpio() - Get a GPIO hog descriptor, names and flags for GPIO 
> API
>   * @np:  device node to get GPIO from
> @@ -131,6 +146,7 @@ EXPORT_SYMBOL(of_get_named_gpio_flags);
>   */
>  static struct gpio_desc *of_parse_own_gpio(struct device_node *np,
>  const char **name,
> +int gpio_index,
>  enum gpio_lookup_flags *lflags,
>  enum gpiod_flags *dflags)
>  {
> @@ -139,8 +155,8 @@ static struct gpio_desc *of_parse_own_gpio(struct 
> device_node *np,
>   struct gg_data gg_data = {
>   .flags = _flags,
>   };
> - u32 tmp;
> - int i, ret;
> + int ncells;
> + int i, start_index, ret;
>  
>   chip_np = np->parent;
>   if (!chip_np)
> @@ -150,17 +166,16 @@ static struct gpio_desc *of_parse_own_gpio(struct 
> device_node *np,
>   *lflags = 0;
>   *dflags = 0;
>  
> - ret = of_property_read_u32(chip_np, "#gpio-cells", );
> - if (ret)
> - return ERR_PTR(ret);
> + ncells = of_gpio_get_gpio_cells_size(chip_np);
> + if (ncells < 0)
> + return ERR_PTR(ncells);
>  
> - if (tmp > MAX_PHANDLE_ARGS)
> - return ERR_PTR(-EINVAL);
> + start_index = ncells * gpio_index;
>  
> - gg_data.gpiospec.args_count = tmp;
> + gg_data.gpiospec.args_count = ncells;
>   gg_data.gpiospec.np = chip_np;
> - for (i = 0; i < tmp; i++) {
> - ret = of_property_read_u32_index(np, "gpios", i,
> + for (i = 0; i < ncells; i++) {
> + ret = of_property_read_u32_index(np, "gpios", start_index + i,
>  _data.gpiospec.args[i]);
>   if (ret)
>   return ERR_PTR(ret);
> @@ -211,18 +226,37 @@ static int of_gpiochip_scan_gpios(struct gpio_chip 
> *chip)
>   enum gpio_lookup_flags lflags;
>   enum gpiod_flags dflags;
>   int ret;
> + int i, ncells, ngpios;
> +
> + ncells = of_gpio_get_gpio_cells_size(chip->of_node);
> + if (ncells < 0)
> + return 0;
>  
>   for_each_available_child_of_node(chip->of_node, np) {
>   if (!of_property_read_bool(np, "gpio-hog"))
>   continue;
>  
> - desc = of_parse_own_gpio(np, , , );
> - if (IS_ERR(desc))
> + ngpios = of_property_count_u32_elems(np, "gpios");
> + if (ngpios < 0)
> + continue;
> +
> + if (ngpios % ncells) {
> + dev_warn(chip->parent,
> + "GPIOs entries are not proper in gpios\n");
>   continue;
> + }
> +
> + ngpios /= ncells;
> + for (i = 0; i < ngpios; i++) {
> + desc = of_parse_own_gpio(np, , i,
> +  , );
> + if (IS_ERR(desc))
> + continue;
>  
> - ret = gpiod_hog(desc, name, lflags, dflags);
> - if (ret < 0)
> - return ret;
> + ret = gpiod_hog(desc, name, lflags, dflags);
> + if (ret < 0)
> + return ret;
> + }
>   }
>  
>   

Re: [PATCH 4/5] gpio: of: Add support to have multiple gpios in gpio-hog

[Markus Pargmann]

Hi,

On Tue, Mar 08, 2016 at 05:32:07PM +0530, Laxman Dewangan wrote:
> The child node for gpio hogs under gpio controller's node
> provide the mechanism to automatic GPIO request and
> configuration as part of the gpio-controller's driver
> probe function.
> 
> Currently, property "gpio" takes one gpios for such
> configuration. Add support to have multiple GPIOs in
> this property so that multiple GPIOs of gpio-controller
> can be configured by this mechanism with one child node.

So if I read this correctly you want to have multiple GPIOs with the
same line name? Why don't you use multiple child nodes with individual
line names?

Best Regards,

Markus

> 
> Signed-off-by: Laxman Dewangan 
> Cc: Benoit Parrot 
> Cc: Alexandre Courbot 
> ---
>  drivers/gpio/gpiolib-of.c | 64 
> ---
>  1 file changed, 49 insertions(+), 15 deletions(-)
> 
> diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
> index d81dbd8..0e4e8fd 100644
> --- a/drivers/gpio/gpiolib-of.c
> +++ b/drivers/gpio/gpiolib-of.c
> @@ -118,6 +118,21 @@ int of_get_named_gpio_flags(struct device_node *np, 
> const char *list_name,
>  }
>  EXPORT_SYMBOL(of_get_named_gpio_flags);
>  
> +static int of_gpio_get_gpio_cells_size(struct device_node *chip_np)
> +{
> + u32 ncells;
> + int ret;
> +
> + ret = of_property_read_u32(chip_np, "#gpio-cells", );
> + if (ret)
> + return ret;
> +
> + if (ncells > MAX_PHANDLE_ARGS)
> + return -EINVAL;
> +
> + return ncells;
> +}
> +
>  /**
>   * of_parse_own_gpio() - Get a GPIO hog descriptor, names and flags for GPIO 
> API
>   * @np:  device node to get GPIO from
> @@ -131,6 +146,7 @@ EXPORT_SYMBOL(of_get_named_gpio_flags);
>   */
>  static struct gpio_desc *of_parse_own_gpio(struct device_node *np,
>  const char **name,
> +int gpio_index,
>  enum gpio_lookup_flags *lflags,
>  enum gpiod_flags *dflags)
>  {
> @@ -139,8 +155,8 @@ static struct gpio_desc *of_parse_own_gpio(struct 
> device_node *np,
>   struct gg_data gg_data = {
>   .flags = _flags,
>   };
> - u32 tmp;
> - int i, ret;
> + int ncells;
> + int i, start_index, ret;
>  
>   chip_np = np->parent;
>   if (!chip_np)
> @@ -150,17 +166,16 @@ static struct gpio_desc *of_parse_own_gpio(struct 
> device_node *np,
>   *lflags = 0;
>   *dflags = 0;
>  
> - ret = of_property_read_u32(chip_np, "#gpio-cells", );
> - if (ret)
> - return ERR_PTR(ret);
> + ncells = of_gpio_get_gpio_cells_size(chip_np);
> + if (ncells < 0)
> + return ERR_PTR(ncells);
>  
> - if (tmp > MAX_PHANDLE_ARGS)
> - return ERR_PTR(-EINVAL);
> + start_index = ncells * gpio_index;
>  
> - gg_data.gpiospec.args_count = tmp;
> + gg_data.gpiospec.args_count = ncells;
>   gg_data.gpiospec.np = chip_np;
> - for (i = 0; i < tmp; i++) {
> - ret = of_property_read_u32_index(np, "gpios", i,
> + for (i = 0; i < ncells; i++) {
> + ret = of_property_read_u32_index(np, "gpios", start_index + i,
>  _data.gpiospec.args[i]);
>   if (ret)
>   return ERR_PTR(ret);
> @@ -211,18 +226,37 @@ static int of_gpiochip_scan_gpios(struct gpio_chip 
> *chip)
>   enum gpio_lookup_flags lflags;
>   enum gpiod_flags dflags;
>   int ret;
> + int i, ncells, ngpios;
> +
> + ncells = of_gpio_get_gpio_cells_size(chip->of_node);
> + if (ncells < 0)
> + return 0;
>  
>   for_each_available_child_of_node(chip->of_node, np) {
>   if (!of_property_read_bool(np, "gpio-hog"))
>   continue;
>  
> - desc = of_parse_own_gpio(np, , , );
> - if (IS_ERR(desc))
> + ngpios = of_property_count_u32_elems(np, "gpios");
> + if (ngpios < 0)
> + continue;
> +
> + if (ngpios % ncells) {
> + dev_warn(chip->parent,
> + "GPIOs entries are not proper in gpios\n");
>   continue;
> + }
> +
> + ngpios /= ncells;
> + for (i = 0; i < ngpios; i++) {
> + desc = of_parse_own_gpio(np, , i,
> +  , );
> + if (IS_ERR(desc))
> + continue;
>  
> - ret = gpiod_hog(desc, name, lflags, dflags);
> - if (ret < 0)
> - return ret;
> + ret = gpiod_hog(desc, name, lflags, dflags);
> + if (ret < 0)
> + return ret;
> + }
>   }
>  
>   return 0;
> -- 
> 2.1.4
> 
> --
> To unsubscribe from this list: 

Re: [PATCH] iio: adc: imx25-gcq: fix do_div

[Markus Pargmann]

Hi,

On Saturday 05 March 2016 18:43:11 Jonathan Cameron wrote:
> On 03/03/16 12:51, Sudip Mukherjee wrote:
> > We are getting build failure with tilepro allmodconfig with the error:
> > 
> > drivers/iio/adc/fsl-imx25-gcq.c:236:4: note: in expansion of macro 'do_div'
> > do_div(priv->channel_vref_mv[reg], 1000);
> > ^
> > 
> > include/asm-generic/div64.h:198:17: note: expected 'uint64_t *
> > {aka long long unsigned int *}' but argument is of type 'u32 *
> > {aka unsigned int *}'
> > 
> > Create a temporary variable of type u64 and use that in do_div.
> > 
> > Signed-off-by: Sudip Mukherjee 
> Markus, can you take a quick look at this. 

Thanks. I think this was already fixed by Arnd.
"iio: adc/imx25-gcq: move incorrect do_div"

Best Regards,

Markus

> 
> > ---
> > 
> > tilepro allmodconfig build log is at:
> > https://travis-ci.org/sudipm-mukherjee/parport/jobs/113325889
> > 
> >  drivers/iio/adc/fsl-imx25-gcq.c | 5 -
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/iio/adc/fsl-imx25-gcq.c 
> > b/drivers/iio/adc/fsl-imx25-gcq.c
> > index 2fd1927..e0636d4 100644
> > --- a/drivers/iio/adc/fsl-imx25-gcq.c
> > +++ b/drivers/iio/adc/fsl-imx25-gcq.c
> > @@ -174,6 +174,7 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> > *pdev,
> > struct device *dev = >dev;
> > unsigned int refp_used[4] = {};
> > int ret, i;
> > +   u64 temp;
> >  
> > /*
> >  * Setup all configurations registers with a default conversion
> > @@ -233,7 +234,9 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> > *pdev,
> > priv->channel_vref_mv[reg] =
> > regulator_get_voltage(priv->vref[refp]);
> > /* Conversion from uV to mV */
> > -   do_div(priv->channel_vref_mv[reg], 1000);
> > +   temp = priv->channel_vref_mv[reg];
> > +   do_div(temp, 1000);
> > +   priv->channel_vref_mv[reg] = temp;
> > break;
> > case MX25_ADC_REFP_INT:
> > priv->channel_vref_mv[reg] = 2500;
> > 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] iio: adc: imx25-gcq: fix do_div

[Markus Pargmann]

Hi,

On Saturday 05 March 2016 18:43:11 Jonathan Cameron wrote:
> On 03/03/16 12:51, Sudip Mukherjee wrote:
> > We are getting build failure with tilepro allmodconfig with the error:
> > 
> > drivers/iio/adc/fsl-imx25-gcq.c:236:4: note: in expansion of macro 'do_div'
> > do_div(priv->channel_vref_mv[reg], 1000);
> > ^
> > 
> > include/asm-generic/div64.h:198:17: note: expected 'uint64_t *
> > {aka long long unsigned int *}' but argument is of type 'u32 *
> > {aka unsigned int *}'
> > 
> > Create a temporary variable of type u64 and use that in do_div.
> > 
> > Signed-off-by: Sudip Mukherjee 
> Markus, can you take a quick look at this. 

Thanks. I think this was already fixed by Arnd.
"iio: adc/imx25-gcq: move incorrect do_div"

Best Regards,

Markus

> 
> > ---
> > 
> > tilepro allmodconfig build log is at:
> > https://travis-ci.org/sudipm-mukherjee/parport/jobs/113325889
> > 
> >  drivers/iio/adc/fsl-imx25-gcq.c | 5 -
> >  1 file changed, 4 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/iio/adc/fsl-imx25-gcq.c 
> > b/drivers/iio/adc/fsl-imx25-gcq.c
> > index 2fd1927..e0636d4 100644
> > --- a/drivers/iio/adc/fsl-imx25-gcq.c
> > +++ b/drivers/iio/adc/fsl-imx25-gcq.c
> > @@ -174,6 +174,7 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> > *pdev,
> > struct device *dev = >dev;
> > unsigned int refp_used[4] = {};
> > int ret, i;
> > +   u64 temp;
> >  
> > /*
> >  * Setup all configurations registers with a default conversion
> > @@ -233,7 +234,9 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> > *pdev,
> > priv->channel_vref_mv[reg] =
> > regulator_get_voltage(priv->vref[refp]);
> > /* Conversion from uV to mV */
> > -   do_div(priv->channel_vref_mv[reg], 1000);
> > +   temp = priv->channel_vref_mv[reg];
> > +   do_div(temp, 1000);
> > +   priv->channel_vref_mv[reg] = temp;
> > break;
> > case MX25_ADC_REFP_INT:
> > priv->channel_vref_mv[reg] = 2500;
> > 
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH 1/3] Device tree binding documentation for gpio-switch

[Markus Pargmann]

On Wednesday 02 March 2016 10:03:27 Rob Herring wrote:
> Reviving this thread...
> 
> On Tue, Dec 15, 2015 at 3:09 AM, Markus Pargmann <m...@pengutronix.de> wrote:
> > Hi,
> >
> > On Monday 14 December 2015 09:45:48 Rob Herring wrote:
> >> On Mon, Dec 14, 2015 at 8:28 AM, Linus Walleij <linus.wall...@linaro.org> 
> >> wrote:
> >> > On Fri, Dec 11, 2015 at 3:06 PM, Rob Herring <robh...@kernel.org> wrote:
> >> >> On Fri, Dec 11, 2015 at 6:39 AM, Linus Walleij 
> >> >> <linus.wall...@linaro.org> wrote:
> >> >>> On Fri, Dec 4, 2015 at 6:31 PM, Martyn Welch
> >> >>> <martyn.we...@collabora.co.uk> wrote:
> >>
> >> [...]
> >>
> >> >>> Markus Pargmann also did a series that add initial values to
> >> >>> hogs, which is the inverse usecase of this, where you want to
> >> >>> *output* something by default, then maybe also make it available
> >> >>> to userspace.
> >> >>>
> >> >>> So what we need to see here is a patch series that does all of these
> >> >>> things:
> >> >>>
> >> >>> - Name lines
> >> >>>
> >> >>> - Sets them to initial values
> >> >>>
> >> >>> - Mark them as read-only
> >> >>>
> >> >>> - Mark them as "not used by the operating system" so that they
> >> >>>   can be default-exported to userspace.
> >> >>
> >> >> No! This should not be a DT property.
> >> >>
> >> >> Whether I want to control a GPIO in the kernel or userspace is not
> >> >> known and can change over time. It could simply depend on kernel
> >> >> config. There is also the case that a GPIO has no connection or kernel
> >> >> driver until some time later when a DT overlay for an expansion board
> >> >> is applied.
> >> >
> >> > That's correct. So from a DT point of view, what really matters is
> >> > to give things a name, and the hogs and initvals syntax already
> >> > has a structure for this that is in use
> >> > (from Documentation/devicetree/bindings/gpio/gpio.txt):
> >> >
> >> > qe_pio_a: gpio-controller@1400 {
> >> > compatible = "fsl,qe-pario-bank-a", "fsl,qe-pario-bank";
> >> > reg = <0x1400 0x18>;
> >> > gpio-controller;
> >> > #gpio-cells = <2>;
> >> >
> >> > line_b {
> >> > gpio-hog;
> >> > gpios = <6 0>;
> >> > output-low;
> >> > line-name = "foo-bar-gpio";
> >> > };
> >> > };
> >> >
> >> > Markus use this also for initial values. That could easily be used in
> >> > a budget version like this:
> >> >
> >> > line_b {
> >> > /* Just naming */
> >> > gpios = <6 0>;
> >> > line-name = "foo-bar-gpio";
> >> > };
> >> >
> >> > That could grow kind of big though. Or maybe not? How many
> >> > GPIO lines are actually properly named in a typical system?
> >>
> >> We should limit it to GPIOs with no connection to another node. For
> >> example, I don't want to see a SD card detect in the list as that
> >> should be in the SD host node. However, that is hard to enforce and
> >> can change over time. Removing it would break userspace potentially.
> >> Of course if the kernel starts own a signal that userspace used, then
> >> that potentially breaks userspace regardless of the DT changing. OTOH,
> >> using GPIOs in userspace is kind of use at your own risk.
> >
> > I see this a bit differently. I would really like to see the each GPIO 
> > having
> > two different names:
> 
> I think we are saying the same thing...
> 
> > - GPIO label: This is the name of the GPIO line in the schematic for example
> 
> Yes.
> 
> > - GPIO use (this is the current semantic of the GPIO name): The use of a 
> > GPIO,
> >   e.g. 'sd-card-detect', 'LED', ...
> 
> This should be determined from the c

Re: [PATCH 1/3] Device tree binding documentation for gpio-switch

[Markus Pargmann]

On Wednesday 02 March 2016 10:03:27 Rob Herring wrote:
> Reviving this thread...
> 
> On Tue, Dec 15, 2015 at 3:09 AM, Markus Pargmann  wrote:
> > Hi,
> >
> > On Monday 14 December 2015 09:45:48 Rob Herring wrote:
> >> On Mon, Dec 14, 2015 at 8:28 AM, Linus Walleij  
> >> wrote:
> >> > On Fri, Dec 11, 2015 at 3:06 PM, Rob Herring  wrote:
> >> >> On Fri, Dec 11, 2015 at 6:39 AM, Linus Walleij 
> >> >>  wrote:
> >> >>> On Fri, Dec 4, 2015 at 6:31 PM, Martyn Welch
> >> >>>  wrote:
> >>
> >> [...]
> >>
> >> >>> Markus Pargmann also did a series that add initial values to
> >> >>> hogs, which is the inverse usecase of this, where you want to
> >> >>> *output* something by default, then maybe also make it available
> >> >>> to userspace.
> >> >>>
> >> >>> So what we need to see here is a patch series that does all of these
> >> >>> things:
> >> >>>
> >> >>> - Name lines
> >> >>>
> >> >>> - Sets them to initial values
> >> >>>
> >> >>> - Mark them as read-only
> >> >>>
> >> >>> - Mark them as "not used by the operating system" so that they
> >> >>>   can be default-exported to userspace.
> >> >>
> >> >> No! This should not be a DT property.
> >> >>
> >> >> Whether I want to control a GPIO in the kernel or userspace is not
> >> >> known and can change over time. It could simply depend on kernel
> >> >> config. There is also the case that a GPIO has no connection or kernel
> >> >> driver until some time later when a DT overlay for an expansion board
> >> >> is applied.
> >> >
> >> > That's correct. So from a DT point of view, what really matters is
> >> > to give things a name, and the hogs and initvals syntax already
> >> > has a structure for this that is in use
> >> > (from Documentation/devicetree/bindings/gpio/gpio.txt):
> >> >
> >> > qe_pio_a: gpio-controller@1400 {
> >> > compatible = "fsl,qe-pario-bank-a", "fsl,qe-pario-bank";
> >> > reg = <0x1400 0x18>;
> >> > gpio-controller;
> >> > #gpio-cells = <2>;
> >> >
> >> > line_b {
> >> > gpio-hog;
> >> > gpios = <6 0>;
> >> > output-low;
> >> > line-name = "foo-bar-gpio";
> >> > };
> >> > };
> >> >
> >> > Markus use this also for initial values. That could easily be used in
> >> > a budget version like this:
> >> >
> >> > line_b {
> >> > /* Just naming */
> >> > gpios = <6 0>;
> >> > line-name = "foo-bar-gpio";
> >> > };
> >> >
> >> > That could grow kind of big though. Or maybe not? How many
> >> > GPIO lines are actually properly named in a typical system?
> >>
> >> We should limit it to GPIOs with no connection to another node. For
> >> example, I don't want to see a SD card detect in the list as that
> >> should be in the SD host node. However, that is hard to enforce and
> >> can change over time. Removing it would break userspace potentially.
> >> Of course if the kernel starts own a signal that userspace used, then
> >> that potentially breaks userspace regardless of the DT changing. OTOH,
> >> using GPIOs in userspace is kind of use at your own risk.
> >
> > I see this a bit differently. I would really like to see the each GPIO 
> > having
> > two different names:
> 
> I think we are saying the same thing...
> 
> > - GPIO label: This is the name of the GPIO line in the schematic for example
> 
> Yes.
> 
> > - GPIO use (this is the current semantic of the GPIO name): The use of a 
> > GPIO,
> >   e.g. 'sd-card-detect', 'LED', ...
> 
> This should be determined from the compatible string and/or -gpios
> prefix. This is the what the function is and "label" is which one.
> 
> > I think it would be good t

Re: [PULL] NBD for 4.6

[Markus Pargmann]

Hi,

On Thursday, March 03, 2016 06:53:28 AM Jens Axboe wrote:
> On 03/03/2016 12:48 AM, Markus Pargmann wrote:
> > Hi Jens,
> >
> > On Sunday, February 21, 2016 03:01:20 PM Markus Pargmann wrote:
> >> Hi Jens,
> >>
> >> This pull request contains 7 patches for 4.6.
> >
> > any news on this pull request?
> 
> Sorry for the delay, pulled in, thanks.

Thanks, I was just nervous to miss the merge window :).

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PULL] NBD for 4.6

[Markus Pargmann]

Hi,

On Thursday, March 03, 2016 06:53:28 AM Jens Axboe wrote:
> On 03/03/2016 12:48 AM, Markus Pargmann wrote:
> > Hi Jens,
> >
> > On Sunday, February 21, 2016 03:01:20 PM Markus Pargmann wrote:
> >> Hi Jens,
> >>
> >> This pull request contains 7 patches for 4.6.
> >
> > any news on this pull request?
> 
> Sorry for the delay, pulled in, thanks.

Thanks, I was just nervous to miss the merge window :).

Best Regards,

Markus

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PULL] NBD for 4.6

[Markus Pargmann]

Hi Jens,

On Sunday, February 21, 2016 03:01:20 PM Markus Pargmann wrote:
> Hi Jens,
> 
> This pull request contains 7 patches for 4.6.

any news on this pull request?

Best Regards,

Markus

> 
> Patch 1 fixes some unnecessarily complicated code I introduced some versions
> ago for debugfs.
> 
> Patch 2 removes the criticised signal usage within NBD to kill the NBD threads
> after a timeout. This code was used for the last years and is now replaced by
> simply killing the tcp connection.
> 
> Patches 3-6 are some smaller cleanups.
> 
> Patch 7 uevents for the userspace. This way udev/systemd can react on 
> connected
> NBD devices.
> 
> Best Regards,
> 
> Markus
> 
> 
> 
> The following changes since commit 92e963f50fc74041b5e9e744c330dca48e04f08d:
> 
>   Linux 4.5-rc1 (2016-01-24 13:06:47 -0800)
> 
> are available in the git repository at:
> 
>   git://git.pengutronix.de/git/mpa/linux-nbd.git tags/nbd-for-4.6
> 
> for you to fetch changes up to 37091fdd831f28a6509008542174ed324dd645bc:
> 
>   nbd: Create size change events for userspace (2016-02-15 10:35:47 +0100)
> 
> 
> NBD for 4.6
> 
> --------
> Dan Streetman (1):
>   nbd: ratelimit error msgs after socket close
> 
> Markus Pargmann (6):
>   nbd: Fix debugfs error handling
>   nbd: Remove signal usage
>   nbd: Timeouts are not user requested disconnects
>   nbd: Cleanup reset of nbd and bdev after a disconnect
>   nbd: Move flag parsing to a function
>   nbd: Create size change events for userspace
> 
>  drivers/block/nbd.c | 335 
> ++--
>  1 file changed, 170 insertions(+), 165 deletions(-)
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PULL] NBD for 4.6

[Markus Pargmann]

Hi Jens,

On Sunday, February 21, 2016 03:01:20 PM Markus Pargmann wrote:
> Hi Jens,
> 
> This pull request contains 7 patches for 4.6.

any news on this pull request?

Best Regards,

Markus

> 
> Patch 1 fixes some unnecessarily complicated code I introduced some versions
> ago for debugfs.
> 
> Patch 2 removes the criticised signal usage within NBD to kill the NBD threads
> after a timeout. This code was used for the last years and is now replaced by
> simply killing the tcp connection.
> 
> Patches 3-6 are some smaller cleanups.
> 
> Patch 7 uevents for the userspace. This way udev/systemd can react on 
> connected
> NBD devices.
> 
> Best Regards,
> 
> Markus
> 
> 
> 
> The following changes since commit 92e963f50fc74041b5e9e744c330dca48e04f08d:
> 
>   Linux 4.5-rc1 (2016-01-24 13:06:47 -0800)
> 
> are available in the git repository at:
> 
>   git://git.pengutronix.de/git/mpa/linux-nbd.git tags/nbd-for-4.6
> 
> for you to fetch changes up to 37091fdd831f28a6509008542174ed324dd645bc:
> 
>   nbd: Create size change events for userspace (2016-02-15 10:35:47 +0100)
> 
> 
> NBD for 4.6
> 
> --------
> Dan Streetman (1):
>   nbd: ratelimit error msgs after socket close
> 
> Markus Pargmann (6):
>   nbd: Fix debugfs error handling
>   nbd: Remove signal usage
>   nbd: Timeouts are not user requested disconnects
>   nbd: Cleanup reset of nbd and bdev after a disconnect
>   nbd: Move flag parsing to a function
>   nbd: Create size change events for userspace
> 
>  drivers/block/nbd.c | 335 
> ++--
>  1 file changed, 170 insertions(+), 165 deletions(-)
> 
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [Nbd] [PATCH] nbd: ratelimit error messages from disconnected devices

[Markus Pargmann]

Hi,

On Tuesday 16 February 2016 14:33:01 Dmitry Monakhov wrote:
> ndb can be explicitly disconnected via NBD_DISCONNECT while
> active user still exists this result in massive spam to logs.
> Let's ratelimits such messages.
> 
> Signed-off-by: Dmitry Monakhov 
> ---
>  drivers/block/nbd.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index e4c5cc1..2dbe840 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -175,7 +175,7 @@ static int sock_xmit(struct nbd_device *nbd, int send, 
> void *buf, int size,
>   unsigned long pflags = current->flags;
>  
>   if (unlikely(!sock)) {
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted %s on closed socket in sock_xmit\n",

Please indent to the first opening bracket.

>   (send ? "send" : "recv"));
>   return -EINVAL;
> @@ -509,7 +509,7 @@ static void nbd_handle_req(struct nbd_device *nbd, struct 
> request *req)
>   mutex_lock(>tx_lock);
>   if (unlikely(!nbd->sock)) {
>   mutex_unlock(>tx_lock);
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted send on closed socket\n");
>   goto error_out;
>   }
> @@ -618,7 +618,7 @@ static void nbd_request_handler(struct request_queue *q)
>   req, req->cmd_type);
>  
>   if (unlikely(!nbd->sock)) {
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted send on closed socket\n");

This conflicts with a patch already applied fixing the same.
Please rebase onto nbd master [1]

Best Regards,

Markus

[1] 
http://git.pengutronix.de/?p=mpa/linux-nbd.git;a=shortlog;h=refs/heads/master


signature.asc
Description: This is a digitally signed message part.

Re: [Nbd] [PATCH] nbd: ratelimit error messages from disconnected devices

[Markus Pargmann]

Hi,

On Tuesday 16 February 2016 14:33:01 Dmitry Monakhov wrote:
> ndb can be explicitly disconnected via NBD_DISCONNECT while
> active user still exists this result in massive spam to logs.
> Let's ratelimits such messages.
> 
> Signed-off-by: Dmitry Monakhov 
> ---
>  drivers/block/nbd.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
> index e4c5cc1..2dbe840 100644
> --- a/drivers/block/nbd.c
> +++ b/drivers/block/nbd.c
> @@ -175,7 +175,7 @@ static int sock_xmit(struct nbd_device *nbd, int send, 
> void *buf, int size,
>   unsigned long pflags = current->flags;
>  
>   if (unlikely(!sock)) {
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted %s on closed socket in sock_xmit\n",

Please indent to the first opening bracket.

>   (send ? "send" : "recv"));
>   return -EINVAL;
> @@ -509,7 +509,7 @@ static void nbd_handle_req(struct nbd_device *nbd, struct 
> request *req)
>   mutex_lock(>tx_lock);
>   if (unlikely(!nbd->sock)) {
>   mutex_unlock(>tx_lock);
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted send on closed socket\n");
>   goto error_out;
>   }
> @@ -618,7 +618,7 @@ static void nbd_request_handler(struct request_queue *q)
>   req, req->cmd_type);
>  
>   if (unlikely(!nbd->sock)) {
> - dev_err(disk_to_dev(nbd->disk),
> + dev_err_ratelimited(disk_to_dev(nbd->disk),
>   "Attempted send on closed socket\n");

This conflicts with a patch already applied fixing the same.
Please rebase onto nbd master [1]

Best Regards,

Markus

[1] 
http://git.pengutronix.de/?p=mpa/linux-nbd.git;a=shortlog;h=refs/heads/master


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] mfd: fsl-imx25-tsadc: select IRQ_DOMAIN

[Markus Pargmann]

On Tuesday 16 February 2016 15:54:24 Arnd Bergmann wrote:
> The newly added tsadc code use irq domains, but it can be enabled
> when they are disabled in Kconfig:
> 
> drivers/mfd/fsl-imx25-tsadc.c:41:22: error: implicit declaration of function 
> 'irq_find_mapping' [-Werror=implicit-function-declaration]
>generic_handle_irq(irq_find_mapping(tsadc->domain, 1));
> drivers/mfd/fsl-imx25-tsadc.c: In function 'mx25_tsadc_setup_irq':
> drivers/mfd/fsl-imx25-tsadc.c:80:18: error: implicit declaration of function 
> 'irq_domain_add_simple' [-Werror=implicit-function-declaration]
> 
> This adds a 'select IRQ_DOMAIN' code, like we have it in all other
> drivers with this requirement.
> 
> Signed-off-by: Arnd Bergmann <a...@arndb.de>
> Fixes: e2fccf5c1515 ("mfd: fsl-imx25-tsadc: Register touchscreen ADC driver")

Thanks,

Reviewed-by: Markus Pargmann <m...@pengutronix.de>

Best Regards,

Markus

> ---
>  drivers/mfd/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
> index aa21dc55eb15..8cab3ecca8fc 100644
> --- a/drivers/mfd/Kconfig
> +++ b/drivers/mfd/Kconfig
> @@ -300,6 +300,7 @@ config MFD_MC13XXX_I2C
>  config MFD_MX25_TSADC
>   tristate "Freescale i.MX25 integrated Touchscreen and ADC unit"
>   select REGMAP_MMIO
> + select IRQ_DOMAIN
>   depends on (SOC_IMX25 && OF) || COMPILE_TEST
>   help
> Enable support for the integrated Touchscreen and ADC unit of the
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] mfd: fsl-imx25-tsadc: select IRQ_DOMAIN

[Markus Pargmann]

On Tuesday 16 February 2016 15:54:24 Arnd Bergmann wrote:
> The newly added tsadc code use irq domains, but it can be enabled
> when they are disabled in Kconfig:
> 
> drivers/mfd/fsl-imx25-tsadc.c:41:22: error: implicit declaration of function 
> 'irq_find_mapping' [-Werror=implicit-function-declaration]
>generic_handle_irq(irq_find_mapping(tsadc->domain, 1));
> drivers/mfd/fsl-imx25-tsadc.c: In function 'mx25_tsadc_setup_irq':
> drivers/mfd/fsl-imx25-tsadc.c:80:18: error: implicit declaration of function 
> 'irq_domain_add_simple' [-Werror=implicit-function-declaration]
> 
> This adds a 'select IRQ_DOMAIN' code, like we have it in all other
> drivers with this requirement.
> 
> Signed-off-by: Arnd Bergmann 
> Fixes: e2fccf5c1515 ("mfd: fsl-imx25-tsadc: Register touchscreen ADC driver")

Thanks,

Reviewed-by: Markus Pargmann 

Best Regards,

Markus

> ---
>  drivers/mfd/Kconfig | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/drivers/mfd/Kconfig b/drivers/mfd/Kconfig
> index aa21dc55eb15..8cab3ecca8fc 100644
> --- a/drivers/mfd/Kconfig
> +++ b/drivers/mfd/Kconfig
> @@ -300,6 +300,7 @@ config MFD_MC13XXX_I2C
>  config MFD_MX25_TSADC
>   tristate "Freescale i.MX25 integrated Touchscreen and ADC unit"
>   select REGMAP_MMIO
> + select IRQ_DOMAIN
>   depends on (SOC_IMX25 && OF) || COMPILE_TEST
>   help
> Enable support for the integrated Touchscreen and ADC unit of the
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

[PATCH 2/7] nbd: Remove signal usage

[Markus Pargmann]

As discussed on the mailing list, the usage of signals for timeout
handling has a lot of potential issues. The nbd driver used for some
time signals for timeouts. These signals where able to get the threads
out of the blocking socket operations.

This patch removes all signal usage and uses a socket shutdown instead.
The socket descriptor itself is cleared later when the whole nbd device
is closed.

The tasks_lock is removed as we do not depend on this anymore. Instead
a new lock for the socket is introduced so we can safely work with the
socket in the timeout handler outside of the two main threads.

Cc: Oleg Nesterov <o...@redhat.com>
Cc: Christoph Hellwig <h...@infradead.org>
Signed-off-by: Markus Pargmann <m...@pengutronix.de>
Reviewed-by: Christoph Hellwig <h...@lst.de>
---
 drivers/block/nbd.c | 126 
 1 file changed, 48 insertions(+), 78 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index d61a04155d99..438f4dc549db 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -60,7 +60,8 @@ struct nbd_device {
bool disconnect; /* a disconnect has been requested by user */
 
struct timer_list timeout_timer;
-   spinlock_t tasks_lock;
+   /* protects initialization and shutdown of the socket */
+   spinlock_t sock_lock;
struct task_struct *task_recv;
struct task_struct *task_send;
 
@@ -129,13 +130,20 @@ static void nbd_end_request(struct nbd_device *nbd, 
struct request *req)
  */
 static void sock_shutdown(struct nbd_device *nbd)
 {
-   if (!nbd->sock)
+   spin_lock_irq(>sock_lock);
+
+   if (!nbd->sock) {
+   spin_unlock_irq(>sock_lock);
return;
+   }
 
dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
+   sockfd_put(nbd->sock);
nbd->sock = NULL;
-   del_timer_sync(>timeout_timer);
+   spin_unlock_irq(>sock_lock);
+
+   del_timer(>timeout_timer);
 }
 
 static void nbd_xmit_timeout(unsigned long arg)
@@ -148,17 +156,15 @@ static void nbd_xmit_timeout(unsigned long arg)
 
nbd->disconnect = true;
 
-   spin_lock_irqsave(>tasks_lock, flags);
+   spin_lock_irqsave(>sock_lock, flags);
 
-   if (nbd->task_recv)
-   force_sig(SIGKILL, nbd->task_recv);
 
-   if (nbd->task_send)
-   force_sig(SIGKILL, nbd->task_send);
+   if (nbd->sock)
+   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
 
-   spin_unlock_irqrestore(>tasks_lock, flags);
+   spin_unlock_irqrestore(>sock_lock, flags);
 
-   dev_err(nbd_to_dev(nbd), "Connection timed out, killed receiver and 
sender, shutting down connection\n");
+   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
connection\n");
 }
 
 /*
@@ -171,7 +177,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, void 
*buf, int size,
int result;
struct msghdr msg;
struct kvec iov;
-   sigset_t blocked, oldset;
unsigned long pflags = current->flags;
 
if (unlikely(!sock)) {
@@ -181,11 +186,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, 
void *buf, int size,
return -EINVAL;
}
 
-   /* Allow interception of SIGKILL only
-* Don't allow other signals to interrupt the transmission */
-   siginitsetinv(, sigmask(SIGKILL));
-   sigprocmask(SIG_SETMASK, , );
-
current->flags |= PF_MEMALLOC;
do {
sock->sk->sk_allocation = GFP_NOIO | __GFP_MEMALLOC;
@@ -212,7 +212,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, void 
*buf, int size,
buf += result;
} while (size > 0);
 
-   sigprocmask(SIG_SETMASK, , NULL);
tsk_restore_flags(current, pflags, PF_MEMALLOC);
 
if (!send && nbd->xmit_timeout)
@@ -406,23 +405,18 @@ static int nbd_thread_recv(struct nbd_device *nbd)
 {
struct request *req;
int ret;
-   unsigned long flags;
 
BUG_ON(nbd->magic != NBD_MAGIC);
 
sk_set_memalloc(nbd->sock->sk);
 
-   spin_lock_irqsave(>tasks_lock, flags);
nbd->task_recv = current;
-   spin_unlock_irqrestore(>tasks_lock, flags);
 
ret = device_create_file(disk_to_dev(nbd->disk), _attr);
if (ret) {
dev_err(disk_to_dev(nbd->disk), "device_create_file failed!\n");
 
-   spin_lock_irqsave(>tasks_lock, flags);
nbd->task_recv = NULL;
-   spin_unlock_irqrestore(>tasks_lock, flags);
 
return ret;
}
@@ -439,19 +433,7 @@ static int nbd_thread_recv(struct nbd_device *nbd)
 
device_remove_file(disk_to_dev(nbd->disk), _attr);
 
-   spin_lock_irqsave(&

[PATCH 2/7] nbd: Remove signal usage

[Markus Pargmann]

As discussed on the mailing list, the usage of signals for timeout
handling has a lot of potential issues. The nbd driver used for some
time signals for timeouts. These signals where able to get the threads
out of the blocking socket operations.

This patch removes all signal usage and uses a socket shutdown instead.
The socket descriptor itself is cleared later when the whole nbd device
is closed.

The tasks_lock is removed as we do not depend on this anymore. Instead
a new lock for the socket is introduced so we can safely work with the
socket in the timeout handler outside of the two main threads.

Cc: Oleg Nesterov 
Cc: Christoph Hellwig 
Signed-off-by: Markus Pargmann 
Reviewed-by: Christoph Hellwig 
---
 drivers/block/nbd.c | 126 
 1 file changed, 48 insertions(+), 78 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index d61a04155d99..438f4dc549db 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -60,7 +60,8 @@ struct nbd_device {
bool disconnect; /* a disconnect has been requested by user */
 
struct timer_list timeout_timer;
-   spinlock_t tasks_lock;
+   /* protects initialization and shutdown of the socket */
+   spinlock_t sock_lock;
struct task_struct *task_recv;
struct task_struct *task_send;
 
@@ -129,13 +130,20 @@ static void nbd_end_request(struct nbd_device *nbd, 
struct request *req)
  */
 static void sock_shutdown(struct nbd_device *nbd)
 {
-   if (!nbd->sock)
+   spin_lock_irq(>sock_lock);
+
+   if (!nbd->sock) {
+   spin_unlock_irq(>sock_lock);
return;
+   }
 
dev_warn(disk_to_dev(nbd->disk), "shutting down socket\n");
kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
+   sockfd_put(nbd->sock);
nbd->sock = NULL;
-   del_timer_sync(>timeout_timer);
+   spin_unlock_irq(>sock_lock);
+
+   del_timer(>timeout_timer);
 }
 
 static void nbd_xmit_timeout(unsigned long arg)
@@ -148,17 +156,15 @@ static void nbd_xmit_timeout(unsigned long arg)
 
nbd->disconnect = true;
 
-   spin_lock_irqsave(>tasks_lock, flags);
+   spin_lock_irqsave(>sock_lock, flags);
 
-   if (nbd->task_recv)
-   force_sig(SIGKILL, nbd->task_recv);
 
-   if (nbd->task_send)
-   force_sig(SIGKILL, nbd->task_send);
+   if (nbd->sock)
+   kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
 
-   spin_unlock_irqrestore(>tasks_lock, flags);
+   spin_unlock_irqrestore(>sock_lock, flags);
 
-   dev_err(nbd_to_dev(nbd), "Connection timed out, killed receiver and 
sender, shutting down connection\n");
+   dev_err(nbd_to_dev(nbd), "Connection timed out, shutting down 
connection\n");
 }
 
 /*
@@ -171,7 +177,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, void 
*buf, int size,
int result;
struct msghdr msg;
struct kvec iov;
-   sigset_t blocked, oldset;
unsigned long pflags = current->flags;
 
if (unlikely(!sock)) {
@@ -181,11 +186,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, 
void *buf, int size,
return -EINVAL;
}
 
-   /* Allow interception of SIGKILL only
-* Don't allow other signals to interrupt the transmission */
-   siginitsetinv(, sigmask(SIGKILL));
-   sigprocmask(SIG_SETMASK, , );
-
current->flags |= PF_MEMALLOC;
do {
sock->sk->sk_allocation = GFP_NOIO | __GFP_MEMALLOC;
@@ -212,7 +212,6 @@ static int sock_xmit(struct nbd_device *nbd, int send, void 
*buf, int size,
buf += result;
} while (size > 0);
 
-   sigprocmask(SIG_SETMASK, , NULL);
tsk_restore_flags(current, pflags, PF_MEMALLOC);
 
if (!send && nbd->xmit_timeout)
@@ -406,23 +405,18 @@ static int nbd_thread_recv(struct nbd_device *nbd)
 {
struct request *req;
int ret;
-   unsigned long flags;
 
BUG_ON(nbd->magic != NBD_MAGIC);
 
sk_set_memalloc(nbd->sock->sk);
 
-   spin_lock_irqsave(>tasks_lock, flags);
nbd->task_recv = current;
-   spin_unlock_irqrestore(>tasks_lock, flags);
 
ret = device_create_file(disk_to_dev(nbd->disk), _attr);
if (ret) {
dev_err(disk_to_dev(nbd->disk), "device_create_file failed!\n");
 
-   spin_lock_irqsave(>tasks_lock, flags);
nbd->task_recv = NULL;
-   spin_unlock_irqrestore(>tasks_lock, flags);
 
return ret;
}
@@ -439,19 +433,7 @@ static int nbd_thread_recv(struct nbd_device *nbd)
 
device_remove_file(disk_to_dev(nbd->disk), _attr);
 
-   spin_lock_irqsave(>tasks_lock, flags);
nbd->task_recv = NULL;
-   spin_unlock_irqrestore(>tasks

[PATCH 3/7] nbd: Timeouts are not user requested disconnects

[Markus Pargmann]

It may be useful to know in the client that a connection timed out. The
current code returns success for a timeout.

This patch reports the error code -ETIMEDOUT for a timeout.

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 438f4dc549db..2e14e51b5ea3 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -57,6 +57,7 @@ struct nbd_device {
int blksize;
loff_t bytesize;
int xmit_timeout;
+   bool timedout;
bool disconnect; /* a disconnect has been requested by user */
 
struct timer_list timeout_timer;
@@ -154,10 +155,9 @@ static void nbd_xmit_timeout(unsigned long arg)
if (list_empty(>queue_head))
return;
 
-   nbd->disconnect = true;
-
spin_lock_irqsave(>sock_lock, flags);
 
+   nbd->timedout = true;
 
if (nbd->sock)
kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
@@ -754,7 +754,10 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
if (max_part > 0)
blkdev_reread_part(bdev);
if (nbd->disconnect) /* user requested, ignore socket errors */
-   return 0;
+   error = 0;
+   if (nbd->timedout)
+   error = -ETIMEDOUT;
+
return error;
}
 
-- 
2.7.0

[PATCH 1/7] nbd: Fix debugfs error handling

[Markus Pargmann]

Static checker complains about the implemented error handling. It is
indeed wrong. We don't care about the return values of created debugfs
files.

We only have to check the return values of created dirs for NULL
pointer. If we use a null pointer as parent directory for files, this
may lead to debugfs files in wrong places.

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 55 ++---
 1 file changed, 14 insertions(+), 41 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index e4c5cc107934..d61a04155d99 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -892,50 +892,23 @@ static const struct file_operations nbd_dbg_flags_ops = {
 static int nbd_dev_dbg_init(struct nbd_device *nbd)
 {
struct dentry *dir;
-   struct dentry *f;
+
+   if (!nbd_dbg_dir)
+   return -EIO;
 
dir = debugfs_create_dir(nbd_name(nbd), nbd_dbg_dir);
-   if (IS_ERR_OR_NULL(dir)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs dir for '%s' 
(%ld)\n",
-   nbd_name(nbd), PTR_ERR(dir));
-   return PTR_ERR(dir);
+   if (!dir) {
+   dev_err(nbd_to_dev(nbd), "Failed to create debugfs dir for 
'%s'\n",
+   nbd_name(nbd));
+   return -EIO;
}
nbd->dbg_dir = dir;
 
-   f = debugfs_create_file("tasks", 0444, dir, nbd, _dbg_tasks_ops);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'tasks', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u64("size_bytes", 0444, dir, >bytesize);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'size_bytes', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u32("timeout", 0444, dir, >xmit_timeout);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'timeout', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u32("blocksize", 0444, dir, >blksize);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'blocksize', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_file("flags", 0444, dir, , _dbg_flags_ops);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'flags', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
+   debugfs_create_file("tasks", 0444, dir, nbd, _dbg_tasks_ops);
+   debugfs_create_u64("size_bytes", 0444, dir, >bytesize);
+   debugfs_create_u32("timeout", 0444, dir, >xmit_timeout);
+   debugfs_create_u32("blocksize", 0444, dir, >blksize);
+   debugfs_create_file("flags", 0444, dir, , _dbg_flags_ops);
 
return 0;
 }
@@ -950,8 +923,8 @@ static int nbd_dbg_init(void)
struct dentry *dbg_dir;
 
dbg_dir = debugfs_create_dir("nbd", NULL);
-   if (IS_ERR(dbg_dir))
-   return PTR_ERR(dbg_dir);
+   if (!dbg_dir)
+   return -EIO;
 
nbd_dbg_dir = dbg_dir;
 
-- 
2.7.0

[PATCH 3/7] nbd: Timeouts are not user requested disconnects

[Markus Pargmann]

It may be useful to know in the client that a connection timed out. The
current code returns success for a timeout.

This patch reports the error code -ETIMEDOUT for a timeout.

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 438f4dc549db..2e14e51b5ea3 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -57,6 +57,7 @@ struct nbd_device {
int blksize;
loff_t bytesize;
int xmit_timeout;
+   bool timedout;
bool disconnect; /* a disconnect has been requested by user */
 
struct timer_list timeout_timer;
@@ -154,10 +155,9 @@ static void nbd_xmit_timeout(unsigned long arg)
if (list_empty(>queue_head))
return;
 
-   nbd->disconnect = true;
-
spin_lock_irqsave(>sock_lock, flags);
 
+   nbd->timedout = true;
 
if (nbd->sock)
kernel_sock_shutdown(nbd->sock, SHUT_RDWR);
@@ -754,7 +754,10 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
if (max_part > 0)
blkdev_reread_part(bdev);
if (nbd->disconnect) /* user requested, ignore socket errors */
-   return 0;
+   error = 0;
+   if (nbd->timedout)
+   error = -ETIMEDOUT;
+
return error;
}
 
-- 
2.7.0

[PATCH 1/7] nbd: Fix debugfs error handling

[Markus Pargmann]

Static checker complains about the implemented error handling. It is
indeed wrong. We don't care about the return values of created debugfs
files.

We only have to check the return values of created dirs for NULL
pointer. If we use a null pointer as parent directory for files, this
may lead to debugfs files in wrong places.

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 55 ++---
 1 file changed, 14 insertions(+), 41 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index e4c5cc107934..d61a04155d99 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -892,50 +892,23 @@ static const struct file_operations nbd_dbg_flags_ops = {
 static int nbd_dev_dbg_init(struct nbd_device *nbd)
 {
struct dentry *dir;
-   struct dentry *f;
+
+   if (!nbd_dbg_dir)
+   return -EIO;
 
dir = debugfs_create_dir(nbd_name(nbd), nbd_dbg_dir);
-   if (IS_ERR_OR_NULL(dir)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs dir for '%s' 
(%ld)\n",
-   nbd_name(nbd), PTR_ERR(dir));
-   return PTR_ERR(dir);
+   if (!dir) {
+   dev_err(nbd_to_dev(nbd), "Failed to create debugfs dir for 
'%s'\n",
+   nbd_name(nbd));
+   return -EIO;
}
nbd->dbg_dir = dir;
 
-   f = debugfs_create_file("tasks", 0444, dir, nbd, _dbg_tasks_ops);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'tasks', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u64("size_bytes", 0444, dir, >bytesize);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'size_bytes', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u32("timeout", 0444, dir, >xmit_timeout);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'timeout', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_u32("blocksize", 0444, dir, >blksize);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'blocksize', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
-
-   f = debugfs_create_file("flags", 0444, dir, , _dbg_flags_ops);
-   if (IS_ERR_OR_NULL(f)) {
-   dev_err(nbd_to_dev(nbd), "Failed to create debugfs file 
'flags', %ld\n",
-   PTR_ERR(f));
-   return PTR_ERR(f);
-   }
+   debugfs_create_file("tasks", 0444, dir, nbd, _dbg_tasks_ops);
+   debugfs_create_u64("size_bytes", 0444, dir, >bytesize);
+   debugfs_create_u32("timeout", 0444, dir, >xmit_timeout);
+   debugfs_create_u32("blocksize", 0444, dir, >blksize);
+   debugfs_create_file("flags", 0444, dir, , _dbg_flags_ops);
 
return 0;
 }
@@ -950,8 +923,8 @@ static int nbd_dbg_init(void)
struct dentry *dbg_dir;
 
dbg_dir = debugfs_create_dir("nbd", NULL);
-   if (IS_ERR(dbg_dir))
-   return PTR_ERR(dbg_dir);
+   if (!dbg_dir)
+   return -EIO;
 
nbd_dbg_dir = dbg_dir;
 
-- 
2.7.0

[PATCH 5/7] nbd: Move flag parsing to a function

[Markus Pargmann]

nbd changes properties of the blockdevice depending on flags that were
received. This patch moves this flag parsing into a separate function
nbd_parse_flags().

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 22 +-
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 34a46c32c24f..b67500d5b338 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -641,6 +641,18 @@ static void nbd_bdev_reset(struct block_device *bdev)
}
 }
 
+static void nbd_parse_flags(struct nbd_device *nbd, struct block_device *bdev)
+{
+   if (nbd->flags & NBD_FLAG_READ_ONLY)
+   set_device_ro(bdev, true);
+   if (nbd->flags & NBD_FLAG_SEND_TRIM)
+   queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
+   if (nbd->flags & NBD_FLAG_SEND_FLUSH)
+   blk_queue_flush(nbd->disk->queue, REQ_FLUSH);
+   else
+   blk_queue_flush(nbd->disk->queue, 0);
+}
+
 static int nbd_dev_dbg_init(struct nbd_device *nbd);
 static void nbd_dev_dbg_close(struct nbd_device *nbd);
 
@@ -742,15 +754,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
 
mutex_unlock(>tx_lock);
 
-   if (nbd->flags & NBD_FLAG_READ_ONLY)
-   set_device_ro(bdev, true);
-   if (nbd->flags & NBD_FLAG_SEND_TRIM)
-   queue_flag_set_unlocked(QUEUE_FLAG_DISCARD,
-   nbd->disk->queue);
-   if (nbd->flags & NBD_FLAG_SEND_FLUSH)
-   blk_queue_flush(nbd->disk->queue, REQ_FLUSH);
-   else
-   blk_queue_flush(nbd->disk->queue, 0);
+   nbd_parse_flags(nbd, bdev);
 
thread = kthread_run(nbd_thread_send, nbd, "%s",
 nbd_name(nbd));
-- 
2.7.0

[PATCH 5/7] nbd: Move flag parsing to a function

[Markus Pargmann]

nbd changes properties of the blockdevice depending on flags that were
received. This patch moves this flag parsing into a separate function
nbd_parse_flags().

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 22 +-
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 34a46c32c24f..b67500d5b338 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -641,6 +641,18 @@ static void nbd_bdev_reset(struct block_device *bdev)
}
 }
 
+static void nbd_parse_flags(struct nbd_device *nbd, struct block_device *bdev)
+{
+   if (nbd->flags & NBD_FLAG_READ_ONLY)
+   set_device_ro(bdev, true);
+   if (nbd->flags & NBD_FLAG_SEND_TRIM)
+   queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
+   if (nbd->flags & NBD_FLAG_SEND_FLUSH)
+   blk_queue_flush(nbd->disk->queue, REQ_FLUSH);
+   else
+   blk_queue_flush(nbd->disk->queue, 0);
+}
+
 static int nbd_dev_dbg_init(struct nbd_device *nbd);
 static void nbd_dev_dbg_close(struct nbd_device *nbd);
 
@@ -742,15 +754,7 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
 
mutex_unlock(>tx_lock);
 
-   if (nbd->flags & NBD_FLAG_READ_ONLY)
-   set_device_ro(bdev, true);
-   if (nbd->flags & NBD_FLAG_SEND_TRIM)
-   queue_flag_set_unlocked(QUEUE_FLAG_DISCARD,
-   nbd->disk->queue);
-   if (nbd->flags & NBD_FLAG_SEND_FLUSH)
-   blk_queue_flush(nbd->disk->queue, REQ_FLUSH);
-   else
-   blk_queue_flush(nbd->disk->queue, 0);
+   nbd_parse_flags(nbd, bdev);
 
thread = kthread_run(nbd_thread_send, nbd, "%s",
 nbd_name(nbd));
-- 
2.7.0

[PATCH 7/7] nbd: Create size change events for userspace

[Markus Pargmann]

The userspace needs to know when nbd devices are ready for use.
Currently no events are created for the userspace which doesn't work for
systemd.

See the discussion here: https://github.com/systemd/systemd/pull/358

This patch uses a central point to setup the nbd-internal sizes. A ioctl
to set a size does not lead to a visible size change. The size of the
block device will be kept at 0 until nbd is connected. As soon as it
connects, the size will be changed to the real value and a uevent is
created. When disconnecting, the blockdevice is set to 0 size and
another uevent is generated.

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 79 +++--
 1 file changed, 58 insertions(+), 21 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4c5d94146aa3..f6b51d76e578 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -100,6 +100,11 @@ static inline struct device *nbd_to_dev(struct nbd_device 
*nbd)
return disk_to_dev(nbd->disk);
 }
 
+static bool nbd_is_connected(struct nbd_device *nbd)
+{
+   return !!nbd->task_recv;
+}
+
 static const char *nbdcmd_to_ascii(int cmd)
 {
switch (cmd) {
@@ -112,6 +117,42 @@ static const char *nbdcmd_to_ascii(int cmd)
return "invalid";
 }
 
+static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
+{
+   bdev->bd_inode->i_size = 0;
+   set_capacity(nbd->disk, 0);
+   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
+
+   return 0;
+}
+
+static void nbd_size_update(struct nbd_device *nbd, struct block_device *bdev)
+{
+   if (!nbd_is_connected(nbd))
+   return;
+
+   bdev->bd_inode->i_size = nbd->bytesize;
+   set_capacity(nbd->disk, nbd->bytesize >> 9);
+   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
+}
+
+static int nbd_size_set(struct nbd_device *nbd, struct block_device *bdev,
+   int blocksize, int nr_blocks)
+{
+   int ret;
+
+   ret = set_blocksize(bdev, blocksize);
+   if (ret)
+   return ret;
+
+   nbd->blksize = blocksize;
+   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
+
+   nbd_size_update(nbd, bdev);
+
+   return 0;
+}
+
 static void nbd_end_request(struct nbd_device *nbd, struct request *req)
 {
int error = req->errors ? -EIO : 0;
@@ -401,7 +442,7 @@ static struct device_attribute pid_attr = {
.show = pid_show,
 };
 
-static int nbd_thread_recv(struct nbd_device *nbd)
+static int nbd_thread_recv(struct nbd_device *nbd, struct block_device *bdev)
 {
struct request *req;
int ret;
@@ -421,6 +462,8 @@ static int nbd_thread_recv(struct nbd_device *nbd)
return ret;
}
 
+   nbd_size_update(nbd, bdev);
+
while (1) {
req = nbd_read_stat(nbd);
if (IS_ERR(req)) {
@@ -431,6 +474,8 @@ static int nbd_thread_recv(struct nbd_device *nbd)
nbd_end_request(nbd, req);
}
 
+   nbd_size_clear(nbd, bdev);
+
device_remove_file(disk_to_dev(nbd->disk), _attr);
 
nbd->task_recv = NULL;
@@ -707,20 +752,19 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
return err;
}
 
-   case NBD_SET_BLKSIZE:
-   nbd->blksize = arg;
-   nbd->bytesize &= ~(nbd->blksize-1);
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-   return 0;
+   case NBD_SET_BLKSIZE: {
+   loff_t bsize = nbd->bytesize;
+   do_div(bsize, arg);
+
+   return nbd_size_set(nbd, bdev, arg, bsize);
+   }
 
case NBD_SET_SIZE:
-   nbd->bytesize = arg & ~(nbd->blksize-1);
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-   return 0;
+   return nbd_size_set(nbd, bdev, nbd->blksize,
+   arg / nbd->blksize);
+
+   case NBD_SET_SIZE_BLOCKS:
+   return nbd_size_set(nbd, bdev, nbd->blksize, arg);
 
case NBD_SET_TIMEOUT:
nbd->xmit_timeout = arg * HZ;
@@ -736,13 +780,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
nbd->flags = arg;
return 0;
 
-   case NBD_SET_SIZE_BLOCKS:
-   nbd->bytesize = ((u64) arg) * nbd->blksize;
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-

[PATCH 7/7] nbd: Create size change events for userspace

[Markus Pargmann]

The userspace needs to know when nbd devices are ready for use.
Currently no events are created for the userspace which doesn't work for
systemd.

See the discussion here: https://github.com/systemd/systemd/pull/358

This patch uses a central point to setup the nbd-internal sizes. A ioctl
to set a size does not lead to a visible size change. The size of the
block device will be kept at 0 until nbd is connected. As soon as it
connects, the size will be changed to the real value and a uevent is
created. When disconnecting, the blockdevice is set to 0 size and
another uevent is generated.

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 79 +++--
 1 file changed, 58 insertions(+), 21 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4c5d94146aa3..f6b51d76e578 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -100,6 +100,11 @@ static inline struct device *nbd_to_dev(struct nbd_device 
*nbd)
return disk_to_dev(nbd->disk);
 }
 
+static bool nbd_is_connected(struct nbd_device *nbd)
+{
+   return !!nbd->task_recv;
+}
+
 static const char *nbdcmd_to_ascii(int cmd)
 {
switch (cmd) {
@@ -112,6 +117,42 @@ static const char *nbdcmd_to_ascii(int cmd)
return "invalid";
 }
 
+static int nbd_size_clear(struct nbd_device *nbd, struct block_device *bdev)
+{
+   bdev->bd_inode->i_size = 0;
+   set_capacity(nbd->disk, 0);
+   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
+
+   return 0;
+}
+
+static void nbd_size_update(struct nbd_device *nbd, struct block_device *bdev)
+{
+   if (!nbd_is_connected(nbd))
+   return;
+
+   bdev->bd_inode->i_size = nbd->bytesize;
+   set_capacity(nbd->disk, nbd->bytesize >> 9);
+   kobject_uevent(_to_dev(nbd)->kobj, KOBJ_CHANGE);
+}
+
+static int nbd_size_set(struct nbd_device *nbd, struct block_device *bdev,
+   int blocksize, int nr_blocks)
+{
+   int ret;
+
+   ret = set_blocksize(bdev, blocksize);
+   if (ret)
+   return ret;
+
+   nbd->blksize = blocksize;
+   nbd->bytesize = (loff_t)blocksize * (loff_t)nr_blocks;
+
+   nbd_size_update(nbd, bdev);
+
+   return 0;
+}
+
 static void nbd_end_request(struct nbd_device *nbd, struct request *req)
 {
int error = req->errors ? -EIO : 0;
@@ -401,7 +442,7 @@ static struct device_attribute pid_attr = {
.show = pid_show,
 };
 
-static int nbd_thread_recv(struct nbd_device *nbd)
+static int nbd_thread_recv(struct nbd_device *nbd, struct block_device *bdev)
 {
struct request *req;
int ret;
@@ -421,6 +462,8 @@ static int nbd_thread_recv(struct nbd_device *nbd)
return ret;
}
 
+   nbd_size_update(nbd, bdev);
+
while (1) {
req = nbd_read_stat(nbd);
if (IS_ERR(req)) {
@@ -431,6 +474,8 @@ static int nbd_thread_recv(struct nbd_device *nbd)
nbd_end_request(nbd, req);
}
 
+   nbd_size_clear(nbd, bdev);
+
device_remove_file(disk_to_dev(nbd->disk), _attr);
 
nbd->task_recv = NULL;
@@ -707,20 +752,19 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
return err;
}
 
-   case NBD_SET_BLKSIZE:
-   nbd->blksize = arg;
-   nbd->bytesize &= ~(nbd->blksize-1);
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-   return 0;
+   case NBD_SET_BLKSIZE: {
+   loff_t bsize = nbd->bytesize;
+   do_div(bsize, arg);
+
+   return nbd_size_set(nbd, bdev, arg, bsize);
+   }
 
case NBD_SET_SIZE:
-   nbd->bytesize = arg & ~(nbd->blksize-1);
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-   return 0;
+   return nbd_size_set(nbd, bdev, nbd->blksize,
+   arg / nbd->blksize);
+
+   case NBD_SET_SIZE_BLOCKS:
+   return nbd_size_set(nbd, bdev, nbd->blksize, arg);
 
case NBD_SET_TIMEOUT:
nbd->xmit_timeout = arg * HZ;
@@ -736,13 +780,6 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
nbd->flags = arg;
return 0;
 
-   case NBD_SET_SIZE_BLOCKS:
-   nbd->bytesize = ((u64) arg) * nbd->blksize;
-   bdev->bd_inode->i_size = nbd->bytesize;
-   set_blocksize(bdev, nbd->blksize);
-   set_capacity(nbd->disk, nbd->bytesize >> 9);
-   return 0;
-

[PATCH 6/7] nbd: ratelimit error msgs after socket close

[Markus Pargmann]

From: Dan Streetman <dan.street...@canonical.com>

Make the "Attempted send on closed socket" error messages generated in
nbd_request_handler() ratelimited.

When the nbd socket is shutdown, the nbd_request_handler() function emits
an error message for every request remaining in its queue.  If the queue
is large, this will spam a large amount of messages to the log.  There's
no need for a separate error message for each request, so this patch
ratelimits it.

In the specific case this was found, the system was virtual and the error
messages were logged to the serial port, which overwhelmed it.

Fixes: 4d48a542b427 ("nbd: fix I/O hang on disconnected nbds")
Signed-off-by: Dan Streetman <dan.street...@canonical.com>
Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b67500d5b338..4c5d94146aa3 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -580,8 +580,8 @@ static void nbd_request_handler(struct request_queue *q)
req, req->cmd_type);
 
if (unlikely(!nbd->sock)) {
-   dev_err(disk_to_dev(nbd->disk),
-   "Attempted send on closed socket\n");
+   dev_err_ratelimited(disk_to_dev(nbd->disk),
+   "Attempted send on closed 
socket\n");
req->errors++;
nbd_end_request(nbd, req);
spin_lock_irq(q->queue_lock);
-- 
2.7.0

[PATCH 6/7] nbd: ratelimit error msgs after socket close

[Markus Pargmann]

From: Dan Streetman 

Make the "Attempted send on closed socket" error messages generated in
nbd_request_handler() ratelimited.

When the nbd socket is shutdown, the nbd_request_handler() function emits
an error message for every request remaining in its queue.  If the queue
is large, this will spam a large amount of messages to the log.  There's
no need for a separate error message for each request, so this patch
ratelimits it.

In the specific case this was found, the system was virtual and the error
messages were logged to the serial port, which overwhelmed it.

Fixes: 4d48a542b427 ("nbd: fix I/O hang on disconnected nbds")
Signed-off-by: Dan Streetman 
Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index b67500d5b338..4c5d94146aa3 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -580,8 +580,8 @@ static void nbd_request_handler(struct request_queue *q)
req, req->cmd_type);
 
if (unlikely(!nbd->sock)) {
-   dev_err(disk_to_dev(nbd->disk),
-   "Attempted send on closed socket\n");
+   dev_err_ratelimited(disk_to_dev(nbd->disk),
+   "Attempted send on closed 
socket\n");
req->errors++;
nbd_end_request(nbd, req);
spin_lock_irq(q->queue_lock);
-- 
2.7.0

[PULL] NBD for 4.6

[Markus Pargmann]

Hi Jens,

This pull request contains 7 patches for 4.6.

Patch 1 fixes some unnecessarily complicated code I introduced some versions
ago for debugfs.

Patch 2 removes the criticised signal usage within NBD to kill the NBD threads
after a timeout. This code was used for the last years and is now replaced by
simply killing the tcp connection.

Patches 3-6 are some smaller cleanups.

Patch 7 uevents for the userspace. This way udev/systemd can react on connected
NBD devices.

Best Regards,

Markus



The following changes since commit 92e963f50fc74041b5e9e744c330dca48e04f08d:

  Linux 4.5-rc1 (2016-01-24 13:06:47 -0800)

are available in the git repository at:

  git://git.pengutronix.de/git/mpa/linux-nbd.git tags/nbd-for-4.6

for you to fetch changes up to 37091fdd831f28a6509008542174ed324dd645bc:

  nbd: Create size change events for userspace (2016-02-15 10:35:47 +0100)


NBD for 4.6


Dan Streetman (1):
  nbd: ratelimit error msgs after socket close

Markus Pargmann (6):
  nbd: Fix debugfs error handling
  nbd: Remove signal usage
  nbd: Timeouts are not user requested disconnects
  nbd: Cleanup reset of nbd and bdev after a disconnect
  nbd: Move flag parsing to a function
  nbd: Create size change events for userspace

 drivers/block/nbd.c | 335 ++--
 1 file changed, 170 insertions(+), 165 deletions(-)

-- 
2.7.0

[PATCH 4/7] nbd: Cleanup reset of nbd and bdev after a disconnect

[Markus Pargmann]

Group all variables that are reset after a disconnect into reset
functions. This patch adds two of these functions, nbd_reset() and
nbd_bdev_reset().

Signed-off-by: Markus Pargmann <m...@pengutronix.de>
---
 drivers/block/nbd.c | 40 +---
 1 file changed, 29 insertions(+), 11 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 2e14e51b5ea3..34a46c32c24f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -617,6 +617,30 @@ out:
return ret;
 }
 
+/* Reset all properties of an NBD device */
+static void nbd_reset(struct nbd_device *nbd)
+{
+   nbd->disconnect = false;
+   nbd->timedout = false;
+   nbd->blksize = 1024;
+   nbd->bytesize = 0;
+   set_capacity(nbd->disk, 0);
+   nbd->flags = 0;
+   nbd->xmit_timeout = 0;
+   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
+   del_timer_sync(>timeout_timer);
+}
+
+static void nbd_bdev_reset(struct block_device *bdev)
+{
+   set_device_ro(bdev, false);
+   bdev->bd_inode->i_size = 0;
+   if (max_part > 0) {
+   blkdev_reread_part(bdev);
+   bdev->bd_invalidated = 1;
+   }
+}
+
 static int nbd_dev_dbg_init(struct nbd_device *nbd);
 static void nbd_dev_dbg_close(struct nbd_device *nbd);
 
@@ -745,19 +769,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
sock_shutdown(nbd);
nbd_clear_que(nbd);
kill_bdev(bdev);
-   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
-   set_device_ro(bdev, false);
-   nbd->flags = 0;
-   nbd->bytesize = 0;
-   bdev->bd_inode->i_size = 0;
-   set_capacity(nbd->disk, 0);
-   if (max_part > 0)
-   blkdev_reread_part(bdev);
+   nbd_bdev_reset(bdev);
+
if (nbd->disconnect) /* user requested, ignore socket errors */
error = 0;
if (nbd->timedout)
error = -ETIMEDOUT;
 
+   nbd_reset(nbd);
+
return error;
}
 
@@ -1023,14 +1043,12 @@ static int __init nbd_init(void)
nbd_dev[i].timeout_timer.data = (unsigned long)_dev[i];
init_waitqueue_head(_dev[i].active_wq);
init_waitqueue_head(_dev[i].waiting_wq);
-   nbd_dev[i].blksize = 1024;
-   nbd_dev[i].bytesize = 0;
disk->major = NBD_MAJOR;
disk->first_minor = i << part_shift;
disk->fops = _fops;
disk->private_data = _dev[i];
sprintf(disk->disk_name, "nbd%d", i);
-   set_capacity(disk, 0);
+   nbd_reset(_dev[i]);
add_disk(disk);
}
 
-- 
2.7.0

[PULL] NBD for 4.6

[Markus Pargmann]

Hi Jens,

This pull request contains 7 patches for 4.6.

Patch 1 fixes some unnecessarily complicated code I introduced some versions
ago for debugfs.

Patch 2 removes the criticised signal usage within NBD to kill the NBD threads
after a timeout. This code was used for the last years and is now replaced by
simply killing the tcp connection.

Patches 3-6 are some smaller cleanups.

Patch 7 uevents for the userspace. This way udev/systemd can react on connected
NBD devices.

Best Regards,

Markus



The following changes since commit 92e963f50fc74041b5e9e744c330dca48e04f08d:

  Linux 4.5-rc1 (2016-01-24 13:06:47 -0800)

are available in the git repository at:

  git://git.pengutronix.de/git/mpa/linux-nbd.git tags/nbd-for-4.6

for you to fetch changes up to 37091fdd831f28a6509008542174ed324dd645bc:

  nbd: Create size change events for userspace (2016-02-15 10:35:47 +0100)


NBD for 4.6


Dan Streetman (1):
  nbd: ratelimit error msgs after socket close

Markus Pargmann (6):
  nbd: Fix debugfs error handling
  nbd: Remove signal usage
  nbd: Timeouts are not user requested disconnects
  nbd: Cleanup reset of nbd and bdev after a disconnect
  nbd: Move flag parsing to a function
  nbd: Create size change events for userspace

 drivers/block/nbd.c | 335 ++--
 1 file changed, 170 insertions(+), 165 deletions(-)

-- 
2.7.0

[PATCH 4/7] nbd: Cleanup reset of nbd and bdev after a disconnect

[Markus Pargmann]

Group all variables that are reset after a disconnect into reset
functions. This patch adds two of these functions, nbd_reset() and
nbd_bdev_reset().

Signed-off-by: Markus Pargmann 
---
 drivers/block/nbd.c | 40 +---
 1 file changed, 29 insertions(+), 11 deletions(-)

diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 2e14e51b5ea3..34a46c32c24f 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -617,6 +617,30 @@ out:
return ret;
 }
 
+/* Reset all properties of an NBD device */
+static void nbd_reset(struct nbd_device *nbd)
+{
+   nbd->disconnect = false;
+   nbd->timedout = false;
+   nbd->blksize = 1024;
+   nbd->bytesize = 0;
+   set_capacity(nbd->disk, 0);
+   nbd->flags = 0;
+   nbd->xmit_timeout = 0;
+   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
+   del_timer_sync(>timeout_timer);
+}
+
+static void nbd_bdev_reset(struct block_device *bdev)
+{
+   set_device_ro(bdev, false);
+   bdev->bd_inode->i_size = 0;
+   if (max_part > 0) {
+   blkdev_reread_part(bdev);
+   bdev->bd_invalidated = 1;
+   }
+}
+
 static int nbd_dev_dbg_init(struct nbd_device *nbd);
 static void nbd_dev_dbg_close(struct nbd_device *nbd);
 
@@ -745,19 +769,15 @@ static int __nbd_ioctl(struct block_device *bdev, struct 
nbd_device *nbd,
sock_shutdown(nbd);
nbd_clear_que(nbd);
kill_bdev(bdev);
-   queue_flag_clear_unlocked(QUEUE_FLAG_DISCARD, nbd->disk->queue);
-   set_device_ro(bdev, false);
-   nbd->flags = 0;
-   nbd->bytesize = 0;
-   bdev->bd_inode->i_size = 0;
-   set_capacity(nbd->disk, 0);
-   if (max_part > 0)
-   blkdev_reread_part(bdev);
+   nbd_bdev_reset(bdev);
+
if (nbd->disconnect) /* user requested, ignore socket errors */
error = 0;
if (nbd->timedout)
error = -ETIMEDOUT;
 
+   nbd_reset(nbd);
+
return error;
}
 
@@ -1023,14 +1043,12 @@ static int __init nbd_init(void)
nbd_dev[i].timeout_timer.data = (unsigned long)_dev[i];
init_waitqueue_head(_dev[i].active_wq);
init_waitqueue_head(_dev[i].waiting_wq);
-   nbd_dev[i].blksize = 1024;
-   nbd_dev[i].bytesize = 0;
disk->major = NBD_MAJOR;
disk->first_minor = i << part_shift;
disk->fops = _fops;
disk->private_data = _dev[i];
sprintf(disk->disk_name, "nbd%d", i);
-   set_capacity(disk, 0);
+   nbd_reset(_dev[i]);
add_disk(disk);
}
 
-- 
2.7.0

Re: [PATCH] iio: adc/imx25-gcq: move incorrect do_div

[Markus Pargmann]

Hi,

On Friday, February 12, 2016 12:15:29 PM Arnd Bergmann wrote:
> The newly added driver uses do_div() to device a 32-bit number, which now
> provokes a warning:
> 
> drivers/iio/adc/fsl-imx25-gcq.c: In function 'mx25_gcq_setup_cfgs':
> include/asm-generic/div64.h:207:28: warning: comparison of distinct pointer 
> types lacks a cast
>   (void)(((typeof((n)) *)0) == ((uint64_t *)0)); \
> 
> This replaces the do_div() call with a straight division operator.
> 
> Signed-off-by: Arnd Bergmann <a...@arndb.de>
> Fixes: 6df2e98c3ea5 ("iio: adc: Add imx25-gcq ADC driver")

Thanks for fixing this.

Reviewed-by: Markus Pargmann <m...@pengutronix.de>

Best Regards,

Markus

> ---
>  drivers/iio/adc/fsl-imx25-gcq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/iio/adc/fsl-imx25-gcq.c b/drivers/iio/adc/fsl-imx25-gcq.c
> index 2fd192735d5b..72b32c1ab257 100644
> --- a/drivers/iio/adc/fsl-imx25-gcq.c
> +++ b/drivers/iio/adc/fsl-imx25-gcq.c
> @@ -233,7 +233,7 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> *pdev,
>   priv->channel_vref_mv[reg] =
>   regulator_get_voltage(priv->vref[refp]);
>   /* Conversion from uV to mV */
> - do_div(priv->channel_vref_mv[reg], 1000);
> + priv->channel_vref_mv[reg] /= 1000;
>   break;
>   case MX25_ADC_REFP_INT:
>   priv->channel_vref_mv[reg] = 2500;
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.

Re: [PATCH] iio: adc/imx25-gcq: move incorrect do_div

[Markus Pargmann]

Hi,

On Friday, February 12, 2016 12:15:29 PM Arnd Bergmann wrote:
> The newly added driver uses do_div() to device a 32-bit number, which now
> provokes a warning:
> 
> drivers/iio/adc/fsl-imx25-gcq.c: In function 'mx25_gcq_setup_cfgs':
> include/asm-generic/div64.h:207:28: warning: comparison of distinct pointer 
> types lacks a cast
>   (void)(((typeof((n)) *)0) == ((uint64_t *)0)); \
> 
> This replaces the do_div() call with a straight division operator.
> 
> Signed-off-by: Arnd Bergmann 
> Fixes: 6df2e98c3ea5 ("iio: adc: Add imx25-gcq ADC driver")

Thanks for fixing this.

Reviewed-by: Markus Pargmann 

Best Regards,

Markus

> ---
>  drivers/iio/adc/fsl-imx25-gcq.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/iio/adc/fsl-imx25-gcq.c b/drivers/iio/adc/fsl-imx25-gcq.c
> index 2fd192735d5b..72b32c1ab257 100644
> --- a/drivers/iio/adc/fsl-imx25-gcq.c
> +++ b/drivers/iio/adc/fsl-imx25-gcq.c
> @@ -233,7 +233,7 @@ static int mx25_gcq_setup_cfgs(struct platform_device 
> *pdev,
>   priv->channel_vref_mv[reg] =
>   regulator_get_voltage(priv->vref[refp]);
>   /* Conversion from uV to mV */
> - do_div(priv->channel_vref_mv[reg], 1000);
> + priv->channel_vref_mv[reg] /= 1000;
>   break;
>   case MX25_ADC_REFP_INT:
>   priv->channel_vref_mv[reg] = 2500;
> 

-- 
Pengutronix e.K.   | |
Industrial Linux Solutions | http://www.pengutronix.de/  |
Peiner Str. 6-8, 31137 Hildesheim, Germany | Phone: +49-5121-206917-0|
Amtsgericht Hildesheim, HRA 2686   | Fax:   +49-5121-206917- |


signature.asc
Description: This is a digitally signed message part.