Re: [PATCH 7/9] vhost: allow userspace to create workers

2021-06-17 Thread Mike Christie
On 6/10/21 3:06 AM, Stefan Hajnoczi wrote: > On Wed, Jun 09, 2021 at 04:03:55PM -0500, Mike Christie wrote: >> On 6/7/21 10:19 AM, Stefan Hajnoczi wrote: >>> My concern is that threads should probably accounted against >>> RLIMIT_NPROC and max_threads rather than

Re: [PATCH 1/3] kthread: allow caller to pass in user_struct

2021-06-24 Thread Mike Christie
On 6/23/21 11:34 PM, kernel test robot wrote: >>> kernel/kthread.c:466:6: warning: function 'kthread_create_for_user' might >>> be a candidate for 'gnu_printf' format attribute >>> [-Wsuggest-attribute=format] > 466 | namefmt, args); > | ^~~ > Will add a __printf(4,

Re: [PATCH 3/3] vhost: pass kthread user to check RLIMIT_NPROC

2021-06-24 Thread Mike Christie
On 6/24/21 3:26 AM, kernel test robot wrote: >>> drivers/vhost/vhost.c:599:57: sparse: sparse: dereference of noderef >>> expression > vim +599 drivers/vhost/vhost.c > >581 >582/* Caller should have device mutex */ >583long vhost_dev_set_owner(struct vhost_dev

Re: [PATCH 7/9] vhost: allow userspace to create workers

2021-06-09 Thread Mike Christie
On 6/7/21 10:19 AM, Stefan Hajnoczi wrote: > My concern is that threads should probably accounted against > RLIMIT_NPROC and max_threads rather than something indirect like 128 * > RLIMIT_NOFILE (a userspace process can only have RLIMIT_NOFILE > vhost-user file descriptors open). > Ah ok, I see

Re: [RFC][PATCH] vhost/vsock: Add vsock_list file to map cid with vhost tasks

2021-05-08 Thread Mike Christie
On 5/5/21 3:38 PM, Steven Rostedt wrote: > The new trace-cmd 3.0 (which is almost ready to be released) allows for > tracing between host and guests with timestamp synchronization such that > the events on the host and the guest can be interleaved in the proper order > that they occur. KernelShark

Re: vhost: multiple worker support

2021-06-03 Thread Mike Christie
On 6/3/21 5:13 AM, Stefan Hajnoczi wrote: > On Tue, May 25, 2021 at 01:05:51PM -0500, Mike Christie wrote: >> Results: >> >> When running with the null_blk driver and vhost-scsi I can get 1.2 >> million IOPs by just running a simple >> >> fio --f

vhost: coding style and flush cleanups

2021-05-25 Thread Mike Christie
The following patches apply over linus's tree and mst's vhost branch. The patches are just some flush cleanups and a patch to reduce flush calls and some coding style fixups. My worker threading patches are built over these patches, but they seem like patches that would be ok even if the

[PATCH 2/5] vhost-scsi: remove extra flushes

2021-05-25 Thread Mike Christie
the vhost_work_dev_flush call flushed. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Jason Wang --- drivers/vhost/scsi.c | 8 1 file changed, 8 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 051a7f8dadba..2f9633ef26aa 100644 --- a/drivers/vhost/scsi.c

[PATCH 3/5] vhost-scsi: reduce flushes during endpoint clearing

2021-05-25 Thread Mike Christie
vhost_scsi_flush will flush everything, so we can clear the backends then flush, then destroy. We don't need to flush before each vq destruction because after the flush we will have made sure there can be no new cmds started and there are no running cmds. Signed-off-by: Mike Christie

[PATCH 1/5] vhost: remove work arg from vhost_work_flush

2021-05-25 Thread Mike Christie
vhost_work_flush doesn't do anything with the work arg. This patch drops it and then renames vhost_work_flush to vhost_work_dev_flush to reflect that the function flushes all the works in the dev and not just a specific queue or work item. Signed-off-by: Mike Christie Acked-by: Jason Wang

[PATCH 4/5] vhost: fix poll coding style

2021-05-25 Thread Mike Christie
We use 3 coding styles in this struct. Switch to just tabs. Signed-off-by: Mike Christie Reviewed-by: Chaitanya Kulkarni Reviewed-by: Stefan Hajnoczi Acked-by: Jason Wang --- drivers/vhost/vhost.h | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers/vhost

[PATCH 5/5] vhost: fix up vhost_work coding style

2021-05-25 Thread Mike Christie
Switch from a mix of tabs and spaces to just tabs. Signed-off-by: Mike Christie --- drivers/vhost/vhost.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h index 575c8180caad..7d5306d1229d 100644 --- a/drivers/vhost/vhost.h

vhost: multiple worker support

2021-05-25 Thread Mike Christie
The following patches apply over linus's tree or mst's vhost branch and my cleanup patchset: https://lists.linuxfoundation.org/pipermail/virtualization/2021-May/054354.html These patches allow us to support multiple vhost workers per device. I ended up just doing Stefan's original idea where

[PATCH 3/9] vhost: modify internal functions to take a vhost_worker

2021-05-25 Thread Mike Christie
to the drivers, so the next patch will convert that code. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 136 -- drivers/vhost/vhost.h | 4 +- 2 files changed, 94 insertions(+), 46 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c

[PATCH 1/9] vhost: move worker thread fields to new struct

2021-05-25 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 94 +-- drivers

[PATCH 7/9] vhost: allow userspace to create workers

2021-05-25 Thread Mike Christie
This patch allows userspace to create workers and bind them to vqs, so you can have N workers per dev and also share N workers with M vqs. The next patch will allow sharing across devices. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c| 94

[PATCH 4/9] vhost: allow vhost_polls to use different vhost_workers

2021-05-25 Thread Mike Christie
This allows vhost_polls to use the worker the vq the poll is associated with. This also exports a helper vhost_vq_work_queue which is used here internally, and will be used in the vhost-scsi patches. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 6 -- drivers/vhost/vhost.c | 19

[PATCH 2/9] vhost: move vhost worker creation to kick setup

2021-05-25 Thread Mike Christie
here. If the tools are using the newer ioctls that have already created the workers then we also detect that here and do nothing. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 23 --- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/vhost.c

[PATCH 8/9] vhost: add vhost_dev pointer to vhost_work

2021-05-25 Thread Mike Christie
The next patch allows a vhost_worker to handle different devices. To prepare for that, this patch adds a pointer to the device on the work so we can get to the different mms in the vhost_worker thread. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 7 --- drivers/vhost/vhost.c

[PATCH 9/9] vhost: support sharing workers across devs

2021-05-25 Thread Mike Christie
that have the same owner (dev->mm value)? Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 16 +++- drivers/vhost/vhost.h | 1 - 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index eb16eb2bbee0..c32f72b1901c 100

[PATCH 6/9] vhost-scsi: make SCSI cmd completion per vq

2021-05-25 Thread Mike Christie
completions running on different threads/CPUs. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 48 +++- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index e585f2180457..b607bff41074 100644

Re: vhost: multiple worker support

2021-06-03 Thread Mike Christie
On 6/3/21 9:37 AM, Stefan Hajnoczi wrote: > On Tue, May 25, 2021 at 01:05:51PM -0500, Mike Christie wrote: >> The following patches apply over linus's tree or mst's vhost branch >> and my cleanup patchset: >> >> https://lists.linuxfoundation.org/pipermail/virtuali

[PATCH 0/3] kthread: pass in user and check RLIMIT_NPROC

2021-06-23 Thread Mike Christie
The vhost driver will create a kthread when userspace does a VHOST_SET_OWNER ioctl, but the thread is charged to the kthreadd thread. We can then end up violating the userspace process's RLIMIT_NPROC. This patchset allows drivers to pass in the user to charge/check. The patches were made over

[PATCH 1/3] kthread: allow caller to pass in user_struct

2021-06-23 Thread Mike Christie
a user_struct and pass it to copy_process. The next patches will then convert the fork/cred code. Signed-off-by: Mike Christie --- include/linux/kthread.h| 5 include/linux/sched/task.h | 2 ++ kernel/kthread.c | 58 -- 3 files changed, 56

[PATCH 2/3] kernel/fork, cred.c: allow copy_process to take user

2021-06-23 Thread Mike Christie
This allows kthread to pass copy_process the user we want to check for the RLIMIT_NPROC limit for and also charge for the new process. It will be used by vhost where userspace has that driver create threads but the kthreadd thread is checked/charged. Signed-off-by: Mike Christie --- include

[PATCH 3/3] vhost: pass kthread user to check RLIMIT_NPROC

2021-06-23 Thread Mike Christie
This has vhost pass in the user to the kthread API, so the process doing the ioctl has its RLIMIT_NPROC checked and its processes count incremented. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/vhost

Re: [PATCH 2/3] kernel/fork, cred.c: allow copy_process to take user

2021-06-29 Thread Mike Christie
On 6/29/21 8:04 AM, Christian Brauner wrote: > On Wed, Jun 23, 2021 at 10:08:03PM -0500, Mike Christie wrote: >> This allows kthread to pass copy_process the user we want to check for the >> RLIMIT_NPROC limit for and also charge for the new process. It will be used >> by vhos

[PATCH RFC 03/14] vhost-scsi: reduce flushes during endpoint clearing

2021-04-28 Thread Mike Christie
vhost_scsi_flush will flush everything, so we can clear the backends then flush, then destroy. We don't need to flush before each vq destruction because after the flush we will have made sure there can be no new cmds started and there are no running cmds. Signed-off-by: Mike Christie

[PATCH RFC 04/14] vhost: fix poll coding style

2021-04-28 Thread Mike Christie
We use like 3 coding styles in this struct. Switch to just tabs. Signed-off-by: Mike Christie Reviewed-by: Chaitanya Kulkarni Reviewed-by: Stefan Hajnoczi Acked-by: Jason Wang --- drivers/vhost/vhost.h | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/drivers

[PATCH RFC 05/14] vhost: move worker thread fields to new struct

2021-04-28 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 94 +-- drivers

[PATCH RFC 06/14] vhost: move vhost worker creation to kick setup

2021-04-28 Thread Mike Christie
here. If the tools are using the newer ioctls that have already created the workers then we also detect that here and do nothing. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 23 --- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/drivers/vhost/vhost.c

[PATCH RFC 10/14] vhost-scsi: make SCSI cmd completion per vq

2021-04-28 Thread Mike Christie
completions running on different threads/CPUs. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 48 +++- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 96462032a972..2f84cf602ab3 100644

[PATCH RFC 07/14] vhost: modify internal functions to take a vhost_worker

2021-04-28 Thread Mike Christie
to the driver, so the next patch will convert that code. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 136 -- drivers/vhost/vhost.h | 4 +- 2 files changed, 94 insertions(+), 46 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c

[PATCH RFC 08/14] vhost: allow vhost_polls to use different vhost_workers

2021-04-28 Thread Mike Christie
This allows vhost_polls to use the worker the vq the poll is associated with. This also exports a helper vhost_vq_work_queue which is used here internally, and will be used in the vhost-scsi patches. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 6 -- drivers/vhost/vhost.c | 19

[PATCH RFC 13/14] vhost: support sharing workers across devs

2021-04-28 Thread Mike Christie
This allows a worker to handle multiple device's vqs. TODO: - How to handle if the devices are in different cgroups/VMs. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 18 +++--- drivers/vhost/vhost.h | 1 - 2 files changed, 7 insertions(+), 12 deletions(-) diff --git

[PATCH RFC 09/14] vhost-scsi: flush IO vqs then send TMF rsp

2021-04-28 Thread Mike Christie
their responses when the tmf's work is run. So this patch has vhost-scsi flush the IO vqs on other worker threads before we send the tmf response. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 17 ++--- drivers/vhost/vhost.c | 6 ++ drivers/vhost/vhost.h | 1 + 3 files

[PATCH RFC 11/14] vhost: allow userspace to create workers

2021-04-28 Thread Mike Christie
This patch allows userspace to create workers and bind them to vqs, so you can have N workers per dev and also share N workers with M vqs. The next patch will allow sharing across devices. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c| 95

[PATCH RFC 14/14] vhost: allow userspace to query vq worker mappings

2021-04-28 Thread Mike Christie
Add an ioctl cmd to allow userspace to figure out the vq's worker. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 10 ++ include/uapi/linux/vhost.h | 3 ++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index

[PATCH RFC 12/14] vhost: add vhost_dev pointer to vhost_work

2021-04-28 Thread Mike Christie
The next patch allows a vhost_worker to handle different devices. To prepare for that, this patch adds a pointer to the device on the work so we can get to the different mms in the vhost_worker thread. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 7 --- drivers/vhost/vhost.c

[PATCH RFC 00/14] vhost: multiple worker support

2021-04-28 Thread Mike Christie
The following patches apply over mst's vhost branch and were tested againt that branch and also mkp's 5.13 branch which has some vhost-scsi changes. These patches allow us to support multiple vhost workers per device. I ended up just doing Stefan's original idea where userspace has the kernel

[PATCH RFC 01/14] vhost: remove work arg from vhost_work_flush

2021-04-28 Thread Mike Christie
vhost_work_flush doesn't do anything with the work arg. This patch drops it and then renames vhost_work_flush to vhost_work_dev_flush to reflect that the function flushes all the works in the dev and not just a specific queue or work item. Signed-off-by: Mike Christie Acked-by: Jason Wang

[PATCH RFC 02/14] vhost-scsi: remove extra flushes

2021-04-28 Thread Mike Christie
the vhost_work_dev_flush call flushed. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Jason Wang --- drivers/vhost/scsi.c | 8 1 file changed, 8 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 0fd596da1834..b3e6fe9b1767 100644 --- a/drivers/vhost/scsi.c

[PATCH RFC 1/1] QEMU vhost-scsi: add support for VHOST_SET_VRING_WORKER

2021-04-28 Thread Mike Christie
ng old kernel by just checking for EPERM. Does this require a feature? Signed-off-by: Mike Christie --- hw/scsi/vhost-scsi.c | 85 +++- hw/virtio/vhost-backend.c| 8 ++ include/hw/virtio/vhost-backend.h| 4 + include/hw/virtio/

Re: [PATCH RFC 00/14] vhost: multiple worker support

2021-05-04 Thread Mike Christie
On 5/4/21 10:56 AM, Stefano Garzarella wrote: > Hi Mike, > > On Wed, Apr 28, 2021 at 05:36:59PM -0500, Mike Christie wrote: >> The following patches apply over mst's vhost branch and were tested >> againt that branch and also mkp's 5.13 branch which has some

Re: [PATCH RFC 12/14] vhost: add vhost_dev pointer to vhost_work

2021-05-04 Thread Mike Christie
On 5/4/21 10:33 AM, Stefano Garzarella wrote: > On Wed, Apr 28, 2021 at 05:37:12PM -0500, Mike Christie wrote: >> The next patch allows a vhost_worker to handle different devices. To >> prepare for that, this patch adds a pointer to the device on the work so >> we can get

Re: [PATCH RFC 11/14] vhost: allow userspace to create workers

2021-05-04 Thread Mike Christie
On 5/4/21 10:30 AM, Stefano Garzarella wrote: > On Wed, Apr 28, 2021 at 05:37:11PM -0500, Mike Christie wrote: >> This patch allows userspace to create workers and bind them to vqs, so you >> can have N workers per dev and also share N workers with M vqs. The next >> pat

Re: question on vhost, limiting kernel threads and NPROC

2021-09-13 Thread Mike Christie
I just realized I forgot to cc the virt list so adding now. Christian see the very bottom for a different fork patch. On 7/12/21 7:05 AM, Stefan Hajnoczi wrote: > On Fri, Jul 09, 2021 at 11:25:37AM -0500, Mike Christie wrote: >> Hi, >> >> The goal of this email is to try a

[PATCH 0/8] Use copy_process/create_io_thread in vhost layer

2021-09-16 Thread Mike Christie
The following patches were made over linus's tree and also apply over Jens's 5.15 io_ring branch and Michael's vhost branch. The patchset allows the vhost layer to do a copy_process on the thread that does the VHOST_SET_OWNER ioctl like how io_uring does a copy_process against its userspace app

[PATCH 2/8] signal: Export ignore_signals

2021-09-16 Thread Mike Christie
The kthread API creates threads that ignore all signals by default so modules like vhost that will move from that API to kernel_copy_process will not be expecting them. This patch exports ignore_signals so those modules can keep their existing behavior. Signed-off-by: Mike Christie --- kernel

[PATCH 1/8] fork: add helper to clone a process

2021-09-16 Thread Mike Christie
and io_uring in later patches in this set. This patch also exports __set_task_comm and wake_up_new_task which is needed by modules to use the new helper. io_uring calls these functions already but its always built into the kernel so was not needed before. Signed-off-by: Mike Christie --- fs

[PATCH 3/8] fork: add option to not clone or dup files

2021-09-16 Thread Mike Christie
option so these worker threads can prevent taking an extra refcount on themselves. Signed-off-by: Mike Christie --- include/linux/sched/task.h | 3 ++- kernel/fork.c | 14 +++--- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/include/linux/sched/task.h b/include/li

[PATCH 5/8] io_uring: switch to kernel_copy_process

2021-09-16 Thread Mike Christie
Convert io_uring and io-wq to use kernel_copy_process. Signed-off-by: Mike Christie --- fs/io-wq.c | 9 +++-- fs/io_uring.c | 5 - include/linux/sched/task.h | 1 - kernel/fork.c | 22 -- 4 files changed, 11 insertions

[PATCH 7/8] vhost: use kernel_copy_process to check RLIMITs and inherit cgroups

2021-09-16 Thread Mike Christie
that owns the device. Note this patch converts us. The next patch will remove the code that is no longer needed. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 49 +++ drivers/vhost/vhost.h | 7 ++- 2 files changed, 37 insertions(+), 19

[PATCH 6/8] vhost: move worker thread fields to new struct

2021-09-16 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi --- drivers/vhost/vhost.c | 98

[PATCH 4/8] fork: move PF_IO_WORKER's kernel frame setup to new flag

2021-09-16 Thread Mike Christie
The vhost worker threads need the same frame setup as io_uring's worker threads, but do not handle signals and may not even be executing IO. This moves the setup part to a new flag. Signed-off-by: Mike Christie --- arch/x86/kernel/process.c | 4 ++-- include/linux/sched.h | 1 + include

[PATCH 8/8] vhost: remove cgroup code

2021-09-16 Thread Mike Christie
We inherit v1 and v2 cgroups from copy_process now, so we can drop the v1 only code. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 26 -- 1 file changed, 26 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 6e58417b13fc

[PATCH V2 1/9] fork: Make IO worker options flag based

2021-09-21 Thread Mike Christie
This patchset adds a couple new options to kernel_clone_args for IO thread like/related users. Instead of adding new fields to kernel_clone_args for each option, this moves us to a flags based approach by first converting io_thread. Signed-off-by: Mike Christie --- include/linux/sched/task.h

[PATCH V2 2/9] fork: pass worker_flags to copy_thread

2021-09-21 Thread Mike Christie
that function can see. Signed-off-by: Mike Christie --- arch/alpha/kernel/process.c | 2 +- arch/arc/kernel/process.c| 2 +- arch/arm/kernel/process.c| 3 ++- arch/arm64/kernel/process.c | 3 ++- arch/csky/kernel/process.c | 3 ++- arch/h8300/kernel/process.c | 3

[PATCH V2 4/9] fork: add option to not clone or dup files

2021-09-21 Thread Mike Christie
reads can prevent taking an extra refcount on themselves. Signed-off-by: Mike Christie --- include/linux/sched/task.h | 1 + kernel/fork.c | 11 +-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/sched/task.h b/include/linux/sched/tas

[PATCH V2 5/9] fork: add helper to clone a process

2021-09-21 Thread Mike Christie
and io_uring in later patches in this set. Signed-off-by: Mike Christie --- include/linux/sched/task.h | 6 - kernel/fork.c | 48 ++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/include/linux/sched/task.h b/include/linux

[PATCH V2 3/9] fork: move PF_IO_WORKER's kernel frame setup to new flag

2021-09-21 Thread Mike Christie
The vhost worker threads need the same frame setup as io_uring's worker threads, but handle signals differently and do not need the same scheduling behavior. This patch separate's the frame setup parts of PF_IO_WORKER into a kernel_clone_args flag, KERN_WORKER_USER. Signed-off-by: Mike Christie

[PATCH V2 7/9] fork: Add worker flag to ignore signals

2021-09-21 Thread Mike Christie
The kthread API creates threads that ignore all signals by default so modules like vhost that will move from that API to kernel_worker will not be expecting them. This patch adds a worker flag that tells kernel_worker to setup the task to ignore signals. Signed-off-by: Mike Christie --- include

[PATCH V2 6/9] io_uring: switch to kernel_worker

2021-09-21 Thread Mike Christie
Convert io_uring and io-wq to use kernel_worker. Signed-off-by: Mike Christie --- fs/io-wq.c | 15 --- fs/io_uring.c | 11 +-- include/linux/sched/task.h | 1 - 3 files changed, 13 insertions(+), 14 deletions(-) diff --git a/fs/io-wq.c b/fs/io

[PATCH V2 9/9] vhost: use kernel_worker to check RLIMITs and inherit v2 cgroups

2021-09-21 Thread Mike Christie
that owns the device. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 68 --- drivers/vhost/vhost.h | 7 - 2 files changed, 31 insertions(+), 44 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index c9a1f706989c

[PATCH V2 8/9] vhost: move worker thread fields to new struct

2021-09-21 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi --- drivers/vhost/vhost.c | 98

[PATCH V2 0/9] Use copy_process/create_io_thread in vhost layer

2021-09-21 Thread Mike Christie
The following patches were made over Linus's tree but also apply over Jens's 5.16 io_uring branch and Michaels' vhost/next branch. This is version 2 of the patchset and should handle all the review comments posted in V1 here:

[PATCH V4 2/8] fork: move PF_IO_WORKER's kernel frame setup to new flag

2021-10-07 Thread Mike Christie
The vhost worker threads need the same frame setup as io_uring's worker threads, but handle signals differently and do not need the same scheduling behavior. This patch separate's the frame setup parts of PF_IO_WORKER into a new PF flag PF_USER_WORKER. Signed-off-by: Mike Christie --- arch

[PATCH V4 8/8] vhost: use kernel_worker to check RLIMITs and inherit v2 cgroups

2021-10-07 Thread Mike Christie
that owns the device. Signed-off-by: Mike Christie Acked-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 65 +++ drivers/vhost/vhost.h | 7 - 2 files changed, 28 insertions(+), 44 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost

[PATCH V4 7/8] vhost: move worker thread fields to new struct

2021-10-07 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Michael S. Tsirkin --- drivers/vhost/vhost.c

[PATCH V4 0/8] Use copy_process/create_io_thread in vhost layer

2021-10-07 Thread Mike Christie
The following patches apply over Linus's, Jens's or mst's trees, but were made over linux-next because patch 6: io_uring: switch to kernel_worker has merge conflicts with Jens Axboe's for-next branch and Paul Moore's selinux tree's next branch. This is V4 of the patchset. It should handle all

[PATCH V4 1/8] fork: Make IO worker options flag based

2021-10-07 Thread Mike Christie
This patchset adds a couple new options to kernel_clone_args for IO thread like/related users. Instead of adding new fields to kernel_clone_args for each option, this moves us to a flags based approach by first converting io_thread. Signed-off-by: Mike Christie Suggested-by: Christian Brauner

[PATCH V4 3/8] fork: add option to not clone or dup files

2021-10-07 Thread Mike Christie
reads can prevent taking an extra refcount on themselves. Signed-off-by: Mike Christie Acked-by: Christian Brauner --- include/linux/sched/task.h | 1 + kernel/fork.c | 11 +-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/sched/task.h b/incl

[PATCH V4 5/8] fork: add helper to clone a process

2021-10-07 Thread Mike Christie
and io_uring in later patches in this set. [added flag validation code from Christian Brauner's patch] Signed-off-by: Mike Christie Acked-by: Christian Brauner --- include/linux/sched/task.h | 4 +++ kernel/fork.c | 71 ++ 2 files changed, 75

[PATCH V4 4/8] fork: Add KERNEL_WORKER flag to ignore signals

2021-10-07 Thread Mike Christie
py_signals() to ease the transition from kthreads to this new api. Signed-off-by: Christian Brauner Signed-off-by: Mike Christie --- include/linux/sched/task.h | 1 + kernel/fork.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/include/linux/sched/task.h b/include/linux/sc

[PATCH V4 6/8] io_uring: switch to kernel_worker

2021-10-07 Thread Mike Christie
Convert io_uring and io-wq to use kernel_worker. Signed-off-by: Mike Christie Reviewed-by: Christian Brauner Reviewed-by: Jens Axboe --- Note: To avoid patch application conflicts this patch was made over linux-next which has Jens Axboe's block tree's for-next branch and Paul Moore's selinux

Re: [PATCH V3 5/9] fork: add helper to clone a process

2021-10-05 Thread Mike Christie
On 10/5/21 7:50 AM, Christian Brauner wrote: > On Mon, Oct 04, 2021 at 02:21:24PM -0500, Mike Christie wrote: >> The vhost layer has similar requirements as io_uring where its worker >> threads need to access the userspace thread's memory, want to inherit the >> parents's c

[PATCH V3 1/9] fork: Make IO worker options flag based

2021-10-04 Thread Mike Christie
This patchset adds a couple new options to kernel_clone_args for IO thread like/related users. Instead of adding new fields to kernel_clone_args for each option, this moves us to a flags based approach by first converting io_thread. Signed-off-by: Mike Christie Suggested-by: Christian Brauner

[PATCH V3 0/9] Use copy_process/create_io_thread in vhost layer

2021-10-04 Thread Mike Christie
The following patches were made over Linus's tree but also apply over Jens's for-next io_uring branch and Michaels' vhost/next branch. This is V3 of the patchset. It should handle all the review comments posted in V1 and V2. If I missed a comment, please let me know. This patchset allows the

[PATCH V3 2/9] fork: pass worker_flags to copy_thread

2021-10-04 Thread Mike Christie
that function can see. Signed-off-by: Mike Christie Acked-by: Christian Brauner Acked-by: Geert Uytterhoeven --- arch/alpha/kernel/process.c | 2 +- arch/arc/kernel/process.c| 2 +- arch/arm/kernel/process.c| 3 ++- arch/arm64/kernel/process.c | 3 ++- arch/csky/kernel/process.c

[PATCH V3 4/9] fork: add option to not clone or dup files

2021-10-04 Thread Mike Christie
reads can prevent taking an extra refcount on themselves. Signed-off-by: Mike Christie Acked-by: Christian Brauner --- include/linux/sched/task.h | 1 + kernel/fork.c | 11 +-- 2 files changed, 10 insertions(+), 2 deletions(-) diff --git a/include/linux/sched/task.h b/incl

[PATCH V3 5/9] fork: add helper to clone a process

2021-10-04 Thread Mike Christie
and io_uring in later patches in this set. Signed-off-by: Mike Christie Acked-by: Christian Brauner --- include/linux/sched/task.h | 6 - kernel/fork.c | 48 ++ 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/include/linux

[PATCH V3 8/9] vhost: move worker thread fields to new struct

2021-10-04 Thread Mike Christie
This is just a prep patch. It moves the worker related fields to a new vhost_worker struct and moves the code around to create some helpers that will be used in the next patches. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi Acked-by: Michael S. Tsirkin --- drivers/vhost/vhost.c

[PATCH V3 3/9] fork: move PF_IO_WORKER's kernel frame setup to new flag

2021-10-04 Thread Mike Christie
The vhost worker threads need the same frame setup as io_uring's worker threads, but handle signals differently and do not need the same scheduling behavior. This patch separate's the frame setup parts of PF_IO_WORKER into a kernel_clone_args flag, KERN_WORKER_USER. Signed-off-by: Mike Christie

[PATCH V3 7/9] fork: Add worker flag to ignore signals

2021-10-04 Thread Mike Christie
The kthread API creates threads that ignore all signals by default so modules like vhost that will move from that API to kernel_worker will not be expecting them. This patch adds a worker flag that tells kernel_worker to setup the task to ignore signals. Signed-off-by: Mike Christie Acked

[PATCH V3 9/9] vhost: use kernel_worker to check RLIMITs and inherit v2 cgroups

2021-10-04 Thread Mike Christie
that owns the device. Signed-off-by: Mike Christie Acked-by: Michael S. Tsirkin --- drivers/vhost/vhost.c | 68 --- drivers/vhost/vhost.h | 7 - 2 files changed, 31 insertions(+), 44 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost

[PATCH V3 6/9] io_uring: switch to kernel_worker

2021-10-04 Thread Mike Christie
Convert io_uring and io-wq to use kernel_worker. Signed-off-by: Mike Christie Reviewed-by: Christian Brauner --- fs/io-wq.c | 15 --- fs/io_uring.c | 11 +-- include/linux/sched/task.h | 1 - 3 files changed, 13 insertions(+), 14 deletions

[PATCH V3 00/11] vhost: multiple worker support

2021-10-21 Thread Mike Christie
The following patches apply over linus's tree and this patchset https://lore.kernel.org/all/20211007214448.6282-1-michael.chris...@oracle.com/ which allows us to check the vhost owner thread's RLIMITs: It looks like that patchset has been ok'd by all the major parties and just needs a small

[PATCH] QEMU vhost-scsi: add support for VHOST_SET_VRING_WORKER

2021-10-21 Thread Mike Christie
ODO: - Better support for sharing workers where we bind based on ISR to vq mapping. Signed-off-by: Mike Christie --- hw/scsi/vhost-scsi.c | 65 hw/virtio/vhost-backend.c| 8 +++ include/hw/virtio/vhost-backend.h| 4 ++ incl

[PATCH V3 02/11] vhost, vhost-net: add helper to check if vq has work

2021-10-21 Thread Mike Christie
This adds a helper to check if a vq has work pending and converts vhost-net to use it. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 2 +- drivers/vhost/vhost.c | 6 +++--- drivers/vhost/vhost.h | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/vhost

[PATCH V3 05/11] vhost: convert poll work to be vq based

2021-10-21 Thread Mike Christie
This has the drivers pass in their poll to vq mapping and then converts the core poll code to use the vq based helpers. Signed-off-by: Mike Christie --- drivers/vhost/net.c | 6 -- drivers/vhost/vhost.c | 10 ++ drivers/vhost/vhost.h | 4 +++- 3 files changed, 13 insertions

[PATCH V3 01/11] vhost: add vhost_worker pointer to vhost_virtqueue

2021-10-21 Thread Mike Christie
This patchset allows userspace to map vqs to different workers. This patch adds a worker pointer to the vq so we can store that info. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 24 +--- drivers/vhost/vhost.h | 1 + 2 files changed, 14 insertions(+), 11

[PATCH V3 07/11] vhost-scsi: make SCSI cmd completion per vq

2021-10-21 Thread Mike Christie
completions running on different threads/CPUs. Signed-off-by: Mike Christie Reviewed-by: Stefan Hajnoczi --- drivers/vhost/scsi.c | 48 +++- 1 file changed, 25 insertions(+), 23 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index

[PATCH V3 06/11] vhost-sock: convert to vq helpers

2021-10-21 Thread Mike Christie
Convert from vhost dev based helpers to vq ones. Signed-off-by: Mike Christie --- drivers/vhost/vsock.c | 8 +--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/vsock.c b/drivers/vhost/vsock.c index 938aefbc75ec..c50c60d0955e 100644 --- a/drivers/vhost/vsock.c

[PATCH V3 04/11] vhost: take worker or vq instead of dev for flushing

2021-10-21 Thread Mike Christie
in the next patches. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 30 +- drivers/vhost/vhost.h | 1 + 2 files changed, 22 insertions(+), 9 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index f2c6981297d0..e4d765f6c821 100644 --- a/drivers

[PATCH V3 03/11] vhost: take worker or vq instead of dev for queueing

2021-10-21 Thread Mike Christie
in the next patches. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 44 +++ drivers/vhost/vhost.h | 1 + 2 files changed, 29 insertions(+), 16 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index f6bfa1c481df..f2c6981297d0

[PATCH V3 10/11] vhost: remove device wide queu/flushing helpers

2021-10-21 Thread Mike Christie
vhost_work_queue and vhost_work_dev_flush are no longer used, so drop them. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c | 12 drivers/vhost/vhost.h | 2 -- 2 files changed, 14 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index c84024afefff

[PATCH V3 08/11] vhost-scsi: convert to vq helpers

2021-10-21 Thread Mike Christie
Convert from vhost dev based helpers to vq ones. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 27 +-- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 0d85ddb68420..08beba73ada4 100644

[PATCH V3 09/11] vhost-scsi: flush IO vqs then send TMF rsp

2021-10-21 Thread Mike Christie
vqs before sending the TMF response. Signed-off-by: Mike Christie --- drivers/vhost/scsi.c | 22 +++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 08beba73ada4..29d9adcdb4fc 100644 --- a/drivers/vhost/scsi.c

[PATCH V3 11/11] vhost: allow userspace to create workers

2021-10-21 Thread Mike Christie
This patch allows userspace to create workers and bind them to vqs. You can have N workers per dev and also share N workers with M vqs. Signed-off-by: Mike Christie --- drivers/vhost/vhost.c| 99 drivers/vhost/vhost.h| 2 +- include/uapi

[PATCH V6 04/10] fork: Add USER_WORKER flag to ignore signals

2021-11-29 Thread Mike Christie
py_signals() to ease the transition from kthreads to this new api. Signed-off-by: Christian Brauner Signed-off-by: Mike Christie Reviewed-by: Christoph Hellwig --- include/linux/sched/task.h | 1 + kernel/fork.c | 3 +++ 2 files changed, 4 insertions(+) diff --git a/include/linux/sc

  1   2   3   4   5   >