[RFC] [PATCH] ipc/util.c: Use binary search for max_idx

2021-04-07 Thread Manfred Spraul
SHIFT, this could be a loop over 16 million entries. As there is no get_last() function for idr structures: Implement a "get_last()" using a binary search. As far as I see, ipc is the only user that needs get_last(), thus implement it in ipc/util.c and not in a central location. Signed-

Re: [PATCH] ipc/msg: add msgsnd_timed and msgrcv_timed syscall for system V message queue

2021-03-04 Thread Manfred Spraul
Hi Eric, On 3/4/21 2:12 AM, Andrew Morton wrote: On Tue, 23 Feb 2021 23:11:43 +0800 Eric Gao wrote: sometimes, we need the msgsnd or msgrcv syscall can return after a limited time, so that the business thread do not be blocked here all the time. In this case, I add the msgsnd_timed and msgrc

Re: [PATCH] ipc/msg.c: wake up senders until there is a queue empty capacity

2020-06-01 Thread Manfred Spraul
g herds is always good, avoiding schedule() is always good. Thus I would start with pipelined_receive, and then we would need feedback from apps that use sysv msg. (old fakeroot is what I remember as test app) On Wed, May 27, 2020 at 08:03:17AM +0200, Manfred Spraul wrote: Hello Artur, On 5/

Re: [PATCH] ipc/msg.c: wake up senders until there is a queue empty capacity

2020-05-26 Thread Manfred Spraul
ueue: implement pipelined receive. On Sun, May 24, 2020 at 03:21:31PM +0200, Manfred Spraul wrote: Hello Artur, On 5/23/20 10:34 PM, Artur Barsegyan wrote: Take into account the total size of the already enqueued messages of previously handled senders before another one. Otherwise, we have s

Re: [PATCH] ipc/msg.c: wake up senders until there is a queue empty capacity

2020-05-24 Thread Manfred Spraul
--     Manfred >From fe2f257b1950a19bf5c6f67e71aa25c2f13bcdc3 Mon Sep 17 00:00:00 2001 From: Manfred Spraul Date: Sun, 24 May 2020 14:47:31 +0200 Subject: [PATCH 2/2] ipc/msg.c: Handle case of senders not enqueuing the message The patch "ipc/msg.c: wake up senders until there is

[PATCH] xarray.h: Correct return code for xa_store_{bh,irq}()

2020-04-30 Thread Manfred Spraul
documentation. Signed-off-by: Manfred Spraul --- include/linux/xarray.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index d79b8e3aa08d..2815c4ec89b1 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -576,7 +576,7

Re: [PATCH -next] ipc: use GFP_ATOMIC under spin lock

2020-04-28 Thread Manfred Spraul
Hello together, On 4/28/20 1:14 PM, Matthew Wilcox wrote: On Tue, Apr 28, 2020 at 03:47:36AM +, Wei Yongjun wrote: The function ipc_id_alloc() is called from ipc_addid(), in which a spin lock is held, so we should use GFP_ATOMIC instead. Fixes: de5738d1c364 ("ipc: convert ipcs_idr to XArra

Re: [ipc/sem.c] 6394de3b86: BUG:kernel_NULL_pointer_dereference,address

2019-10-23 Thread Manfred Spraul
Hello, On 10/21/19 10:35 AM, kernel test robot wrote: FYI, we noticed the following commit (built with gcc-7): commit: 6394de3b868537a90dd9128607192b0e97109f6b ("[PATCH 4/5] ipc/sem.c: Document and update memory barriers") url: https://github.com/0day-ci/linux/commits/Manfred-Spr

[PATCH 3/5] ipc/mqueue.c: Update/document memory barriers

2019-10-20 Thread Manfred Spraul
n another wake_q. Then: - CPU1: spin_unlock() of the task that goes to sleep is the RELEASE - CPU2: the spin_lock() of the waker is the ACQUIRE - CPU2: smp_mb__before_atomic inside wake_q_add() is the RELEASE - CPU3: smp_mb__after_spinlock() inside try_to_wake_up() is the ACQUIRE Signed-off-by: Manfr

[PATCH 5/5] ipc/sem.c: Document and update memory barriers

2019-10-20 Thread Manfred Spraul
Add comments to all barriers, and mention the rules in the block regarding locking. - Switch to using wake_q_add_safe(). Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/sem.c | 66 ++- 1 file changed, 41 insertions(+),

[PATCH 2/5] ipc/mqueue.c: Remove duplicated code

2019-10-20 Thread Manfred Spraul
Patch from Davidlohr, I just added this change log. pipelined_send() and pipelined_receive() are identical, so merge them. Signed-off-by: Davidlohr Bueso Signed-off-by: Manfred Spraul --- ipc/mqueue.c | 31 ++- 1 file changed, 18 insertions(+), 13 deletions(-) diff

[PATCH 4/5] ipc/msg.c: Update and document memory barriers.

2019-10-20 Thread Manfred Spraul
refount may have been decreased to 0 already when wake_q_add() tries to get a reference. Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/msg.c | 43 --- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/ipc/msg.c b

[PATCH 0/5] V3: Clarify/standardize memory barriers for ipc

2019-10-20 Thread Manfred Spraul
Hi, Updated series, based on input from Davidlohr and Peter Zijlstra: - I've dropped the documentation update for wake_q_add, as what it states is normal: When you call a function and pass a parameter to a structure, you as caller are responsible to ensure that the parameter is valid, and

[PATCH 1/5] smp_mb__{before,after}_atomic(): Update Documentation

2019-10-20 Thread Manfred Spraul
sentences. Fixes: 654672d4ba1a ("locking/atomics: Add _{acquire|release|relaxed}() variants of some atomic operations") Signed-off-by: Manfred Spraul Acked-by: Waiman Long Cc: Davidlohr Bueso Cc: Peter Zijlstra Cc: Will Deacon --- Documentation/memory-barriers.txt | 16 ++

Re: [PATCH 3/6] ipc/mqueue.c: Update/document memory barriers

2019-10-14 Thread Manfred Spraul
Hi Peter, On 10/14/19 3:58 PM, Peter Zijlstra wrote: On Mon, Oct 14, 2019 at 02:59:11PM +0200, Peter Zijlstra wrote: On Sat, Oct 12, 2019 at 07:49:55AM +0200, Manfred Spraul wrote: for (;;) { + /* memory barrier not required, we hold info->l

Re: [PATCH 6/6] Documentation/memory-barriers.txt: Clarify cmpxchg()

2019-10-14 Thread Manfred Spraul
Hello Peter, On 10/14/19 3:03 PM, Peter Zijlstra wrote: On Sat, Oct 12, 2019 at 07:49:58AM +0200, Manfred Spraul wrote: The documentation in memory-barriers.txt claims that smp_mb__{before,after}_atomic() are for atomic ops that do not return a value. This is misleading and doesn't matc

[PATCH 2/6] ipc/mqueue.c: Remove duplicated code

2019-10-11 Thread Manfred Spraul
Patch entirely from Davidlohr: pipelined_send() and pipelined_receive() are identical, so merge them. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- ipc/mqueue.c | 31 ++- 1 file changed, 18 insertions(+), 13 deletions(-) diff --git a/ipc/mqueue.c b/ipc

[PATCH 4/6] ipc/msg.c: Update and document memory barriers.

2019-10-11 Thread Manfred Spraul
refount may have been decreased to 0 already when wake_q_add() tries to get a reference. Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/msg.c | 44 ++-- 1 file changed, 38 insertions(+), 6 deletions(-) diff --git a/ipc/msg.c b

[PATCH 5/6] ipc/sem.c: Document and update memory barriers

2019-10-11 Thread Manfred Spraul
Add comments to all barriers, and mention the rules in the block regarding locking. Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/sem.c | 63 --- 1 file changed, 51 insertions(+), 12 deletions(-) diff --git a/ipc/sem.

[PATCH 6/6] Documentation/memory-barriers.txt: Clarify cmpxchg()

2019-10-11 Thread Manfred Spraul
tomic_add(); 2) smp_mb__before_atomic(); atomic_xchg_relaxed(); 3) smp_mb__before_atomic(); atomic_fetch_add_relaxed(); Invalid would be: smp_mb__before_atomic(); atomic_set(); Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso Cc:

[PATCH 3/6] ipc/mqueue.c: Update/document memory barriers

2019-10-11 Thread Manfred Spraul
() is the RELEASE - CPU3: smp_mb__after_spinlock() inside try_to_wake_up() is the ACQUIRE Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/mqueue.c | 32 +--- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/ipc/mqueue.c b/ipc

[PATCH 1/6] wake_q: Cleanup + Documentation update.

2019-10-11 Thread Manfred Spraul
simplify searching. 4) wake_q.next is accessed without synchroniyation by wake_q_add(), using cmpxchg_relaxed(), and by wake_up_q(). Therefore: Use WRITE_ONCE in wake_up_q(), to ensure that the compiler doesn't perform any tricks. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- k

[PATCH 0/6] V2: Clarify/standardize memory barriers for ipc

2019-10-11 Thread Manfred Spraul
Hi, Updated series, based on input from Davidlohr: - Mixing WRITE_ONCE(), when not holding a lock, and "normal" writes, when holding a lock, makes the code less readable. Thus use _ONCE() everywhere, for both WRITE_ONCE() and READ_ONCE(). - According to my understanding, wake_q_add() does no

Re: [PATCH 2/5] ipc/mqueue.c: Update/document memory barriers

2019-10-11 Thread Manfred Spraul
On 10/11/19 6:55 PM, Davidlohr Bueso wrote: On Fri, 11 Oct 2019, Manfred Spraul wrote: Update and document memory barriers for mqueue.c: - ewp->state is read without any locks, thus READ_ONCE is required. In general we relied on the barrier for not needing READ/WRITE_ONCE, but I agree t

[PATCH 2/5] ipc/mqueue.c: Update/document memory barriers

2019-10-11 Thread Manfred Spraul
add() - document why __set_current_state() may be used: Reading task->state cannot happen before the wake_q_add() call, which happens while holding info->lock. Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/mqueue.c | 32 +--

[PATCH 3/5] ipc/msg.c: Update and document memory barriers.

2019-10-11 Thread Manfred Spraul
code relies on the barrier inside wake_q_add(). Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/msg.c | 39 ++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/ipc/msg.c b/ipc/msg.c index 8dec945fa030..1e2c0a3d4998

[PATCH 4/5] ipc/sem.c: Document and update memory barriers

2019-10-11 Thread Manfred Spraul
D_ONCE & WRITE_ONCE for the situations where spinlocks provide exclusion. - Add comments to all barriers, and mention the rules in the block regarding locking. Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso --- ipc/sem.c | 64

[PATCH 5/5] Documentation/memory-barriers.txt: Clarify cmpxchg()

2019-10-11 Thread Manfred Spraul
tomic_add(); 2) smp_mb__before_atomic(); atomic_xchg_relaxed(); 3) smp_mb__before_atomic(); atomic_fetch_add_relaxed(); Invalid would be: smp_mb__before_atomic(); atomic_set(); Signed-off-by: Manfred Spraul Cc: Waiman Long Cc: Davidlohr Bueso Cc:

[PATCH 0/3] Clarify/standardize memory barriers for ipc

2019-10-11 Thread Manfred Spraul
Hi, Partially based on the findings from Waiman Long: a) The memory barriers in ipc are not properly documented, and at least for some architectures insufficient: Reading the xyz->status is only a control barrier, thus smp_acquire__after_ctrl_dep() was missing in mqueue.c and msg.c sem.c containe

[PATCH 1/5] wake_q: Cleanup + Documentation update.

2019-10-11 Thread Manfred Spraul
doesn't perform any tricks. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- kernel/sched/core.c | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index dd05a378631a..2cf3f7321303 100644 --- a/kernel/sched/core.c

Re: wake_q memory ordering

2019-10-11 Thread Manfred Spraul
Hi Davidlohr, On 10/10/19 9:25 PM, Davidlohr Bueso wrote: On Thu, 10 Oct 2019, Peter Zijlstra wrote: On Thu, Oct 10, 2019 at 02:13:47PM +0200, Manfred Spraul wrote: Therefore smp_mb__{before,after}_atomic() may be combined with cmpxchg_relaxed, to form a full memory barrier, on all archs

Re: wake_q memory ordering

2019-10-10 Thread Manfred Spraul
Hi Peter, On 10/10/19 1:42 PM, Peter Zijlstra wrote: On Thu, Oct 10, 2019 at 12:41:11PM +0200, Manfred Spraul wrote: Hi, Waiman Long noticed that the memory barriers in sem_lock() are not really documented, and while adding documentation, I ended up with one case where I'm not certain

wake_q memory ordering

2019-10-10 Thread Manfred Spraul
Hi, Waiman Long noticed that the memory barriers in sem_lock() are not really documented, and while adding documentation, I ended up with one case where I'm not certain about the wake_q code: Questions: - Does smp_mb__before_atomic() + a (failed) cmpxchg_relaxed provide an   ordering guarante

Re: [PATCH] ipc/sem: Fix race between to-be-woken task and waker

2019-09-29 Thread Manfred Spraul
Hi Waiman, I have now written the mail 3 times: Twice I thought that I found a race, but during further analysis, it always turns out that the spin_lock() is sufficient. First, to avoid any obvious things: Until the series with e.g. 27d7be1801a4824e, there was a race inside sem_lock(). Thus

Re: [PATCH] ipc/sem: Fix race between to-be-woken task and waker

2019-09-26 Thread Manfred Spraul
Hi, On 9/26/19 8:12 PM, Waiman Long wrote: On 9/26/19 5:34 AM, Peter Zijlstra wrote: On Fri, Sep 20, 2019 at 11:54:02AM -0400, Waiman Long wrote: While looking at a customr bug report about potential missed wakeup in the system V semaphore code, I spot a potential problem. The fact that semaph

Re: [PATCH v11 2/3] ipc: Conserve sequence numbers in ipcmni_extend mode

2019-03-10 Thread Manfred Spraul
On 2/27/19 9:30 PM, Waiman Long wrote: On 11/20/2018 02:41 PM, Manfred Spraul wrote: From 6bbade73d21884258a995698f21ad3128df8e98a Mon Sep 17 00:00:00 2001 From: Manfred Spraul Date: Sat, 29 Sep 2018 15:43:28 +0200 Subject: [PATCH 2/2] ipc/util.c: use idr_alloc_cyclic() for ipc allocations A

Re: general protection fault in put_pid

2019-01-07 Thread Manfred Spraul
On 1/3/19 11:18 PM, Shakeel Butt wrote: Hi Manfred, On Sun, Dec 23, 2018 at 4:26 AM Manfred Spraul wrote: Hello Dmitry, On 12/23/18 10:57 AM, Dmitry Vyukov wrote: I can reproduce this infinite memory consumption with the C program: https://gist.githubusercontent.com/dvyukov

Re: general protection fault in put_pid

2018-12-23 Thread Manfred Spraul
Hi Dmitry, let's simplify the mail, otherwise noone can follow: On 12/23/18 11:42 AM, Dmitry Vyukov wrote: My naive attempts to re-reproduce this failed so far. But I noticed that _all_ logs for these 3 crashes: https://syzkaller.appspot.com/bug?extid=c92d3646e35bc5d1a909 https://syzkaller.ap

Re: general protection fault in put_pid

2018-12-23 Thread Manfred Spraul
Hello Dmitry, On 12/23/18 10:57 AM, Dmitry Vyukov wrote: I can reproduce this infinite memory consumption with the C program: https://gist.githubusercontent.com/dvyukov/03ec54b3429ade16fa07bf8b2379aff3/raw/ae4f654e279810de2505e8fa41b73dc1d8e6/gistfile1.txt But this is working as intended,

Re: general protection fault in put_pid

2018-12-22 Thread Manfred Spraul
Hi Dmitry, On 12/20/18 4:36 PM, Dmitry Vyukov wrote: On Wed, Dec 19, 2018 at 10:04 AM Manfred Spraul wrote: Hello Dmitry, On 12/12/18 11:55 AM, Dmitry Vyukov wrote: On Tue, Dec 11, 2018 at 9:23 PM syzbot wrote: Hello, syzbot found the following crash on: HEAD commit:f5d582777bcb

Re: general protection fault in put_pid

2018-12-19 Thread Manfred Spraul
? I would continue to search, and then send a series with all findings. --     Manfred >From 733e888993b71fb3c139f71de61534bc603a2bcb Mon Sep 17 00:00:00 2001 From: Manfred Spraul Date: Wed, 19 Dec 2018 09:26:48 +0100 Subject: [PATCH] ipc/sem.c: ensure proper locking during namespace teardo

Re: BUG: corrupted list in freeary

2018-12-01 Thread Manfred Spraul
Hi Dmitry, On 11/30/18 6:58 PM, Dmitry Vyukov wrote: On Thu, Nov 29, 2018 at 9:13 AM, Manfred Spraul wrote: Hello together, On 11/27/18 4:52 PM, syzbot wrote: Hello, syzbot found the following crash on: HEAD commit:e195ca6cb6f2 Merge branch 'for-linus' of git://git.kernel..

Re: [RFC, PATCH] ipc/util.c: use idr_alloc_cyclic() for ipc allocations

2018-10-03 Thread Manfred Spraul
On 10/2/18 8:27 PM, Waiman Long wrote: On 10/02/2018 12:19 PM, Manfred Spraul wrote: A bit related to the patch series that increases IPC_MNI: (User space) id reuse create the risk of data corruption: Process A: calls ipc function Process A: sleeps just at the beginning of the syscall Process

[RFC, PATCH] ipc/util.c: use idr_alloc_cyclic() for ipc allocations

2018-10-02 Thread Manfred Spraul
ids. Signed-off-by: Manfred Spraul --- Open questions: - Is there a significant performance advantage, especially there are many ipc ids? - Over how many ids should the code cycle always? - Further review remarks? ipc/util.c | 22 +- 1 file changed, 21 insertions(+), 1

Re: [PATCH -next] ipc/sem: prevent queue.status tearing in semop

2018-07-17 Thread Manfred Spraul
Hello Davidlohr, On 07/17/2018 07:26 AM, Davidlohr Bueso wrote: In order for load/store tearing to work, _all_ accesses to the variable in question need to be done around READ and WRITE_ONCE() macros. Ensure everyone does so for q->status variable for semtimedop(). What is the background of the

[PATCH 01/12] ipc: ipc: compute kern_ipc_perm.id under the ipc lock.

2018-07-12 Thread Manfred Spraul
finding of syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com: syzbot found an issue with kern_ipc_perm.seq Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Reviewed-by: Davidlohr Bueso --- ipc/msg.c | 19 ++- ipc/sem.c | 18 +- ipc/shm.c | 19

[PATCH 03/12] ipc/util.c: Use ipc_rcu_putref() for failues in ipc_addid()

2018-07-12 Thread Manfred Spraul
: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 ++ ipc/util.c | 10 -- 4 files changed, 12 insertions(+), 4 deletions(-) diff --git a/ipc/msg.c b/ipc/msg.c index 49358f474fc9..38119c1f0da3 100644 --- a/ipc

[PATCH 04/12] ipc: Rename ipcctl_pre_down_nolock().

2018-07-12 Thread Manfred Spraul
ter in the idr, without acquiring the object lock. - The caller is responsible for locking. - _check means that the sequence number is checked. Signed-off-by: Manfred Spraul Reviewed-by: Davidlohr Bueso --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 +- ipc/util.c | 8 ipc/

[PATCH 05/12] ipc/util.c: correct comment in ipc_obtain_object_check

2018-07-12 Thread Manfred Spraul
=true may disappear at the end of the next rcu grace period. Signed-off-by: Manfred Spraul Reviewed-by: Davidlohr Bueso --- ipc/util.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ipc/util.c b/ipc/util.c index cffd12240f67..5cc37066e659 100644 --- a/ipc/util.c +++ b/ipc

[PATCH 06/12] ipc: drop ipc_lock()

2018-07-12 Thread Manfred Spraul
, we can simply move the logic into shm_lock() and get rid of the function altogether. [changelog mostly by manfred] Signed-off-by: Davidlohr Bueso Signed-off-by: Manfred Spraul --- ipc/shm.c | 29 +++-- ipc/util.c | 36 ipc/util.h

[PATCH 08/12] lib/rhashtable: guarantee initial hashtable allocation

2018-07-12 Thread Manfred Spraul
resizing (when more memory becomes available) is the least of our problems. Signed-off-by: Davidlohr Bueso Acked-by: Herbert Xu Signed-off-by: Manfred Spraul --- lib/rhashtable.c | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/lib/rhashtable.c b/lib/rhashtable.c

[PATCH 07/12] lib/rhashtable: simplify bucket_table_alloc()

2018-07-12 Thread Manfred Spraul
a positive consequence as for the same reasons we want nowarn semantics in bucket_table_alloc(). Signed-off-by: Davidlohr Bueso Acked-by: Michal Hocko (commit id extended to 12 digits, line wraps updated) Signed-off-by: Manfred Spraul --- lib/rhashtable.c | 7 ++- 1 file changed, 2 insert

[PATCH 09/12] ipc: get rid of ids->tables_initialized hack

2018-07-12 Thread Manfred Spraul
c structure isn't found, and can therefore call into ipcget() callbacks. Now that rhashtable initialization cannot fail, we can properly get rid of the hack altogether. Signed-off-by: Davidlohr Bueso (commit id extended to 12 digits) Signed-off-by: Manfred Spraul --- include/linux/ipc_na

[PATCH 10/12] ipc: simplify ipc initialization

2018-07-12 Thread Manfred Spraul
From: Davidlohr Bueso Now that we know that rhashtable_init() will not fail, we can get rid of a lot of the unnecessary cleanup paths when the call errored out. Signed-off-by: Davidlohr Bueso (variable name added to util.h to resolve checkpatch warning) Signed-off-by: Manfred Spraul --- ipc

[PATCH 11/12] ipc/util.c: Further variable name cleanups

2018-07-12 Thread Manfred Spraul
e variable name. seq: sequence number, to avoid quick collisions of the user space id key: user space key, used for the rhash tree Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov --- include/linux/ipc_namespace.h | 2 +- ipc/msg.c | 6 +++--- ipc/sem.c |

[PATCH 12/12] ipc/util.c: update return value of ipc_getref from int to bool

2018-07-12 Thread Manfred Spraul
ions. Signed-off-by: Manfred Spraul --- ipc/util.c | 2 +- ipc/util.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/ipc/util.c b/ipc/util.c index fb69c911655a..6306eb25180b 100644 --- a/ipc/util.c +++ b/ipc/util.c @@ -461,7 +461,7 @@ void ipc_set_key_private(struct

[PATCH 0/12 V3] ipc: cleanups & bugfixes, rhashtable update

2018-07-12 Thread Manfred Spraul
Hi, I have added all all review findings and rediffed the patches - patch #1-#6: Fix syzcall findings & further race cleanups patch #1 has an updated subject/comment patch #2 contains the squashed result of Dmitrys change and my own updates. patch #6 is replaced by

[PATCH 02/12] ipc: reorganize initialization of kern_ipc_perm.seq

2018-07-12 Thread Manfred Spraul
ipc_addid() do not fullfill that, i.e. more bugfixes are required. The patch is a squash of a patch from Dmitry and my own changes. Reported-by: syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso Cc: Michael

Re: [PATCH 0/12 V2] ipc: cleanups & bugfixes, rhashtable update

2018-07-09 Thread Manfred Spraul
Hi Davidlohr, On 07/09/2018 10:09 PM, Davidlohr Bueso wrote: On Mon, 09 Jul 2018, Manfred Spraul wrote: @Davidlohr: Please double check that I have taken the correct patches, and that I didn't break anything. Everything seems ok. Patch 8 had an alternative patch that didn't cha

Re: [PATCH 12/12] ipc/util.c: Further ipc_idr_alloc cleanups.

2018-07-09 Thread Manfred Spraul
Hello Dmitry, On 07/09/2018 07:05 PM, Dmitry Vyukov wrote: On Mon, Jul 9, 2018 at 5:10 PM, Manfred Spraul wrote: If idr_alloc within ipc_idr_alloc fails, then the return value (-ENOSPC) is used to calculate new->id. Technically, this is not a bug, because new->id is never accessed

[PATCH 01/12] ipc: reorganize initialization of kern_ipc_perm.id

2018-07-09 Thread Manfred Spraul
syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com: syzbot found an issue with kern_ipc_perm.seq Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso --- ipc/msg.c | 19 ++- ipc/sem.c | 18 +- ipc/shm.c | 19 ++- 3

[PATCH 07/12] ipc_idr_alloc refactoring

2018-07-09 Thread Manfred Spraul
From: Dmitry Vyukov ipc_idr_alloc refactoring Signed-off-by: Dmitry Vyukov Signed-off-by: Manfred Spraul --- ipc/util.c | 51 +-- 1 file changed, 13 insertions(+), 38 deletions(-) diff --git a/ipc/util.c b/ipc/util.c index 8bc166bb4981

[PATCH 09/12] lib/rhashtable: guarantee initial hashtable allocation

2018-07-09 Thread Manfred Spraul
resizing (when more memory becomes available) is the least of our problems. Signed-off-by: Davidlohr Bueso Acked-by: Herbert Xu Signed-off-by: Manfred Spraul --- lib/rhashtable.c | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/lib/rhashtable.c b/lib/rhashtable.c

[PATCH 08/12] lib/rhashtable: simplify bucket_table_alloc()

2018-07-09 Thread Manfred Spraul
a positive consequence as for the same reasons we want nowarn semantics in bucket_table_alloc(). Signed-off-by: Davidlohr Bueso Acked-by: Michal Hocko (commit id extended to 12 digits, line wraps updated) Signed-off-by: Manfred Spraul --- lib/rhashtable.c | 7 ++- 1 file changed, 2 insert

[PATCH 06/12] ipc: rename ipc_lock() to ipc_lock_idr()

2018-07-09 Thread Manfred Spraul
does not check the sequence counter. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- ipc/shm.c | 4 ++-- ipc/util.c | 10 ++ ipc/util.h | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/ipc/shm.c b/ipc/shm.c index 426ba1039a7b..cd8655c7bb77 100644 --- a/ipc

[PATCH 10/12] ipc: get rid of ids->tables_initialized hack

2018-07-09 Thread Manfred Spraul
c structure isn't found, and can therefore call into ipcget() callbacks. Now that rhashtable initialization cannot fail, we can properly get rid of the hack altogether. Signed-off-by: Davidlohr Bueso (commit id extended to 12 digits) Signed-off-by: Manfred Spraul --- include/linux/ipc_na

[PATCH 11/12] ipc: simplify ipc initialization

2018-07-09 Thread Manfred Spraul
From: Davidlohr Bueso Now that we know that rhashtable_init() will not fail, we can get rid of a lot of the unnecessary cleanup paths when the call errored out. Signed-off-by: Davidlohr Bueso (variable name added to util.h to resolve checkpatch warning) Signed-off-by: Manfred Spraul --- ipc

[PATCH 04/12] ipc: Rename ipcctl_pre_down_nolock().

2018-07-09 Thread Manfred Spraul
ter in the idr, without acquiring the object lock. - The caller is responsible for locking. - _check means that the sequence number is checked. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 +- ipc/util.c | 8 ipc/util.h | 2 +

[PATCH 12/12] ipc/util.c: Further ipc_idr_alloc cleanups.

2018-07-09 Thread Manfred Spraul
If idr_alloc within ipc_idr_alloc fails, then the return value (-ENOSPC) is used to calculate new->id. Technically, this is not a bug, because new->id is never accessed. But: Clean it up anyways: On error, just return, do not set new->id. And improve the documentation. Signed-off-by

[PATCH 03/12] ipc/util.c: Use ipc_rcu_putref() for failues in ipc_addid()

2018-07-09 Thread Manfred Spraul
: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 ++ ipc/util.c | 12 ++-- 4 files changed, 14 insertions(+), 4 deletions(-) diff --git a/ipc/msg.c b/ipc/msg.c index 829c2062ded4..5bf5cb8017ea 100644 --- a/ipc

[PATCH 0/12 V2] ipc: cleanups & bugfixes, rhashtable update

2018-07-09 Thread Manfred Spraul
Hi, I have merged the patches from Dmitry, Davidlohr and myself: - patch #1-#6: Fix syzcall findings & further race cleanups - patch #7: Cleanup from Dmitry for ipc_idr_alloc. - patch #8-#11: rhashtable improvement from Davidlohr - patch #12: Another cleanup for ipc_idr_alloc. @Davidlohr: Please

[PATCH 05/12] ipc/util.c: correct comment in ipc_obtain_object_check

2018-07-09 Thread Manfred Spraul
=true may disappear at the end of the next rcu grace period. Signed-off-by: Manfred Spraul Cc: Davidlohr Bueso --- ipc/util.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ipc/util.c b/ipc/util.c index bbb1ce212a0d..8133f10832a9 100644 --- a/ipc/util.c +++ b/ipc/util.c

[PATCH 02/12] ipc: reorganize initialization of kern_ipc_perm.seq

2018-07-09 Thread Manfred Spraul
not fullfill that, i.e. more bugfixes are required. Reported-by: syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso Cc: Michael Kerrisk --- Documentation/sysctl/kernel.txt | 3 ++- ipc/util.c

Re: [PATCH 2/6] ipc: reorganize initialization of kern_ipc_perm.seq

2018-07-05 Thread Manfred Spraul
Hi Dmitry, On 07/05/2018 10:36 AM, Dmitry Vyukov wrote: [...] Hi Manfred, The series looks like a significant improvement to me. Thanks! I feel that this code can be further simplified (unless I am missing something here). Please take a look at this version: https://github.com/dvyukov/linux/c

[PATCH 3/6] ipc/util.c: Use ipc_rcu_putref() for failues in ipc_addid()

2018-07-04 Thread Manfred Spraul
: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 ++ ipc/util.c | 12 ++-- 4 files changed, 14 insertions(+), 4 deletions(-) diff --git a/ipc/msg.c b/ipc/msg.c index 829c2062ded4..5bf5cb8017ea 100644 --- a/ipc

[PATCH 2/6] ipc: reorganize initialization of kern_ipc_perm.seq

2018-07-04 Thread Manfred Spraul
not fullfill that, i.e. more bugfixes are required. Reported-by: syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso Cc: Michael Kerrisk Signed-off-by: Manfred Spraul --- Documentation/sysctl/kernel.txt | 3

[PATCH 6/6] ipc/util.c: correct comment in ipc_obtain_object_check

2018-07-04 Thread Manfred Spraul
=true may disappear at the end of the next rcu grace period. Signed-off-by: Manfred Spraul --- ipc/util.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/ipc/util.c b/ipc/util.c index 4f2db913acf9..776a9ce2905f 100644 --- a/ipc/util.c +++ b/ipc/util.c @@ -646,8 +646,8

[PATCH 4/6] ipc: Rename ipcctl_pre_down_nolock().

2018-07-04 Thread Manfred Spraul
ter in the idr, without acquiring the object lock. - The caller is responsible for locking. - _check means that some checks are made. Signed-off-by: Manfred Spraul --- ipc/msg.c | 2 +- ipc/sem.c | 2 +- ipc/shm.c | 2 +- ipc/util.c | 6 +++--- ipc/util.h | 2 +- 5 files changed, 7 insert

[PATCH 5/6] ipc: rename ipc_lock() to ipc_lock_idr()

2018-07-04 Thread Manfred Spraul
does not check the sequence counter. Signed-off-by: Manfred Spraul --- ipc/shm.c | 4 ++-- ipc/util.c | 10 ++ ipc/util.h | 2 +- 3 files changed, 9 insertions(+), 7 deletions(-) diff --git a/ipc/shm.c b/ipc/shm.c index 426ba1039a7b..cd8655c7bb77 100644 --- a/ipc/shm.c +++ b/ipc/shm.c

[PATCH 1/6] ipc: reorganize initialization of kern_ipc_perm.id

2018-07-04 Thread Manfred Spraul
syzbot+2827ef6b3385deb07...@syzkaller.appspotmail.com: syzbot found an issue with kern_ipc_perm.seq Signed-off-by: Manfred Spraul Cc: Dmitry Vyukov Cc: Kees Cook Cc: Davidlohr Bueso Signed-off-by: Manfred Spraul --- ipc/msg.c | 19 ++- ipc/sem.c | 18 +- ipc/shm.c

[PATCH 0/5] ipc: cleanups & bugfixes

2018-07-04 Thread Manfred Spraul
Hi, Dmitry convinced me that I should properly review the initialization of new ipc objects, and I found another issue. The series corrects 3 issues with ipc_addid(), and also renames two functions and corrects a wrong comment. 0001-ipc-reorganize-initialization-of-kern_ipc_perm.id: Acce

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Manfred Spraul
Hello Dmitry, On 07/04/2018 12:03 PM, Dmitry Vyukov wrote: On Wed, Jul 4, 2018 at 11:18 AM, Manfred Spraul wrote: There are 2 relevant values: kern_ipc_perm.id and kern_ipc_perm.seq. For kern_ipc_perm.id, it is possible to move the access to the codepath that hold the lock. For

[RFC] ipc: refcounting / use after free?

2018-07-04 Thread Manfred Spraul
The ipc code uses the equivalent of rcu_read_lock(); kfree_rcu(a, rcu); if (a->deleted) { rcu_read_unlock(); return FAILURE; } <...> Is this safe, or is dereferencing "a" after having called call_rcu() a use-after-free? Acco

Re: ipc/msg: zalloc struct msg_queue when creating a new msq

2018-07-04 Thread Manfred Spraul
oyed, then the newly created object has temporarily sequence number 0 as well. --     Manfred >From 4791e604dcb618ed7ea1f42b2f6ca9cfe3c113c3 Mon Sep 17 00:00:00 2001 From: Manfred Spraul Date: Wed, 4 Jul 2018 10:04:49 +0200 Subject: [PATCH] ipc: fix races with kern_ipc_perm.id and .seq

Re: [REVIEW][PATCH 11/11] ipc/sem: Fix semctl(..., GETPID, ...) between pid namespaces

2018-04-02 Thread Manfred Spraul
Hi, On 03/30/2018 09:09 PM, Davidlohr Bueso wrote: On Wed, 28 Mar 2018, Davidlohr Bueso wrote: On Fri, 23 Mar 2018, Eric W. Biederman wrote: Today the last process to update a semaphore is remembered and reported in the pid namespace of that process.  If there are processes in any other pid

Re: [RFC][PATCH] ipc: Remove IPCMNI

2018-03-29 Thread Manfred Spraul
Hello Mathew, On 03/29/2018 12:56 PM, Matthew Wilcox wrote: On Thu, Mar 29, 2018 at 10:47:45AM +0200, Manfred Spraul wrote: This can be implemented trivially with the current code using idr_alloc_cyclic. Is there a performance impact? Right now, the idr tree is only large if there are lots of

Re: [RFC][PATCH] ipc: Remove IPCMNI

2018-03-29 Thread Manfred Spraul
Hello together, On 03/29/2018 04:14 AM, Davidlohr Bueso wrote: Cc'ing mtk, Manfred and linux-api. See below. On Thu, 15 Mar 2018, Waiman Long wrote: On 03/15/2018 03:00 PM, Eric W. Biederman wrote: Waiman Long writes: On 03/14/2018 08:49 PM, Eric W. Biederman wrote: The define IPCMNI was

Re: stable/linux-3.16.y build: 178 builds: 1 failed, 177 passed, 2 errors, 57 warnings (v3.16.52)

2018-01-13 Thread Manfred Spraul
Hi Arnd, On 01/03/2018 12:15 AM, Arnd Bergmann wrote: 2 ipc/sem.c:377:6: warning: '___p1' may be used uninitialized in this function [-Wmaybe-uninitialized] This code was last touched in 3.16 by the backport of commit 5864a2fd3088 ("ipc/sem.c: fix complex_count vs. simple op race") The warn

Re: BUG: unable to handle kernel paging request in ipcget

2017-12-23 Thread Manfred Spraul
Hi, On 12/23/2017 08:33 AM, syzbot wrote: Hello, syzkaller hit the following crash on 6084b576dca2e898f5c101baef151f7bfdbb606d git://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/master compiler: gcc (GCC) 7.1.1 20170620 .config is attached Raw console output is attached. Unfor

Re: shmctl(SHM_STAT) vs. /proc/sysvipc/shm permissions discrepancies

2017-12-20 Thread Dr. Manfred Spraul
Hi Michal, On 12/19/2017 10:48 AM, Michal Hocko wrote: Hi, we have been contacted by our partner about the following permission discrepancy 1. Create a shared memory segment with permissions 600 with user A using shmget(key, 1024, 0600 | IPC_CREAT) 2. ipcs -m should return an output as follo

Re: [PATCH 2/2] ipc: Fix ipc data structures inconsistency

2017-12-01 Thread Manfred Spraul
Hi, On 12/01/2017 06:20 PM, Davidlohr Bueso wrote: On Thu, 30 Nov 2017, Philippe Mikoyan wrote: As described in the title, this patch fixes id_ds inconsistency when ctl_stat runs concurrently with some ds-changing function, e.g. shmat, msgsnd or whatever. For instance, if shmctl(IPC_STAT) is

Re: [PATCH v2 0/9] Remove spin_unlock_wait()

2017-07-10 Thread Manfred Spraul
Hi Alan, On 07/08/2017 06:21 PM, Alan Stern wrote: Pardon me for barging in, but I found this whole interchange extremely confusing... On Sat, 8 Jul 2017, Ingo Molnar wrote: * Paul E. McKenney wrote: On Sat, Jul 08, 2017 at 10:35:43AM +0200, Ingo Molnar wrote: * Manfred Spraul wrote

Re: [PATCH v2 0/9] Remove spin_unlock_wait()

2017-07-07 Thread Manfred Spraul
Hi Ingo, On 07/07/2017 10:31 AM, Ingo Molnar wrote: There's another, probably just as significant advantage: queued_spin_unlock_wait() is 'read-only', while spin_lock()+spin_unlock() dirties the lock cache line. On any bigger system this should make a very measurable difference - if spin_unloc

Re: [PATCH v2 1/9] net/netfilter/nf_conntrack_core: Fix net_conntrack_lock()

2017-07-06 Thread Manfred Spraul
Hi Paul, On 07/06/2017 01:31 AM, Paul E. McKenney wrote: From: Manfred Spraul As we want to remove spin_unlock_wait() and replace it with explicit spin_lock()/spin_unlock() calls, we can use this to simplify the locking. In addition: - Reading nf_conntrack_locks_all needs ACQUIRE memory

Re: [PATCH RFC 01/26] netfilter: Replace spin_unlock_wait() with lock/unlock pair

2017-07-06 Thread Manfred Spraul
the hysteresis code, the slow path is at least factor 10 rarer than it was before. Especially: Who is able to test it? Signed-off-by: Manfred Spraul Cc: Alan Stern --- ipc/sem.c | 33 +++-- 1 file changed, 19 insertions(+), 14 deletions(-) diff --git a/ipc/sem.

Re: [PATCH RFC 01/26] netfilter: Replace spin_unlock_wait() with lock/unlock pair

2017-07-03 Thread Manfred Spraul
On 07/03/2017 07:14 PM, Paul E. McKenney wrote: On Mon, Jul 03, 2017 at 10:39:49AM -0400, Alan Stern wrote: On Sat, 1 Jul 2017, Manfred Spraul wrote: As we want to remove spin_unlock_wait() and replace it with explicit spin_lock()/spin_unlock() calls, we can use this to simplify the locking

Re: [PATCH RFC 06/26] ipc: Replace spin_unlock_wait() with lock/unlock pair

2017-07-01 Thread Manfred Spraul
lowed immediately by spin_unlock(). This should be safe from a performance perspective because exit_sem() is rarely invoked in production. Signed-off-by: Paul E. McKenney Cc: Andrew Morton Cc: Davidlohr Bueso Cc: Manfred Spraul Cc: Will Deacon Cc: Peter Zijlstra Cc: Alan Stern Cc: Andrea

Re: [PATCH v2] refcount: Create unchecked atomic_t implementation

2017-06-08 Thread Manfred Spraul
Hi Davidlohr, On 06/08/2017 10:09 PM, Davidlohr Bueso wrote: Yes, this would be a prerequisite for ipc; which I initially thought didn't take a performance hit. Did you see a regression for ipc? The fast paths don't use the refcount, it is only used for rare situations: - GETALL, SETALL fo

Re: [PATCH 1/3] ipc: convert ipc_namespace.count from atomic_t to refcount_t

2017-05-28 Thread Manfred Spraul
On 05/27/2017 09:41 PM, Kees Cook wrote: On Mon, Feb 20, 2017 at 3:29 AM, Elena Reshetova wrote: refcount_t type and corresponding API should be used instead of atomic_t when the variable is used as a reference counter. This allows to avoid accidental refcounter overflows that might lead to use

Re: [PATCH 0/20 V3] Misc cleanups for ipc

2017-05-25 Thread Manfred Spraul
Hi Kees, On 05/25/2017 09:45 PM, Kees Cook wrote: On Thu, May 25, 2017 at 11:50 AM, Manfred Spraul wrote: Hi all, Updated series. The series got longer, because I merged all patches from Kees. Main changes are: - sems[] instead of sem[0]. - Immediately use BUILD_BUG_ON() - Immediately move

  1   2   3   4   5   6   7   >