On 2024/6/1 08:31, Jeff Johnson wrote:
make allmodconfig && make W=1 C=1 reports:
WARNING: modpost: missing MODULE_DESCRIPTION() in lib/test_objpool.o
Add the missing invocation of the MODULE_DESCRIPTION() macro.
Signed-off-by: Jeff Johnson
---
lib/test_objpool.c | 3 ++-
1 file changed, 2
Hello Andi,
On 2023/11/23 06:17, Andi Shyti wrote:
Hi Wuqiang,
On Tue, Nov 21, 2023 at 10:23:43PM +0800, wuqiang.matt wrote:
arch_cmpxchg() should check data size rather than pointer size in case
CONFIG_ARC_HAS_LLSC is defined. So rename __cmpxchg to __cmpxchg_32 to
emphasize it's exp
The xtensa architecture already has __cmpxchg_local defined but not used.
The purpose of __cmpxchg_local() is solely for arch_cmpxchg_local(), just
as the definition of arch_cmpxchg_local() for other architectures like x86,
arm and powerpc.
Signed-off-by: wuqiang.matt
---
arch/xtensa/include
rwise generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/hexagon/include/asm/cmpxchg.h | 51 +-
1
rwise generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/openrisc/include/asm/cmpxchg.h | 6 ++
1 file changed, 6 inser
rwise generci_cmpxchg[64]_local is to be used.
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-kbuild-all/202310272207.tlpflya4-...@intel.com/
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/arc/include/asm/cmpxchg.h | 28
1
ive cmpxchg variants if avaialble, as Arnd advised
Signed-off-by: wuqiang.matt
Reviewed-by: Masami Hiramatsu (Google)
---
arch/arc/include/asm/cmpxchg.h | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpx
r arc/openrisc/hexagon. Patch 5
defines arch_cmpxchg_local as existing __cmpxchg_local rather the
generic variant.
wuqiang.matt (5):
arch,locking/atomic: arc: arch_cmpxchg should check data size
arch,locking/atomic: arc: add arch_cmpxchg[64]_local
arch,locking/atomic: openrisc: add arch_cmpxc
ative:746406039 1000174750 1493236240 1998318364 2942911180
objpool: 349164852 467284332 702296756 934459713 1387898285
objpool+: 348388180 462750976 696606096 927865887 1368402195
Fixes: b4edb8d2d464 ("lib: objpool added: ring-array based lockless MPMC")
v1 ->
On 2023/11/20 13:18, Masami Hiramatsu (Google) wrote:
On Tue, 14 Nov 2023 19:51:48 +0800
"wuqiang.matt" wrote:
objpool overrun stress with test_objpool on OrangePi5+ SBC triggered the
following kernel warnings:
WARNING: CPU: 6 PID: 3115 at lib/objpool.c:168 objpool_push+0xc0/0
2T 4T 8T16T
native: 30890508 60399915 123111980 242257008 494002946
objpool:14742531 28883047 57739948 115886644 232455421
objpool+: 14107220 29032998 57286084 113730493 232232850
24T32T48T 64T96T
-...@intel.com/
Signed-off-by: wuqiang.matt
---
lib/test_objpool.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/test_objpool.c b/lib/test_objpool.c
index a94078402138..bfdb81599832 100644
--- a/lib/test_objpool.c
+++ b/lib/test_objpool.c
@@ -311,7 +311,7 @@ static
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
Signed-off-by: wuqiang.matt
---
arch/openrisc/include/asm/cmpxchg.h | 6 ++
1 file chang
flya4-...@intel.com/
Signed-off-by: wuqiang.matt
---
arch/hexagon/include/asm/cmpxchg.h | 51 +-
1 file changed, 50 insertions(+), 1 deletion(-)
diff --git a/arch/hexagon/include/asm/cmpxchg.h
b/arch/hexagon/include/asm/cmpxchg.h
index bf6cf5579cf4..2b5e5bbaf807 100644
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
Signed-off-by: wuqiang.matt
---
arch/arc/include/asm/cmpxchg.h | 28 ++
UG_ON checking is uncecessary.
Signed-off-by: wuqiang.matt
---
arch/arc/include/asm/cmpxchg.h | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arc/include/asm/cmpxchg.h b/arch/arc/include/asm/cmpxchg.h
index e138fde067de..bf46514f6f12 100644
--- a/arch/arc/in
This patch series implement arch_cmpxchg[64]_local for arc,
openrisc and hexagon.
For architectures that support native cmpxchg, we'd like to
implement arch_cmpxchg[64]_local with the native variants of
supported data size. If not, the generci_cmpxchg[64]_local
will be used.
wuqiang.ma
On 2023/10/30 01:05, Guenter Roeck wrote:
On Mon, Oct 23, 2023 at 07:24:52PM +0800, wuqiang.matt wrote:
The objpool_push can only happen on local cpu node, so only the local
cpu can touch slot->tail and slot->last, which ensures the correctness
of using cmpxchg without lock prefix
On 2023/10/24 22:52, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local() for arch_cmpxchg_local() implementation
in SH architecture because it does not implement arch_cmpxchg_local().
Reported-by: kernel test robot
Closes:
https://lore.kernel.org/oe-k
On 2023/10/25 09:51, wuqiang.matt wrote:
On 2023/10/25 07:42, Masami Hiramatsu (Google) wrote:
On Tue, 24 Oct 2023 16:08:12 +0100
Mark Rutland wrote:
On Tue, Oct 24, 2023 at 11:52:54PM +0900, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local() for
original loop of all cpu nodes removed
Signed-off-by: wuqiang.matt
---
lib/objpool.c | 26 +-
1 file changed, 9 insertions(+), 17 deletions(-)
diff --git a/lib/objpool.c b/lib/objpool.c
index a032701beccb..7474a3a60cad 100644
--- a/lib/objpool.c
+++ b/lib/objpool.c
On 2023/10/25 07:42, Masami Hiramatsu (Google) wrote:
On Tue, 24 Oct 2023 16:08:12 +0100
Mark Rutland wrote:
On Tue, Oct 24, 2023 at 11:52:54PM +0900, Masami Hiramatsu (Google) wrote:
From: Masami Hiramatsu (Google)
Use generic_cmpxchg_local() for arch_cmpxchg_local() implementation
in SH a
On 2023/10/24 09:01, Masami Hiramatsu (Google) wrote:
On Mon, 23 Oct 2023 11:43:04 -0400
Steven Rostedt wrote:
On Mon, 23 Oct 2023 19:24:52 +0800
"wuqiang.matt" wrote:
The objpool_push can only happen on local cpu node, so only the local
cpu can touch slot->tail and slot
526317 1454991286 1861053557 2059530343 2171732306
Signed-off-by: wuqiang.matt
---
lib/objpool.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/objpool.c b/lib/objpool.c
index ce0087f64400..a032701beccb 100644
--- a/lib/objpool.c
+++ b/lib/objpool.c
@@ -166,7
://lore.kernel.org/oe-kbuild-all/202310191512.vvypku5z-...@intel.com/
Signed-off-by: wuqiang.matt
---
kernel/trace/rethook.c | 2 --
lib/test_objpool.c | 6 --
2 files changed, 8 deletions(-)
diff --git a/kernel/trace/rethook.c b/kernel/trace/rethook.c
index 13c8e6773892..6fd7d4ecbbc6 100644
--- a
On 2023/10/18 10:18, Chengming Zhou wrote:
On 2023/10/17 21:56, wuqiang.matt wrote:
objpool is a scalable implementation of high performance queue for
object allocation and reclamation, such as kretprobe instances.
With leveraging percpu ring-array to mitigate hot spots of memory
contention
This patch will remove freelist.h from kernel source tree, since the
only use cases (kretprobe and rethook) are converted to objpool.
Signed-off-by: wuqiang.matt
---
include/linux/freelist.h | 129 ---
1 file changed, 129 deletions(-)
delete mode 100644
objpool, a scalable and lockless ring-array based object pool, was
introduced to replace the original freelist (a LIFO queue based on
singly linked list) to improve kretprobe scalability.
Signed-off-by: wuqiang.matt
---
MAINTAINERS | 7 +++
1 file changed, 7 insertions(+)
diff --git a
376679233824782945292
objpool: 328192025 439439564 668534502 887401381 1319972072
Signed-off-by: wuqiang.matt
---
include/linux/kprobes.h | 11 ++---
include/linux/rethook.h | 16 ++-
kernel/kprobes.c| 93 +
kernel/trace/fprob
to control the objpool lifecycle
5) group 5: asynchronous mode with hrtimer
hrtimer softirq is introduced to stress async objpool operations
Signed-off-by: wuqiang.matt
---
lib/Kconfig.debug | 11 +
lib/Makefile | 2 +
lib/test_objpool.c | 689
array,
which consumes more memory than linked lists
Signed-off-by: wuqiang.matt
---
include/linux/objpool.h | 176 +
lib/Makefile| 2 +-
lib/objpool.c | 286
3 files changed, 463 insertions(+), 1 deletion
releasing of objpool now covered
wuqiang.matt (5):
lib: objpool added: ring-array based lockless MPMC
lib: objpool test module added
kprobes: kretprobe scalability improvement with objpool
kprobes: freelist.h removed
MAINTAINERS: objpool added
MAINTAINERS | 7 +
include
l for releasing */
+void objpool_fini(struct objpool_head *pool)
+{
+ int count = 1; /* extra ref for objpool itself */
+
+ /* drop all remained objects from objpool */
+ while (objpool_pop(pool))
+ count++;
+
+ if (refcount_sub_and_test(count, &pool->ref)
On 2023/10/16 21:21, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:49 +0800
"wuqiang.matt" wrote:
kretprobe is using freelist to manage return-instances, but freelist,
as LIFO queue based on singly linked list, scales badly and reduces
the overall throughput of
On 2023/10/16 20:18, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 16 Oct 2023 10:45:30 +0800
"wuqiang.matt" wrote:
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hira
On 2023/10/16 20:18, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 16 Oct 2023 10:45:30 +0800
"wuqiang.matt" wrote:
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hira
On 2023/10/16 07:26, Masami Hiramatsu (Google) wrote:
On Mon, 16 Oct 2023 00:06:11 +0800
"wuqiang.matt" wrote:
On 2023/10/15 23:43, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:47 +0800
"wuqiang.matt" wrote:
objpool is a scalable implementation of high p
On 2023/10/15 23:43, Masami Hiramatsu (Google) wrote:
On Sun, 15 Oct 2023 13:32:47 +0800
"wuqiang.matt" wrote:
objpool is a scalable implementation of high performance queue for
object allocation and reclamation, such as kretprobe instances.
With leveraging percpu ring-array to mi
objpool, a scalable and lockless ring-array based object pool, was
introduced to replace the original freelist (a LIFO queue based on
singly linked list) to improve kretprobe scalability.
Signed-off-by: wuqiang.matt
---
MAINTAINERS | 7 +++
1 file changed, 7 insertions(+)
diff --git a
This patch will remove freelist.h from kernel source tree, since the
only use cases (kretprobe and rethook) are converted to objpool.
Signed-off-by: wuqiang.matt
---
include/linux/freelist.h | 129 ---
1 file changed, 129 deletions(-)
delete mode 100644
376679233824782945292
objpool: 328192025 439439564 668534502 887401381 990067903
Signed-off-by: wuqiang.matt
---
include/linux/kprobes.h | 11 ++---
include/linux/rethook.h | 16 ++-
kernel/kprobes.c| 93 +
kernel/trace/fprob
to control the objpool lifecycle
5) group 5: asynchronous mode with hrtimer
hrtimer softirq is introduced to stress async objpool operations
Signed-off-by: wuqiang.matt
---
lib/Kconfig.debug | 11 +
lib/Makefile | 2 +
lib/test_objpool.c | 689
array,
which consumes more memory than linked lists
Signed-off-by: wuqiang.matt
---
include/linux/objpool.h | 176 +
lib/Makefile| 2 +-
lib/objpool.c | 286
3 files changed, 463 insertions(+), 1 deletion
This patch series introduces a scalable and lockless ring-array based
object pool to improve scalability of kretprobed routines.
v10:
*) objpool simplified and improved according to Masami's advices
wuqiang.matt (5):
lib: objpool added: ring-array based lockless MPMC
lib: objpool
On 2023/10/13 09:59, Masami Hiramatsu (Google) wrote:
On Fri, 13 Oct 2023 01:36:05 +0800
"wuqiang.matt" wrote:
On 2023/10/12 22:02, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
On Mon, 9 Oct 2023 17:23:34 +0800
wuqiang wrote:
Hello Masami,
Just got time for the new patch an
return slot->entries[head & slot->mask]
The pre-condition should be: CPU 1 tries to push to a full slot, in this case
tail = head + capacity but tail & mask == head & mask
Regards,
wuqiang
On 2023/9/25 17:42, Masami Hiramatsu (Google) wrote:
Hi Wuqiang,
ref upon
+* objpool was already grabbed during pool
+* initialization in objpool_init()
+ */
+ if (refcount_dec_and_test(&pool->ref))
+ objpool_free(pool);
+ } while (obj);
+}
+EXPORT_SYMBOL_GPL
47 matches
Mail list logo
