[PATCH] vfio/type1: Empty batch for pfnmap pages

2021-03-24 Thread Daniel Jordan
pinning") Link: https://lkml.kernel.org/r/20210323133254.33ed9...@omen.home.shazbot.org/ Reported-by: Alex Williamson Suggested-by: Alex Williamson Signed-off-by: Daniel Jordan --- Alex, I couldn't immediately find a way to trigger this bug, but I can run your test case if you like. This i

Re: [PATCH v2 3/3] vfio/type1: Batch page pinning

2021-03-23 Thread Daniel Jordan
Hi Alex, Alex Williamson writes: > I've found a bug in this patch that we need to fix. The diff is a > little difficult to follow, It was an awful diff, I remember... > so I'll discuss it in the resulting function below... > > (1) Imagine the user has passed a vaddr range that alternates

Re: [PATCH v3] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu onlining

2021-03-18 Thread Daniel Jordan
> callback > sched_cpu_activate() hence it may not be observable by sched_setaffinity() if > it is called immediately after uevent. > > Out of line uevent can be avoided if we will ensure that cpuset_hotplug_work > has run to completion using cpuset_wait_for_hotplug() after onlining th

Re: [PATCH v2] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu onlining

2021-03-18 Thread Daniel Jordan
Alexey Klimov writes: > The first section of comment seems problematic to me with regards to such > move: > > * As this needs to hold the cpu maps lock it's impossible > * to call device_offline() because that ends up calling > * cpu_down()

Re: [PATCH 4/4] sched/cpuacct: Make user/system times in cpuacct.stat more precise

2021-03-17 Thread Daniel Jordan
Andrey Ryabinin writes: > static int cpuacct_stats_show(struct seq_file *sf, void *v) > { ... > for_each_possible_cpu(cpu) { > u64 *cpustat = per_cpu_ptr(ca->cpustat, cpu)->cpustat; > > - val[CPUACCT_STAT_USER] += cpustat[CPUTIME_USER]; > -

Re: [PATCH 3/4] sched/cpuacct: fix user/system in shown cpuacct.usage*

2021-03-17 Thread Daniel Jordan
Andrey Ryabinin writes: > cpuacct has 2 different ways of accounting and showing user > and system times. > > The first one uses cpuacct_account_field() to account times > and cpuacct.stat file to expose them. And this one seems to work ok. > > The second one is uses cpuacct_charge() function

Re: [PATCH 2/4] cgroup: Fix 'usage_usec' time in root's cpu.stat

2021-03-17 Thread Daniel Jordan
_usec looks ok now. Reviewed-by: Daniel Jordan Tested-by: Daniel Jordan

Re: [PATCH 1/4] cputime,cpuacct: Include guest time in user time in cpuacct.stat

2021-03-17 Thread Daniel Jordan
Andrey Ryabinin writes: > cpuacct.stat in no-root cgroups shows user time without guest time > included int it. This doesn't match with user time shown in root > cpuacct.stat and /proc//stat. Yeah, that's inconsistent. > Make account_guest_time() to add user time to cgroup's cpustat to > fix

[PATCH] vfio/type1: fix vaddr_get_pfns() return in vfio_pin_page_external()

2021-03-08 Thread Daniel Jordan
(). Fix it up in vfio_pin_page_external(). Found by inspection. Fixes: be16c1fd99f4 ("vfio/type1: Change success value of vaddr_get_pfn()") Signed-off-by: Daniel Jordan --- I couldn't test this due to lack of hardware. drivers/vfio/vfio_iommu_type1.c | 8 +++- 1 file changed, 7 insert

[PATCH v2 0/3] vfio/type1: Batch page pinning

2021-02-19 Thread Daniel Jordan
KED_VM exceeded I didn't test vfio_pin_page_external() because there was no readily available hardware, but the changes there are pretty minimal. Daniel Jordan (3): vfio/type1: Change success value of vaddr_get_pfn() vfio/type1: Prepare for batched pinning with struct vfio_batch vfio/type1:

[PATCH v2 2/3] vfio/type1: Prepare for batched pinning with struct vfio_batch

2021-02-19 Thread Daniel Jordan
allocate memory. vaddr_get_pfn() becomes vaddr_get_pfns() to prepare for handling multiple pages, though for now only one page is stored in the pages array. Signed-off-by: Daniel Jordan --- drivers/vfio/vfio_iommu_type1.c | 71 +++-- 1 file changed, 58 insertions(+), 13

[PATCH v2 1/3] vfio/type1: Change success value of vaddr_get_pfn()

2021-02-19 Thread Daniel Jordan
. Signed-off-by: Daniel Jordan --- drivers/vfio/vfio_iommu_type1.c | 21 ++--- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index ec9fd95a138b..7abaaad518a6 100644 --- a/drivers/vfio/vfio_iommu_type1.c

[PATCH v2 3/3] vfio/type1: Batch page pinning

2021-02-19 Thread Daniel Jordan
0.48% -0.48% __get_user_pages_remote +0.39% slot_rmap_walk_next +0.32% vfio_pin_map_dma +0.26% kvm_handle_hva_range ... Suggested-by: Matthew Wilcox (Oracle) Signed-off-by: Daniel

Re: [PATCH 3/3] vfio/type1: batch page pinning

2021-02-18 Thread Daniel Jordan
Alex Williamson writes: > This might not be the first batch we fill, I think this needs to unwind > rather than direct return. So it does, well spotted. And it's the same thing with the ENODEV case farther up. > Series looks good otherwise. Thanks for going through it!

Re: [PATCH v2] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu onlining

2021-02-12 Thread Daniel Jordan
Alexey Klimov writes: > int cpu_device_up(struct device *dev) Yeah, definitely better to do the wait here. > int cpuhp_smt_disable(enum cpuhp_smt_control ctrlval) > { > - int cpu, ret = 0; > + struct device *dev; > + cpumask_var_t mask; > + int cpu, ret; > + > + if

Re: [PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu onlining

2021-02-04 Thread Daniel Jordan
Alexey Klimov writes: > When a CPU offlined and onlined via device_offline() and device_online() > the userspace gets uevent notification. If, after receiving "online" uevent, > userspace executes sched_setaffinity() on some task trying to move it > to a recently onlined CPU, then it often fails

Re: [PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu onlining

2021-02-04 Thread Daniel Jordan
Peter Zijlstra writes: > On Thu, Feb 04, 2021 at 12:50:34PM +, Alexey Klimov wrote: >> On Thu, Feb 4, 2021 at 9:46 AM Peter Zijlstra wrote: >> > >> > On Thu, Feb 04, 2021 at 01:01:57AM +, Alexey Klimov wrote: >> > > @@ -1281,6 +1282,11 @@ static int cpu_up(unsigned int cpu, enum >> > >

[PATCH 3/3] vfio/type1: batch page pinning

2021-02-03 Thread Daniel Jordan
0.48% -0.48% __get_user_pages_remote +0.39% slot_rmap_walk_next +0.32% vfio_pin_map_dma +0.26% kvm_handle_hva_range ... Suggested-by: Matthew Wilcox (Oracle) Signed-off-by: Daniel

[PATCH 2/3] vfio/type1: prepare for batched pinning with struct vfio_batch

2021-02-03 Thread Daniel Jordan
allocate memory. vaddr_get_pfn() becomes vaddr_get_pfns() to prepare for handling multiple pages, though for now only one page is stored in the pages array. Signed-off-by: Daniel Jordan --- drivers/vfio/vfio_iommu_type1.c | 71 +++-- 1 file changed, 58 insertions(+), 13

[PATCH 1/3] vfio/type1: change success value of vaddr_get_pfn()

2021-02-03 Thread Daniel Jordan
. Signed-off-by: Daniel Jordan --- drivers/vfio/vfio_iommu_type1.c | 21 ++--- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 0b4dedaa9128..4d608bc552a4 100644 --- a/drivers/vfio/vfio_iommu_type1.c

[PATCH 0/3] vfio/type1: batch page pinning

2021-02-03 Thread Daniel Jordan
lay() -> vfio_pin_pages_remote() Each was run... - with varying sizes - with/without disable_hugepages=1 - with/without LOCKED_VM exceeded I didn't test vfio_pin_page_external() because there was no readily available hardware, but the changes there are pretty minimal. Series based on v5.11-rc6.

Re: [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online

2021-01-20 Thread Daniel Jordan
Alexey Klimov writes: > Daniel, thank you for taking a look. I don't mind reviewing+testing > another approach that you described. Eh, I like yours better :) >> Absent further discussion, Alexey, do you plan to post another version? > > I plan to update this patch and re-send in the next couple

Re: [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online

2021-01-14 Thread Daniel Jordan
Daniel Jordan writes: > Peter Zijlstra writes: >>> The nature of this bug is also described here (with different consequences): >>> https://lore.kernel.org/lkml/20200211141554.24181-1-qais.you...@arm.com/ >> >> Yeah, pesky deadlocks.. someone was going to try ag

Re: [PATCH] cgroup: Remove unnecessary call to strstrip()

2021-01-14 Thread Daniel Jordan
> Content-Transfer-Encoding: 8bit > > The functions cgroup_threads_start and cgroup_procs_start are almost You meant cgroup_threads_write and cgroup_procs_write. > kernel/cgroup/cgroup.c | 55 +++--- > 1 file changed, 14 insertions(+), 41 deletions(-) Ok, sure, that's a good thing. Reviewed-by: Daniel Jordan

Re: [RFC v2 PATCH 0/4] speed up page allocation for __GFP_ZERO

2020-12-22 Thread Daniel Jordan
Liang Li writes: > The first version can be found at: https://lkml.org/lkml/2020/4/12/42 > > Zero out the page content usually happens when allocating pages with > the flag of __GFP_ZERO, this is a time consuming operation, it makes > the population of a large vma area very slowly. This patch

Re: [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online

2020-12-08 Thread Daniel Jordan
Peter Zijlstra writes: >> The nature of this bug is also described here (with different consequences): >> https://lore.kernel.org/lkml/20200211141554.24181-1-qais.you...@arm.com/ > > Yeah, pesky deadlocks.. someone was going to try again. I dug up the synchronous patch

Re: [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online

2020-12-08 Thread Daniel Jordan
Alexey Klimov writes: > I also in doubts if we need cpuset_wait_for_hotplug() in > cpuhp_online_cpu_device() > since an online uevent is sent there too. We do need it there if we go with this fix. Your reproducer hits the same issue when it's changed to exercise smt/control instead of

Re: [PATCH 6/6] mm/gup: migrate pinned pages out of movable zone

2020-12-07 Thread Daniel Jordan
Jason Gunthorpe writes: > On Fri, Dec 04, 2020 at 03:05:46PM -0500, Daniel Jordan wrote: >> Well Alex can correct me, but I went digging and a comment from the >> first type1 vfio commit says the iommu API didn't promise to unmap >> subpages of previous mappings, so doin

Re: [PATCH 6/6] mm/gup: migrate pinned pages out of movable zone

2020-12-07 Thread Daniel Jordan
Pavel Tatashin writes: > On Fri, Dec 4, 2020 at 3:06 PM Daniel Jordan > wrote: >> >> Jason Gunthorpe writes: >> >> > On Wed, Dec 02, 2020 at 08:34:32PM -0500, Pavel Tatashin wrote: >> >> What I meant is the users of the interface do it increme

Re: [PATCH 6/6] mm/gup: migrate pinned pages out of movable zone

2020-12-04 Thread Daniel Jordan
Jason Gunthorpe writes: > On Wed, Dec 02, 2020 at 08:34:32PM -0500, Pavel Tatashin wrote: >> What I meant is the users of the interface do it incrementally not in >> large chunks. For example: >> >> vfio_pin_pages_remote >>vaddr_get_pfn >> ret = pin_user_pages_remote(mm, vaddr, 1,

[tip: sched/core] cpuset: fix race between hotplug work and later CPU offline

2020-11-20 Thread tip-bot2 for Daniel Jordan
The following commit has been merged into the sched/core branch of tip: Commit-ID: 406100f3da08066c00105165db8520bbc7694a36 Gitweb: https://git.kernel.org/tip/406100f3da08066c00105165db8520bbc7694a36 Author:Daniel Jordan AuthorDate:Thu, 12 Nov 2020 12:17:11 -05:00

[PATCH v2] cpuset: fix race between hotplug work and later CPU offline

2020-11-12 Thread Daniel Jordan
cpuset's effective mask, not just the top one. Fixes: 0ccea8feb980 ("cpuset: Make generate_sched_domains() work with partition") Signed-off-by: Daniel Jordan Cc: Johannes Weiner Cc: Li Zefan Cc: Peter Zijlstra Cc: Prateek Sood Cc: Tejun Heo Cc: Waiman Long Cc: cgro...@vger.kernel.org C

Re: [PATCH] cpuset: fix race between hotplug work and later CPU offline

2020-11-10 Thread Daniel Jordan
Peter Zijlstra writes: > On Thu, Oct 29, 2020 at 02:18:45PM -0400, Daniel Jordan wrote: >> rebuild_sched_domains_locked() prevented the race during the cgroup2 >> cpuset series up until the Fixes commit changed its check. Make the >> check more robust so that it can detect

[PATCH] cpuset: fix race between hotplug work and later CPU offline

2020-10-29 Thread Daniel Jordan
cpuset's effective mask, not just the top one. Fixes: 0ccea8feb980 ("cpuset: Make generate_sched_domains() work with partition") Signed-off-by: Daniel Jordan Cc: Johannes Weiner Cc: Li Zefan Cc: Peter Zijlstra Cc: Prateek Sood Cc: Tejun Heo Cc: Waiman Long Cc: cgro...@vger.kernel.org C

Re: [PATCH v2] Remove __init from padata_do_multithreaded and padata_mt_helper.

2020-10-27 Thread Daniel Jordan
On 10/27/20 12:46 AM, Nico Pache wrote: > On Wed, Jul 08, 2020 at 03:51:40PM -0400, Daniel Jordan wrote: > > (I was away for a while) > > > > On Thu, Jul 02, 2020 at 11:55:48AM -0400, Nico Pache wrote: > > > Allow padata_do_multithreaded functio

[PATCH] module: statically initialize init section freeing data

2020-10-08 Thread Daniel Jordan
al flag") Reported-by: Corentin Labbe Tested-by: Corentin Labbe Tested-on: sun50i-h6-pine-h64 Tested-on: imx8mn-ddr4-evk Tested-on: sun50i-a64-bananapi-m64 Signed-off-by: Daniel Jordan --- kernel/module.c | 13 +++-- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/kernel/mod

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-10-08 Thread Daniel Jordan
On Wed, Oct 07, 2020 at 09:41:17PM +0200, Corentin Labbe wrote: > I have added CONFIG_FTRACE=y and your second patch. > The boot log can be seen at http://kernel.montjoie.ovh/108789.log > > But it seems the latest dump_stack addition flood a bit. Heh, sorry for making it spew, there wasn't such

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-10-05 Thread Daniel Jordan
On Thu, Oct 01, 2020 at 07:50:22PM +0200, Corentin Labbe wrote: > On Tue, Mar 03, 2020 at 04:30:17PM -0500, Daniel Jordan wrote: > > Barring other ideas, Corentin, would you be willing to boot with > > > > trace_event=initcall:*,module:* trace_options=stacktrace >

Re: WARNING: at kernel/workqueue.c:1473 __queue_work+0x3b8/0x3d0

2020-09-30 Thread Daniel Jordan
On Fri, Sep 25, 2020 at 08:12:03PM +0200, Corentin Labbe wrote: > On Tue, Mar 03, 2020 at 04:31:11PM -0500, Daniel Jordan wrote: > > On Tue, Mar 03, 2020 at 08:48:19AM +0100, Corentin Labbe wrote: > > > The patch fix the issue. Thanks! > > > > Thanks for trying i

Re: [PATCH v19 00/20] per memcg lru_lock

2020-09-24 Thread Daniel Jordan
On Thu, Sep 24, 2020 at 11:28:15AM +0800, Alex Shi wrote: > The new version rebased on v5.9-rc6 with line by line review by Hugh Dickins. These thpscale numbers are from a proto-v19, which now seems to be gone from Alex Shi's github. Once again, this is a regression test where memcg is enabled

Re: [PATCH] mm/migrate: correct thp migration stats.

2020-09-17 Thread Daniel Jordan
1a5bae25e3cf ("mm/vmstat: add events for THP migration without split") > Signed-off-by: Zi Yan Reviewed-by: Daniel Jordan If you wanted, you could also do this. diff --git a/mm/migrate.c b/mm/migrate.c index d1ad964165e5..6bc9559afc70 100644 --- a/mm/migrate.c +++ b/mm/migr

Re: [PATCH v18 00/32] per memcg lru_lock: reviews

2020-09-17 Thread Daniel Jordan
On Thu, Sep 17, 2020 at 08:39:34AM -0700, Alexander Duyck wrote: > On Thu, Sep 17, 2020 at 7:26 AM Daniel Jordan > wrote: > > > > On Thu, Sep 17, 2020 at 10:37:45AM +0800, Alex Shi wrote: > > > 在 2020/9/16 上午12:58, Daniel Jordan 写道: > > > > On Tue, Sep 15,

Re: [PATCH v18 00/32] per memcg lru_lock: reviews

2020-09-17 Thread Daniel Jordan
On Thu, Sep 17, 2020 at 10:37:45AM +0800, Alex Shi wrote: > 在 2020/9/16 上午12:58, Daniel Jordan 写道: > > On Tue, Sep 15, 2020 at 01:21:56AM -0700, Hugh Dickins wrote: > >> On Sun, 13 Sep 2020, Alex Shi wrote: > >>> Uh, I updated the testing with some new results her

Re: [PATCH v18 00/32] per memcg lru_lock: reviews

2020-09-15 Thread Daniel Jordan
-scalability.git/ && \ cd vm-scalability && make usemem COPY readtwice.patch /vm-scalability/ RUN cd vm-scalability && patch -p1 < readtwice.patch #!/usr/bin/env bash # # Originally by Alex Shi # Changes from Daniel Jordan SCRIPT_DIR="$( cd "$( dirname &quo

[PATCH] padata: fix possible padata_works_lock deadlock

2020-09-02 Thread Daniel Jordan
obs from a pool") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- kernel/padata.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/kernel/padata.c b/kernel/padata.c index

[PATCH v2] padata: add another maintainer and another list

2020-08-27 Thread Daniel Jordan
At Steffen's request, I'll help maintain padata for the foreseeable future. While at it, let's have patches go to lkml too since the code is now used outside of crypto. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel

Re: [PATCH] padata: add a reviewer

2020-08-27 Thread Daniel Jordan
On Thu, Aug 27, 2020 at 08:44:09AM +0200, Steffen Klassert wrote: > Please also consider to add yourself as one of the maintainers. Ok, sure! I'll take you up on that.

Re: [PATCH v18 00/32] per memcg lru_lock

2020-08-27 Thread Daniel Jordan
On Wed, Aug 26, 2020 at 04:59:28PM +0800, Alex Shi wrote: > I clean up my testing and make it reproducable by a Dockerfile and a case > patch which > attached. Ok, I'll give that a shot once I've taken care of sysbench. > >>> Even better would be a description of the problem you're having in

[PATCH] padata: add a reviewer

2020-08-26 Thread Daniel Jordan
I volunteer to review padata changes for the foreseeable future. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- MAINTAINERS | 1 + 1 file changed, 1 insertion(+) diff --git a/MAINTAINERS b/MAINTAINERS

Re: [PATCH v18 00/32] per memcg lru_lock

2020-08-25 Thread Daniel Jordan
On Tue, Aug 25, 2020 at 11:26:58AM +0800, Alex Shi wrote: > 在 2020/8/25 上午9:56, Daniel Jordan 写道: > > Alex, do you have a pointer to the modified readtwice case? > > Sorry, no. my developer machine crashed, so I lost case my container and > modified > case. I am strugglin

Re: [PATCH v18 00/32] per memcg lru_lock

2020-08-24 Thread Daniel Jordan
On Mon, Aug 24, 2020 at 01:24:20PM -0700, Hugh Dickins wrote: > On Mon, 24 Aug 2020, Andrew Morton wrote: > > On Mon, 24 Aug 2020 20:54:33 +0800 Alex Shi > > wrote: > Andrew demurred on version 17 for lack of review. Alexander Duyck has > been doing a lot on that front since then. I have

Re: [PATCH 2/2] mm, util: account_locked_vm() does not hold mmap_lock

2020-07-30 Thread Daniel Jordan
On Wed, Jul 29, 2020 at 12:21:11PM -0700, Hugh Dickins wrote: > On Sun, 26 Jul 2020, Pengfei Li wrote: > > > Since mm->locked_vm is already an atomic counter, account_locked_vm() > > does not need to hold mmap_lock. > > I am worried that this patch, already added to mmotm, along with its > 1/2

Re: [PATCH V4] mm/vmstat: Add events for THP migration without split

2020-07-24 Thread Daniel Jordan
-by: Daniel Jordan > diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h > index 705b33d1e395..4d434398d64d 100644 > --- a/include/trace/events/migrate.h > +++ b/include/trace/events/migrate.h > @@ -46,13 +46,18 @@ MIGRATE_REASON > TRACE_EVEN

Re: [PATCH v3] x86/mm: use max memory block size on bare metal

2020-07-15 Thread Daniel Jordan
On Tue, Jul 14, 2020 at 04:54:50PM -0400, Daniel Jordan wrote: > Some of our servers spend significant time at kernel boot initializing > memory block sysfs directories and then creating symlinks between them > and the corresponding nodes. The slowness happens because the machines &g

[PATCH v3] x86/mm: use max memory block size on bare metal

2020-07-14 Thread Daniel Jordan
isor use the largest supported size (2G) to minimize overhead on big machines. Kernel boot goes 7% faster on the aforementioned servers, shaving off half a second. Signed-off-by: Daniel Jordan Cc: Andrew Morton Cc: Andy Lutomirski Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Pavel

[PATCH 5/6] padata: fold padata_alloc_possible() into padata_alloc()

2020-07-14 Thread Daniel Jordan
There's no reason to have two interfaces when there's only one caller. Removing _possible saves text and simplifies future changes. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- Documentation/core-api

[PATCH 4/6] padata: remove effective cpumasks from the instance

2020-07-14 Thread Daniel Jordan
A padata instance has effective cpumasks that store the user-supplied masks ANDed with the online mask, but this middleman is unnecessary. parallel_data keeps the same information around. Removing this saves text and code churn in future changes. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc

[PATCH 3/6] padata: inline single call of pd_setup_cpumasks()

2020-07-14 Thread Daniel Jordan
pd_setup_cpumasks() has only one caller. Move its contents inline to prepare for the next cleanup. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- kernel/padata.c | 32 +--- 1

[PATCH 1/6] padata: remove start function

2020-07-14 Thread Daniel Jordan
e. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- crypto/pcrypt.c| 3 --- include/linux/padata.h | 1 - kernel/padata.c| 26 +- 3 files changed, 1 insertion(+), 29

[PATCH 6/6] padata: remove padata_parallel_queue

2020-07-14 Thread Daniel Jordan
Only its reorder field is actually used now, so remove the struct and embed @reorder directly in parallel_data. No functional change, just a cleanup. Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel@vger.kernel.org --- include

[PATCH 2/6] padata: remove stop function

2020-07-14 Thread Daniel Jordan
there are also no outstanding jobs. Removing it simplifies padata and saves text. [0] https://lore.kernel.org/linux-crypto/20191119225017.mjrak2fwa5vcc...@gondor.apana.org.au/ Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: linux-kernel

[PATCH 0/6] padata cleanups

2020-07-14 Thread Daniel Jordan
=linux-dmjordan.git;a=shortlog;h=refs/heads/padata-mt-wip-v0.5 Based on v5.8-rc5. As always, feedback is welcome. Daniel Daniel Jordan (6): padata: remove start function padata: remove stop function padata: inline single call of pd_setup_cpumasks() padata: remove effective cpumasks from

Re: [PATCH v2] Remove __init from padata_do_multithreaded and padata_mt_helper.

2020-07-08 Thread Daniel Jordan
(I was away for a while) On Thu, Jul 02, 2020 at 11:55:48AM -0400, Nico Pache wrote: > Allow padata_do_multithreaded function to be called after bootstrap. The functions are __init because they're currently only needed during boot, and using __init allows the text to be freed once it's over,

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-07-08 Thread Daniel Jordan
(I'm back now) On Fri, Jun 26, 2020 at 02:47:06PM +0200, Michal Hocko wrote: > On Mon 22-06-20 15:17:39, Daniel Jordan wrote: > > Hello Michal, > > > > (I've been away and may be slow to respond for a little while) > > > > On Fri, Jun 19, 2020 at 02:07:04PM +

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-22 Thread Daniel Jordan
Hello Michal, (I've been away and may be slow to respond for a little while) On Fri, Jun 19, 2020 at 02:07:04PM +0200, Michal Hocko wrote: > On Tue 09-06-20 18:54:51, Daniel Jordan wrote: > [...] > > @@ -1390,6 +1391,15 @@ static unsigned long probe_memory_block_size(void) > >

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-11 Thread Daniel Jordan
On Thu, Jun 11, 2020 at 10:05:38AM -0700, Dave Hansen wrote: > One other nit for this. We *do* have actual hardware hotplug, and I'm > pretty sure the alignment guarantees for hardware hotplug are pretty > weak. For instance, the alignment guarantees for persistent memory are > still only 64MB

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-11 Thread Daniel Jordan
On Thu, Jun 11, 2020 at 07:16:02AM -0700, Dave Hansen wrote: > On 6/9/20 3:54 PM, Daniel Jordan wrote: > > + /* > > +* Use max block size to minimize overhead on bare metal, where > > +* alignment for memory hotplug isn't a concern. > > +*/ >

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-10 Thread Daniel Jordan
On Wed, Jun 10, 2020 at 09:30:00AM +0200, David Hildenbrand wrote: > On 10.06.20 09:20, David Hildenbrand wrote: > > On 10.06.20 00:54, Daniel Jordan wrote: > >> @@ -1390,6 +1391,15 @@ static unsigned long probe_memory_block_size(void) > &g

Re: [PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-09 Thread Daniel Jordan
On Tue, Jun 09, 2020 at 06:54:51PM -0400, Daniel Jordan wrote: > Some of our servers spend significant time at kernel boot initializing > memory block sysfs directories and then creating symlinks between them > and the corresponding nodes. The slowness happens because the machines &g

[PATCH v2] x86/mm: use max memory block size on bare metal

2020-06-09 Thread Daniel Jordan
isor use the largest supported size (2G) to minimize overhead on big machines. Kernel boot goes 7% faster on the aforementioned servers, shaving off half a second. Signed-off-by: Daniel Jordan Cc: Andrew Morton Cc: Andy Lutomirski Cc: Dave Hansen Cc: David Hildenbrand Cc: Michal Hocko Cc: Pavel

Re: [PATCH V2] mm/vmstat: Add events for THP migration without split

2020-06-09 Thread Daniel Jordan
On Tue, Jun 09, 2020 at 05:05:45PM +0530, Anshuman Khandual wrote: > On 06/05/2020 07:54 PM, Zi Yan wrote: > > On 4 Jun 2020, at 23:35, Anshuman Khandual wrote: > >> Thanks Zi, for such a detailed explanation. Ideally, we should separate THP > >> migration from base page migration in terms of

[PATCH] padata: upgrade smp_mb__after_atomic to smp_mb in padata_do_serial

2020-06-08 Thread Daniel Jordan
in padata_do_serial() depends on it. Upgrade the barrier from smp_mb__after_atomic to smp_mb to get correct ordering again. Fixes: 3facced7aeed1 ("padata: remove reorder_objects") Signed-off-by: Daniel Jordan Cc: Herbert Xu Cc: Steffen Klassert Cc: linux-cry...@vger.kernel.org Cc: li

Re: [PATCH] x86/mm: use max memory block size with unaligned memory end

2020-06-04 Thread Daniel Jordan
On Thu, Jun 04, 2020 at 01:00:55PM -0700, Dave Hansen wrote: > On 6/4/20 11:12 AM, Daniel Jordan wrote: > >> E.g., on powerpc that's 16MB so they have *a lot* of memory blocks. > >> That's why that's not papering over the problem. Increasing the memory > >> bloc

Re: [PATCH] x86/mm: use max memory block size with unaligned memory end

2020-06-04 Thread Daniel Jordan
On Thu, Jun 04, 2020 at 08:55:19PM +0200, David Hildenbrand wrote: > >> E.g., on powerpc that's 16MB so they have *a lot* of memory blocks. > >> That's why that's not papering over the problem. Increasing the memory > >> block size isn't always the answer. > > > > Ok. If you don't mind, what's

Re: [PATCH] x86/mm: use max memory block size with unaligned memory end

2020-06-04 Thread Daniel Jordan
On Thu, Jun 04, 2020 at 07:45:40PM +0200, David Hildenbrand wrote: > On 04.06.20 19:22, Daniel Jordan wrote: > > IMHO the root cause of this is really the small block size. Building a > > cache > > on top to avoid iterating over tons of small blocks seems like papering

Re: [PATCH] x86/mm: use max memory block size with unaligned memory end

2020-06-04 Thread Daniel Jordan
On Thu, Jun 04, 2020 at 09:22:03AM +0200, David Hildenbrand wrote: > On 04.06.20 05:54, Daniel Jordan wrote: > > Some of our servers spend 14 out of the 21 seconds of kernel boot > > initializing memory block sysfs directories and then creating symlinks > > between them and th

[PATCH] x86/mm: use max memory block size with unaligned memory end

2020-06-03 Thread Daniel Jordan
That saves nearly all of the 14 seconds so the kernel boots 3x faster. There are some simple ways to avoid the linear searches, but for now it makes no difference with a 2G block. Signed-off-by: Daniel Jordan --- arch/x86/mm/init_64.c | 9 + 1 file changed, 9 insertions(+) diff --git a/a

Re: [PATCH] mm/vmstat: Add events for PMD based THP migration without split

2020-06-03 Thread Daniel Jordan
On Wed, Jun 03, 2020 at 10:06:31AM +0530, Anshuman Khandual wrote: > Does this look okay and sufficient ? > > --- a/Documentation/vm/page_migration.rst > +++ b/Documentation/vm/page_migration.rst > @@ -253,5 +253,20 @@ which are function pointers of struct > address_space_operations. >

Re: [PATCH] mm/vmstat: Add events for PMD based THP migration without split

2020-06-02 Thread Daniel Jordan
On Mon, Jun 01, 2020 at 09:48:09PM -0700, John Hubbard wrote: > However, the fact that this is under discussion hints at the need for a > bit of documentation help. What do you think about adding some notes about > all of this to, say, Documentation/vm/page_migration.rst ? Yes, that would be

Re: [PATCH] mm/vmstat: Add events for PMD based THP migration without split

2020-06-01 Thread Daniel Jordan
Hi Anshuman, On Fri, May 22, 2020 at 09:04:04AM +0530, Anshuman Khandual wrote: > This adds the following two new VM events which will help in validating PMD > based THP migration without split. Statistics reported through these events > will help in performance debugging. > > 1.

Re: [PATCH v6 12/12] mmap locking API: convert mmap_sem comments

2020-05-29 Thread Daniel Jordan
On Tue, May 19, 2020 at 10:29:08PM -0700, Michel Lespinasse wrote: > Convert comments that reference mmap_sem to reference mmap_lock instead. > > Signed-off-by: Michel Lespinasse Not opposed to leaving lockaphores in :) Reviewed-by: Daniel Jordan

Re: [PATCH v6 11/12] mmap locking API: convert mmap_sem API comments

2020-05-29 Thread Daniel Jordan
On Tue, May 19, 2020 at 10:29:07PM -0700, Michel Lespinasse wrote: > Convert comments that reference old mmap_sem APIs to reference > corresponding new mmap locking APIs instead. > > Signed-off-by: Michel Lespinasse Reviewed-by: Daniel Jordan

Re: [PATCH v6 10/12] mmap locking API: rename mmap_sem to mmap_lock

2020-05-29 Thread Daniel Jordan
> Signed-off-by: Michel Lespinasse > Reviewed-by: Vlastimil Babka Reviewed-by: Daniel Jordan

Re: [PATCH v6 09/12] mmap locking API: add mmap_assert_locked() and mmap_assert_write_locked()

2020-05-29 Thread Daniel Jordan
ned-off-by: Michel Lespinasse Reviewed-by: Daniel Jordan

Re: [PATCH v6 08/12] mmap locking API: add MMAP_LOCK_INITIALIZER

2020-05-29 Thread Daniel Jordan
by: Laurent Dufour > Reviewed-by: Vlastimil Babka Reviewed-by: Daniel Jordan

Re: [PATCH -V3] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-28 Thread Daniel Jordan
On Thu, May 28, 2020 at 01:32:40PM +0800, Huang, Ying wrote: > Daniel Jordan writes: > > > On Mon, May 25, 2020 at 08:26:48AM +0800, Huang Ying wrote: > >> diff --git a/mm/swapfile.c b/mm/swapfile.c > >> index 423c234aca15..0abd93d2a4fc 100644 > >> --- a

Re: [PATCH -V3] swap: Reduce lock contention on swap cache from swap slots allocation

2020-05-27 Thread Daniel Jordan
On Mon, May 25, 2020 at 08:26:48AM +0800, Huang Ying wrote: > diff --git a/mm/swapfile.c b/mm/swapfile.c > index 423c234aca15..0abd93d2a4fc 100644 > --- a/mm/swapfile.c > +++ b/mm/swapfile.c > @@ -615,7 +615,8 @@ static bool scan_swap_map_try_ssd_cluster(struct > swap_info_struct *si, >

[PATCH v3 7/8] mm: make deferred init's max threads arch-specific

2020-05-27 Thread Daniel Jordan
Using padata during deferred init has only been tested on x86, so for now limit it to this architecture. If another arch wants this, it can find the max thread limit that's best for it and override deferred_page_init_max_threads(). Signed-off-by: Daniel Jordan Tested-by: Josh Triplett

[PATCH v3 1/8] padata: remove exit routine

2020-05-27 Thread Daniel Jordan
padata_driver_exit() is unnecessary because padata isn't built as a module and doesn't exit. padata's init routine will soon allocate memory, so getting rid of the exit function now avoids pointless code to free it. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- kernel/padata.c | 6

[PATCH v3 0/8] padata: parallelize deferred page init

2020-05-27 Thread Daniel Jordan
ast-restart_kvmforum2019.pdf https://www.youtube.com/watch?v=pBsHnf93tcQ https://lore.kernel.org/linux-mm/1588812129-8596-1-git-send-email-anthony.yzn...@oracle.com/ [2] https://lore.kernel.org/linux-mm/20181105165558.11698-1-daniel.m.jor...@oracle.com/ Daniel Jordan (8): padata: remove exit rout

[PATCH v3 8/8] padata: document multithreaded jobs

2020-05-27 Thread Daniel Jordan
Add Documentation for multithreaded jobs. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- Documentation/core-api/padata.rst | 41 +++ 1 file changed, 31 insertions(+), 10 deletions(-) diff --git a/Documentation/core-api/padata.rst b/Documentation/core

[PATCH v3 4/8] padata: add basic support for multithreaded jobs

2020-05-27 Thread Daniel Jordan
and Steve Sistare. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- include/linux/padata.h | 29 kernel/padata.c| 152 - 2 files changed, 178 insertions(+), 3 deletions(-) diff --git a/include/linux/padata.h b/include/linux/padata.h

[PATCH v3 3/8] padata: allocate work structures for parallel jobs from a pool

2020-05-27 Thread Daniel Jordan
so remove it. If the global pool is exhausted, a parallel job is run in the current task instead to throttle a system trying to do too much in parallel. Signed-off-by: Daniel Jordan Tested-by: Josh Triplett --- include/linux/padata.h | 8 +-- kernel/padata.c

[PATCH v3 6/8] mm: parallelize deferred_init_memmap()

2020-05-27 Thread Daniel Jordan
as init process With this patch series: [0.231435] node 1 initialised, 24189223 pages in 32ms [0.236718] node 0 initialised, 23398907 pages in 36ms [1] https://static.sched.com/hosted_files/kvmforum2019/66/VMM-fast-restart_kvmforum2019.pdf Signed-off-by: Daniel Jordan Tested

[PATCH v3 2/8] padata: initialize earlier

2020-05-27 Thread Daniel Jordan
padata will soon initialize the system's struct pages in parallel, so it needs to be ready by page_alloc_init_late(). The error return from padata_driver_init() triggers an initcall warning, so add a warning to padata_init() to avoid silent failure. Signed-off-by: Daniel Jordan Tested-by: Josh

[PATCH v3 5/8] mm: don't track number of pages during deferred initialization

2020-05-27 Thread Daniel Jordan
that the page count can vary, just remove it. The boot message now looks like node 0 deferred pages initialised in 97ms Signed-off-by: Daniel Jordan Suggested-by: Alexander Duyck --- mm/page_alloc.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm

Re: [PATCH 4.4 26/65] sched/fair, cpumask: Export for_each_cpu_wrap()

2020-05-27 Thread Daniel Jordan
On 5/27/20 4:09 AM, Greg KH wrote: > On Wed, May 27, 2020 at 07:50:56AM +, nobuhiro1.iwama...@toshiba.co.jp > wrote: >>> Subject: [PATCH 4.4 26/65] sched/fair, cpumask: Export for_each_cpu_wrap() ... >> >> This commit also needs the following commits: >> >> commit

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Thu, May 21, 2020 at 09:46:35AM -0700, Alexander Duyck wrote: > It is more about not bothering with the extra tracking. We don't > really need it and having it doesn't really add much in the way of > value. Yeah, it can probably go. > > > > @@ -1863,11 +1892,32 @@ static int __init

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Thu, May 21, 2020 at 08:00:31AM -0700, Alexander Duyck wrote: > So I was thinking about my suggestion further and the loop at the end > isn't quite correct as I believe it could lead to gaps. The loop on > the end should probably be: > for_each_free_mem_pfn_range_in_zone_from(i,

Re: [PATCH v2 5/7] mm: parallelize deferred_init_memmap()

2020-05-21 Thread Daniel Jordan
On Wed, May 20, 2020 at 06:29:32PM -0700, Alexander Duyck wrote: > On Wed, May 20, 2020 at 11:27 AM Daniel Jordan > > @@ -1814,16 +1815,44 @@ deferred_init_maxorder(u64 *i, struct zone *zone, > > unsigned long *start_pfn, > > return nr_pages; > > } &

  1   2   3   4   5   >