Re: [PATCH 08/14] mm/hmm: cleanup special vma handling (VM_SPECIAL)

2018-03-16 Thread John Hubbard
unsigned long addr, i = 0; for (addr = range->start; addr < range->end; addr += PAGE_SIZE, i++) range->pfns[i] = HMM_PFN_SPECIAL; Either way, Reviewed-by: John Hubbard thanks, -- John Hubbard NVIDIA

Re: [PATCH 03/14] mm/hmm: HMM should have a callback before MM is destroyed v2

2018-03-16 Thread John Hubbard
On 03/16/2018 08:47 PM, John Hubbard wrote: > On 03/16/2018 07:36 PM, John Hubbard wrote: >> On 03/16/2018 12:14 PM, jgli...@redhat.com wrote: >>> From: Ralph Campbell >>> >> >> >> >>> +static void hmm_release(struct mmu_notifier *mn, str

Re: [PATCH v5] mmap.2: MAP_FIXED updated documentation

2017-12-18 Thread John Hubbard
On 12/18/2017 11:15 AM, Michael Kerrisk (man-pages) wrote: > On 12/12/2017 01:23 AM, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> -- Expand the documentation to discuss the hazards in >>enough detail to allow avoiding them. >> >>

Re: [PATCH] mm: disable `vm.max_map_count' sysctl limit

2017-11-27 Thread John Hubbard
ompletely* changed to use a separate tracking system altogether). The existing limit seems rather too low, at least from my perspective. Maybe it would be better, if expressed as a function of RAM size? [1] https://devblogs.nvidia.com/parallelforall/unified-memory-in-cuda-6/ This is a way to automatically (via page faulting) migrate memory between CPUs and devices (GPUs, here). This is before HMM, of course. thanks, John Hubbard

Re: [PATCH] mm: disable `vm.max_map_count' sysctl limit

2017-11-28 Thread John Hubbard
On 11/28/2017 12:12 AM, Michal Hocko wrote: > On Mon 27-11-17 15:26:27, John Hubbard wrote: > [...] >> Let me add a belated report, then: we ran into this limit while implementing >> an early version of Unified Memory[1], back in 2013. The implementation >> at the t

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-04 Thread John Hubbard
must be a multiple of SHMLBA (), which in turn is either the system page size (on many architectures) or a multiple of the system page size (on some architectures)." What do you think? thanks, John Hubbard NVIDIA > Which should at least hint the reader that this is architecture specific. >

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-04 Thread John Hubbard
On 12/04/2017 03:31 AM, Mike Rapoport wrote: > On Sun, Dec 03, 2017 at 06:14:11PM -0800, john.hubb...@gmail.com wrote: >> From: John Hubbard >> [...] >> +.IP >> +Given the above limitations, one of the very few ways to use this option >> +safely is: mmap() a reg

[PATCH v3] mmap.2: MAP_FIXED updated documentation

2017-12-04 Thread john . hubbard
From: John Hubbard Previously, MAP_FIXED was "discouraged", due to portability issues with the fixed address. In fact, there are other, more serious issues. Also, in some limited cases, this option can be used safely. Expand the documentation to discuss both the hazards, and

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-04 Thread John Hubbard
On 12/04/2017 11:05 PM, Michal Hocko wrote: > On Mon 04-12-17 18:14:18, John Hubbard wrote: >> On 12/04/2017 02:55 AM, Cyril Hrubis wrote: >>> Hi! >>> I know that we are not touching the rest of the existing description for >>> MAP_FIXED however the sec

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-04 Thread John Hubbard
On 12/04/2017 11:08 PM, Michal Hocko wrote: > On Mon 04-12-17 18:52:27, John Hubbard wrote: >> On 12/04/2017 03:31 AM, Mike Rapoport wrote: >>> On Sun, Dec 03, 2017 at 06:14:11PM -0800, john.hubb...@gmail.com wrote: >>>> From: John Hubbard >>>> >> [

[PATCH v4] mmap.2: MAP_FIXED updated documentation

2017-12-05 Thread john . hubbard
From: John Hubbard Previously, MAP_FIXED was "discouraged", due to portability issues with the fixed address. In fact, there are other, more serious issues. Also, alignment requirements were a bit vague. So: -- Expand the documentation to discuss the hazards in enough detai

Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE

2017-12-05 Thread John Hubbard
n MAP_REQUIRED. I came up with some awful names > (MAP_TODDLER, MAP_TANTRUM, MAP_ULTIMATUM, MAP_BOSS, MAP_PROGRAM_MANAGER, > etc). But I think we should drop FIXED from the middle of the name. > In that case, maybe: MAP_EXACT ? ...because that's the characteristic behavior. It doesn't clobber, but you don't need to say that in the name, now that we're not including _FIXED_ in the middle. thanks, John Hubbard NVIDIA

Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE

2017-12-06 Thread John Hubbard
On 12/05/2017 11:35 PM, Florian Weimer wrote: > On 12/06/2017 08:33 AM, John Hubbard wrote: >> In that case, maybe: >> >> MAP_EXACT >> >> ? ...because that's the characteristic behavior. > > Is that true?  mmap still silently rounding up the length

Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation

2017-12-13 Thread John Hubbard
On 12/13/2017 06:52 PM, Jann Horn wrote: > On Wed, Dec 13, 2017 at 10:31 AM, Michal Hocko wrote: >> From: John Hubbard >> >> -- Expand the documentation to discuss the hazards in >>enough detail to allow avoiding them. >> >> -- M

Re: [PATCH 2/2] mmap.2: MAP_FIXED updated documentation

2017-12-14 Thread John Hubbard
On 12/13/2017 06:52 PM, Jann Horn wrote: > On Wed, Dec 13, 2017 at 10:31 AM, Michal Hocko wrote: >> From: John Hubbard [...] >> +.IP >> +Furthermore, this option is extremely hazardous (when used on its own), >> because >> +it forcibly removes pre-

[PATCH 1/1] acpi: unbreak ACPI_HANDLE(), encapsulate fwnode_operations

2017-09-15 Thread john . hubbard
From: John Hubbard Due to commit db3e50f3234b ("device property: Get rid of struct fwnode_handle type field"), ACPI_HANDLE() inadvertently became a GPL-only call. The call path that led to that was: ACPI_HANDLE() ACPI_COMPANION() to_acpi_device_node() is_acpi_d

[PATCH 0/1] acpi: unbreak ACPI_HANDLE(), encapsulate fwnode_operations

2017-09-15 Thread john . hubbard
From: John Hubbard Hi everyone, I really don't know for sure which fix is going to be preferred--the following patch, or just an obvious one-line fix that changes DECLARE_ACPI_FWNODE_OPS() so that it invokes EXPORT_SYMBOL, instead of EXPORT_SYMBOL_GPL. I explained the reasoning in PATCH 1/1

Re: [PATCH] mmap.2: document new MAP_FIXED_NOREPLACE flag

2018-04-11 Thread John Hubbard
On 04/11/2018 08:37 AM, Jann Horn wrote: > On Wed, Apr 11, 2018 at 2:04 PM, wrote: >> From: Michal Hocko >> >> 4.17+ kernels offer a new MAP_FIXED_NOREPLACE flag which allows the caller to >> atomicaly probe for a given address range. >> >> [wording heav

Re: [PATCH] mmap.2: MAP_FIXED is okay if the address range has been reserved

2018-04-12 Thread John Hubbard
Maybe if you break up the sentence, and possibly omit non-MAP_FIXED discussion, it will help. > +and take appropriate action if the kernel places the new mapping at a > +different address. > .TP > .BR MAP_FIXED_NOREPLACE " (since Linux 4.17)" > .\" commit a4ff8e8620d3f4f50ac4b41e8067b7d395056843 > thanks, -- John Hubbard NVIDIA

Re: [PATCH] mmap.2: MAP_FIXED is okay if the address range has been reserved

2018-04-12 Thread John Hubbard
On 04/12/2018 11:49 AM, Jann Horn wrote: > On Thu, Apr 12, 2018 at 8:37 PM, Michael Kerrisk (man-pages) > wrote: >> Hi John, >> >> On 12 April 2018 at 20:33, John Hubbard wrote: >>> On 04/12/2018 08:39 AM, Jann Horn wrote: >>>> Clarify that MAP_FIXED

Re: [PATCH] mmap.2: MAP_FIXED is okay if the address range has been reserved

2018-04-12 Thread John Hubbard
On 04/12/2018 12:18 PM, Jann Horn wrote: > On Thu, Apr 12, 2018 at 8:59 PM, John Hubbard wrote: >> On 04/12/2018 11:49 AM, Jann Horn wrote: >>> On Thu, Apr 12, 2018 at 8:37 PM, Michael Kerrisk (man-pages) >>> wrote: >>>> Hi John, >>>>

Re: [PATCH 00/15] HMM (Heterogeneous Memory Management) v24

2017-06-29 Thread John Hubbard
ooks like your patch was not rejected, but I can't tell if (!rejected == accepted), there. :) We'll continue testing, but I expect at this point that anything we find can be patched up after HMM finally gets merged. thanks, John Hubbard NVIDIA > > Everything else is the same. Below is

Re: [PATCH] vmalloc: respect the GFP_NOIO and GFP_NOFS flags

2017-06-30 Thread John Hubbard
On 06/29/2017 07:25 PM, Mikulas Patocka wrote: > The __vmalloc function has a parameter gfp_mask with the allocation flags, > however it doesn't fully respect the GFP_NOIO and GFP_NOFS flags. The > pages are allocated with the specified gfp flags, but the pagetables are > always allocated with

Re: [RFC v2 0/5] surface heterogeneous memory performance information

2017-07-06 Thread John Hubbard
On 07/06/2017 02:52 PM, Ross Zwisler wrote: [...] > > The naming collision between Jerome's "Heterogeneous Memory Management > (HMM)" and this "Heterogeneous Memory (HMEM)" series is unfortunate, but I > was trying to stick with the word "Heterogeneous" because of the naming of > the ACPI 6.2

Re: [RFC v2 3/5] hmem: add heterogeneous memory sysfs support

2017-07-06 Thread John Hubbard
On 07/06/2017 02:52 PM, Ross Zwisler wrote: [...] > diff --git a/drivers/acpi/Makefile b/drivers/acpi/Makefile > index b1aacfc..31e3f20 100644 > --- a/drivers/acpi/Makefile > +++ b/drivers/acpi/Makefile > @@ -72,6 +72,7 @@ obj-$(CONFIG_ACPI_PROCESSOR)+= processor.o > obj-$(CONFIG_ACPI)

Re: [RFC PATCH 0/2] mm: introduce MAP_FIXED_SAFE

2017-11-21 Thread John Hubbard
On 11/20/2017 01:05 AM, Michal Hocko wrote: > On Fri 17-11-17 00:45:49, John Hubbard wrote: >> On 11/16/2017 04:14 AM, Michal Hocko wrote: >>> [Ups, managed to screw the subject - fix it] >>> >>> On Thu 16-11-17 11:18:58, Michal Hocko wrote: >>>> Hi,

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-06 Thread John Hubbard
you're thinking that since the SHMLBA cannot be put in the man pages, we could instead provide MapAlignment as sort of a different way to document the requirement? -- thanks, John Hubbard NVIDIA

Re: [PATCH 0/2] mm: introduce MAP_FIXED_SAFE

2017-12-06 Thread John Hubbard
On 12/06/2017 04:19 PM, Kees Cook wrote: > On Wed, Dec 6, 2017 at 1:08 AM, Michal Hocko wrote: >> On Wed 06-12-17 08:33:37, Rasmus Villemoes wrote: >>> On 2017-12-06 05:50, Michael Ellerman wrote: Michal Hocko writes: > On Wed 29-11-17 14:25:36, Kees Cook wrote: > It is safe in

[PATCH] mmap.2: MAP_FIXED is no longer discouraged

2017-12-01 Thread john . hubbard
From: John Hubbard MAP_FIXED has been widely used for a very long time, yet the man page still claims that "the use of this option is discouraged". The documentation assumes that "less portable" == "must be discouraged". Instead of discouraging something tha

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

2017-12-02 Thread John Hubbard
to > virtually any library call. This is because almost any library call may be > implemented by using dlopen(3) to load another shared library, which will be > mapped into the process's address space. The PAM libraries are an excellent > example, as well as more obvious examples like brk(2), malloc(3) and even > pthread_create(3)." > > What do you think? > I'm working on some updated wording to capture these points. I'm even slower at writing than I am at coding, so there will be a somewhat-brief pause here... :) thanks, John Hubbard NVIDIA

Re: [PATCH] mmap.2: MAP_FIXED is no longer discouraged

2017-12-02 Thread John Hubbard
Linux 4.16 and later) have a MAP_FIXED_SAFE option that avoids the corruption problem; if available, MAP_FIXED_SAFE should be preferred over MAP_FIXED. thanks, John Hubbard NVIDIA

[PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-03 Thread john . hubbard
From: John Hubbard Previously, MAP_FIXED was "discouraged", due to portability issues with the fixed address. In fact, there are other, more serious issues. Also, in some limited cases, this option can be used safely. Expand the documentation to discuss both the hazards, and

Re: [PATCH] mmap.2: document new MAP_FIXED_SAFE flag

2017-11-29 Thread John Hubbard
ernels and C libraries may set up quite +different mapping ranges. ...because that advice is just wrong (it presumes that "less portable" == "must be discouraged"). Should I send out a separate patch for that, or is it better to glom it together with this one? thanks, John Hubbard NVIDIA

Re: [PATCH v2] mmap.2: document new MAP_FIXED_SAFE flag

2017-11-30 Thread John Hubbard
AP_FIXED_SAFE flag > > 4.16+ kernels offer a new MAP_FIXED_SAFE flag which allows the caller to > atomicaly probe for a given address range. > > [wording heavily updated by John Hubbard ] > Signed-off-by: Michal Hocko > --- > man2/mmap.2 | 22 ++ > 1 f

Re: [PATCH v2] mmap.2: MAP_FIXED updated documentation

2017-12-09 Thread John Hubbard
t: place the mapping at exactly that address. addr must be suitably aligned: for most architectures a multiple of page size is sufficient; however, some architectures may impose additional restrictions. ...which is basically what Cyril was asking for, in his ear

Re: [PATCH v4] mmap.2: MAP_FIXED updated documentation

2017-12-10 Thread John Hubbard
On 12/10/2017 02:31 AM, Michal Hocko wrote: > On Tue 05-12-17 19:14:34, john.hubb...@gmail.com wrote: >> From: John Hubbard >> >> Previously, MAP_FIXED was "discouraged", due to portability >> issues with the fixed address. In fact, there are other, mo

[PATCH v5] mmap.2: MAP_FIXED updated documentation

2017-12-11 Thread john . hubbard
From: John Hubbard -- Expand the documentation to discuss the hazards in enough detail to allow avoiding them. -- Mention the upcoming MAP_FIXED_SAFE flag. -- Enhance the alignment requirement slightly. CC: Michael Ellerman CC: Jann Horn CC: Matthew Wilcox CC: Michal

Re: [PATCH 0/1] mm: restore full accuracy in COW page reuse

2021-01-15 Thread John Hubbard
I proposed this exact idea a few days ago [1]. It's remarkable that we both picked nearly identical values for the layout! :) But as the responses show, security problems prevent pursuing that approach. [1] https://lore.kernel.org/r/45806a5a-65c2-67ce-fc92-dc8c2144d...@nvidia.com thanks, -- John Hubbard NVIDIA

Re: [PATCH 2/2] mm: soft_dirty: userfaultfd: introduce wrprotect_tlb_flush_pending

2021-01-07 Thread John Hubbard
pages that can be waited for, and pages that should not be waited for in the kernel. I hope this helps, but if it's too much of a side-track, please disregard. thanks, -- John Hubbard NVIDIA

Re: [PATCH 2/2] mm: soft_dirty: userfaultfd: introduce wrprotect_tlb_flush_pending

2021-01-07 Thread John Hubbard
On 1/7/21 2:00 PM, Linus Torvalds wrote: On Thu, Jan 7, 2021 at 1:53 PM John Hubbard wrote: Now, I do agree that from a QoI standpoint, it would be really lovely if we actually enforced it. I'm not entirely sure we can, but maybe it would be reasonable to use that mm->has_pin

Re: [PATCH 2/2] mm: soft_dirty: userfaultfd: introduce wrprotect_tlb_flush_pending

2021-01-07 Thread John Hubbard
to the RDMA cases, but still does what we need here. thanks, -- John Hubbard NVIDIA

Re: [PATCH] selftests/vm: 10x speedup for hmm-tests

2020-10-06 Thread John Hubbard
On 10/2/20 6:17 PM, John Hubbard wrote: This patch reduces the running time for hmm-tests from about 10+ seconds, to just under 1.0 second, for an approximately 10x speedup. That brings it in line with most of the other tests in selftests/vm, which mostly run in < 1 sec. This is done with a

[PATCH v2] mm, dump_page: do not crash with bad compound_mapcount()

2020-08-04 Thread John Hubbard
Signed-off-by: John Hubbard --- Hi, I'm assuming that a fix is not required for -stable, but let me know if others feel differently. The dump_page() code has changed a lot in that area. Changes since v1 [1]: 1) Rebased onto mmotm 2) Used a simpler head_*count() approach. 3) Added Matthew's

[PATCH] selftests/vm: fix: make check_config.sh executable

2020-10-02 Thread John Hubbard
t 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") Signed-off-by: John Hubbard --- Hi, This applies to today's 20201001) linux-next. I forgot if this needs to be a separate patch or if it can be fixed up in Andrew's tree. thanks, John Hubbard NVIDIA tool

Re: [PATCH] selftests/vm: fix: make check_config.sh executable

2020-10-02 Thread John Hubbard
On 10/1/20 11:48 PM, John Hubbard wrote: commit 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") created the new check_config.sh file without the execute bits set. This is a problem because the Makefile runs it with "./", so now "make"

[PATCH 0/1] selftests/vm: 8x compaction_test speedup

2020-10-02 Thread John Hubbard
ms, and a few take 1.0 sec. This drops the compaction_test run time from 27, to 3.3 sec. Enjoy. :) thanks, John Hubbard NVIDIA John Hubbard (1): selftests/vm: 8x compaction_test speedup tools/testing/selftests/vm/compaction_test.c | 11 ++- 1 file changed, 6 insertions(+), 5 deletions

[PATCH 1/1] selftests/vm: 8x compaction_test speedup

2020-10-02 Thread John Hubbard
. Instead, do the mmaps 100 MB at a time. (Going past 100 MB doesn't make things go much faster, because other parts of the program are using the remaining time.) Signed-off-by: John Hubbard --- tools/testing/selftests/vm/compaction_test.c | 11 ++- 1 file changed, 6 insertions(+), 5

Re: [PATCH] selftests/vm: fix: make check_config.sh executable

2020-10-02 Thread John Hubbard
On 10/2/20 12:59 AM, Stephen Rothwell wrote: Hi John, On Thu, 1 Oct 2020 23:48:11 -0700 John Hubbard wrote: commit 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") created the new check_config.sh file without the execute bits set. This is a probl

Re: [PATCH] selftests/vm: fix: make check_config.sh executable

2020-10-02 Thread John Hubbard
On 10/2/20 1:10 AM, John Hubbard wrote: On 10/2/20 12:59 AM, Stephen Rothwell wrote: Hi John, On Thu, 1 Oct 2020 23:48:11 -0700 John Hubbard wrote: commit 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") created the new check_config.sh file without t

[PATCH v2 2/2] selftests/vm: fix run_vmtest.sh: restore executable bits, and "s" in name

2020-10-02 Thread John Hubbard
executable bits. Fixes: cb2ab76685d7 ("selftests/vm: rename run_vmtests --> run_vmtests.sh") Signed-off-by: John Hubbard --- tools/testing/selftests/vm/{run_vmtest.sh => run_vmtests.sh} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename tools/testing/selftests/vm/{run_vm

[PATCH v2 1/2] selftests/vm: fix: make check_config.sh executable

2020-10-02 Thread John Hubbard
t 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") Signed-off-by: John Hubbard --- tools/testing/selftests/vm/check_config.sh | 0 1 file changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 tools/testing/selftests/vm/check_config.sh di

[PATCH v2 0/2] selftests/vm: fix up executable bits, and a file name

2020-10-02 Thread John Hubbard
I suspect these patches will actually be merged back into Andrew's tree?) thanks, John Hubbard NVIDIA John Hubbard (2): selftests/vm: fix: make check_config.sh executable selftests/vm: fix run_vmtest.sh: restore executable bits, and "s" in name tools/testing/selftest

Re: [PATCH v2 2/2] selftests/vm: fix run_vmtest.sh: restore executable bits, and "s" in name

2020-10-02 Thread John Hubbard
On 10/2/20 2:59 PM, Andrew Morton wrote: On Fri, 2 Oct 2020 01:40:49 -0700 John Hubbard wrote: commit cb2ab76685d7 ("selftests/vm: rename run_vmtests --> run_vmtests.sh") changed the name of run_vmtests to run_vmtest.sh, but inadvertently dropped the executable bits. We

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-02 Thread John Hubbard
brought this up. It's definitely time to add FOLL_LONGTERM wherever it's missing. thanks, -- John Hubbard NVIDIA There is already a dax specific check (added in b7f0554a56f2 ("mm: fail get_vaddr_frames() for filesystem-dax mappings")), so this seems like the prudent thing to do. Signed

[PATCH v3 1/2] selftests/vm: fix an improper dependency upon executable script permissions

2020-10-02 Thread John Hubbard
ce touch, make check_config.sh executable as well. Fixes: commit 30fb9454ab23 ("selftests/vm: hmm-tests: remove the libhugetlbfs dependency") Signed-off-by: John Hubbard --- tools/testing/selftests/vm/Makefile| 2 +- tools/testing/selftests/vm/check_config.sh | 0 2 files changed, 1

[PATCH v3 0/2] selftests/vm: fix a rename typo, fix executable issues

2020-10-02 Thread John Hubbard
Here's v3, which treats the executable permissions of the two scripts here as optional nice-to-have features. It fixes the rename as well. Again, I've put in commit hashes that are only valid in linux-next, so those will need adjustment if these patches remain as distinct patches. John Hubbard

[PATCH v3 2/2] selftests/vm: fix a rename typo: run_vmtest.sh --> run_vmtests.sh

2020-10-02 Thread John Hubbard
executable bit on that same file, as a nice touch if it sticks. (Which it won't, if someone uses patch(1) to apply this, but it's worth setting it for everyone else.) Fixes: cb2ab76685d7 ("selftests/vm: rename run_vmtests --> run_vmtests.sh") Signed-off-by: John Hubbard --- tools/testing/self

[PATCH] selftests/vm: 10x speedup for hmm-tests

2020-10-02 Thread John Hubbard
ces the number of iterations of several tests, from 256, to 10. Thanks to Ralph Campbell for suggesting changing NTIMES as a way to get the speedup. Suggested-by: Ralph Campbell Signed-off-by: John Hubbard --- This is based on mmotm. tools/testing/selftests/vm/hmm-tests.c | 2 +- 1 file changed

Re: [PATCH] selftests/vm: 10x speedup for hmm-tests

2020-10-02 Thread John Hubbard
On 10/2/20 10:23 PM, SeongJae Park wrote: On Fri, 2 Oct 2020 18:17:21 -0700 John Hubbard wrote: This patch reduces the running time for hmm-tests from about 10+ seconds, to just under 1.0 second, for an approximately 10x speedup. That brings it in line with most of the other tests

Re: [PATCH 2/2] mm/frame-vec: use FOLL_LONGTERM

2020-10-03 Thread John Hubbard
On 10/3/20 2:45 AM, Daniel Vetter wrote: On Sat, Oct 3, 2020 at 12:39 AM John Hubbard wrote: On 10/2/20 10:53 AM, Daniel Vetter wrote: For $reasons I've stumbled over this code and I'm not sure the change to the new gup functions in 55a650c35fea ("mm/gup: frame_vector: convert get_user_

Re: [PATCH v2 8/8] selftests/vm: hmm-tests: remove the libhugetlbfs dependency

2020-10-04 Thread John Hubbard
On 10/4/20 12:55 AM, Lukas Bulwahn wrote: On Tue, 29 Sep 2020, John Hubbard wrote: ... John, your change makes: tools/testing/selftests$ make clean fail with: make[1]: Entering directory '/home/lukas/repositories/kernel.org/pub/scm/linux/kernel/git/next/linux-next/tools/testing/selftests/vm

Re: btrfs crash in kobject_del while running xfstest

2020-08-11 Thread John Hubbard
Somehow the copy-paste of Chris Mason's name failed (user error on my end), sorry about that Chris! On 8/11/20 4:17 AM, John Hubbard wrote: Hi, Here's an early warning of a possible problem. I'm seeing a new btrfs crash when running xfstests, as of 00e4db51259a5f936fec1424b884f029479d3981

Re: [PATCH 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
On 9/29/20 9:35 AM, Jason Gunthorpe wrote: On Mon, Sep 28, 2020 at 01:10:24PM -0700, John Hubbard wrote: On 9/28/20 5:57 AM, Jason Gunthorpe wrote: On Sun, Sep 27, 2020 at 11:21:53PM -0700, John Hubbard wrote: diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm

Re: [PATCH 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
On 9/29/20 10:55 AM, Jason Gunthorpe wrote: On Tue, Sep 29, 2020 at 10:44:31AM -0700, John Hubbard wrote: On 9/29/20 9:35 AM, Jason Gunthorpe wrote: On Mon, Sep 28, 2020 at 01:10:24PM -0700, John Hubbard wrote: On 9/28/20 5:57 AM, Jason Gunthorpe wrote: On Sun, Sep 27, 2020 at 11:21:53PM

Re: [PATCH 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
On 9/29/20 12:08 PM, Jason Gunthorpe wrote: On Tue, Sep 29, 2020 at 11:59:55AM -0700, John Hubbard wrote: On 9/29/20 10:55 AM, Jason Gunthorpe wrote: On Tue, Sep 29, 2020 at 10:44:31AM -0700, John Hubbard wrote: On 9/29/20 9:35 AM, Jason Gunthorpe wrote: On Mon, Sep 28, 2020 at 01:10:24PM

Re: [PATCH 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
and doesn't break workflow for developers. Do you have a link or two for that? Especially about the pushback, and conclusions reached, if any. thanks, -- John Hubbard NVIDIA

Re: [PATCH 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
On 9/29/20 1:20 PM, Shuah Khan wrote: On 9/29/20 2:11 PM, John Hubbard wrote: ... Do you have a link or two for that? Especially about the pushback, and conclusions reached, if any. Unfortunately no. A I recall it was workflow related issues and ease of running individual subsystem tests

[PATCH v2 4/8] selftests/vm: minor cleanup: Makefile and gup_test.c

2020-09-29 Thread John Hubbard
of one special rule already. Signed-off-by: John Hubbard --- tools/testing/selftests/vm/Makefile | 10 -- tools/testing/selftests/vm/gup_test.c | 1 - 2 files changed, 4 insertions(+), 7 deletions(-) diff --git a/tools/testing/selftests/vm/Makefile b/tools/testing/selftests/vm

[PATCH v2 7/8] selftests/vm: run_vmtest.sh: update and clean up gup_test invocation

2020-09-29 Thread John Hubbard
Run benchmarks on the _fast variants of gup and pup, as originally intended. Run the new gup_test sub-test: dump pages. In addition to exercising the dump_page() call, it also demonstrates the various options you can use to specify which pages to dump, and how. Signed-off-by: John Hubbard

[PATCH v2 6/8] selftests/vm: gup_test: introduce the dump_pages() sub-test

2020-09-29 Thread John Hubbard
ges -F 1:use pin_user_pages() instead of get_user_pages() 0 19 0x1000: dump pages 0, 19, and 4096 Signed-off-by: John Hubbard --- mm/Kconfig| 6 +++ mm/gup_test.c | 54 ++- mm/gup_test.h

[PATCH v2 8/8] selftests/vm: hmm-tests: remove the libhugetlbfs dependency

2020-09-29 Thread John Hubbard
-test.c (local_config.h), and one for inclusion in the Makefile (local_config.mk). Cc: Ralph Campbell Signed-off-by: John Hubbard --- tools/testing/selftests/vm/.gitignore | 1 + tools/testing/selftests/vm/Makefile| 24 +++-- tools/testing/selftests/vm/check_config.sh

[PATCH v2 5/8] selftests/vm: only some gup_test items are really benchmarks

2020-09-29 Thread John Hubbard
reasonably needs to check. 3. Don't do iterations, for non-benchmark items. 4. Print out a shorter, more appropriate report for the non-benchmark tests. 5. Add the command that was run, to the report. This really helps, as there are quite a lot of options now. Signed-off-by: John Hubbard

[PATCH v2 1/8] mm/gup_benchmark: rename to mm/gup_test

2020-09-29 Thread John Hubbard
importantly, however, subsequent patches are about to add some functionality that is non-benchmark related. Closely related changes: * Kconfig: in addition to renaming the options from GUP_BENCHMARK to GUP_TEST, update the help text to reflect that it's no longer a benchmark-only test. Signed-

[PATCH v2 0/8] selftests/vm: gup_test, hmm-tests, assorted improvements

2020-09-29 Thread John Hubbard
bhugetlbfs is not available. Further details in the commit description of patch #8. Other smaller things that this series does: a) Remove code duplication by creating gup_test.h. b) Clear up the sub-test organization, and their invocation within run_vmtests.sh. c) Other minor assorted improvemen

[PATCH v2 3/8] selftests/vm: rename run_vmtests --> run_vmtests.sh

2020-09-29 Thread John Hubbard
Rename to *.sh, in order to match the conventions of all of the other items in selftest/vm. The only reason not to use a .sh suffix a shell script like this, might be to make it look more like a normal program, but that's not an issue here. Signed-off-by: John Hubbard --- tools/testing

[PATCH v2 2/8] selftests/vm: use a common gup_test.h

2020-09-29 Thread John Hubbard
, but this low-level, kernel-developer-oriented selftests/vm system is very much not subject to ABI stability. So "expansion" and "reserved" fields are unnecessary here. Signed-off-by: John Hubbard --- mm/gup_test.c | 17 +---

Re: [PATCH 1/4] mm/gup_benchmark: Take the mmap lock around GUP

2020-09-30 Thread John Hubbard
ight thing, in other words, after adding the new DUMP_USER_PAGES_TEST (a non-fast variant). Reviewed-by: John Hubbard [1] https://lore.kernel.org/r/20200929212747.251804-1-jhubb...@nvidia.com thanks, -- John Hubbard NVIDIA mm/gup_benchmark.c | 15 --- 1 file changed, 12 insert

Re: [PATCH 1/5] kselftests: vm: Add mremap tests

2020-10-01 Thread John Hubbard
SIMPLE_TEST(PUD, PTE, _2GB), + MAKE_SIMPLE_TEST(PUD, _1MB, _2GB), + MAKE_SIMPLE_TEST(PUD, PMD, _2GB), + MAKE_SIMPLE_TEST(PUD, PUD, _2GB), Too concise. Not fun lining these up with the stdout report. thanks, -- John Hubbard NVIDIA

Re: [PATCH v2 01/17] drm/exynos: Stop using frame_vector helpers

2020-10-16 Thread John Hubbard
: Kyungmin Park Cc: Kukjin Kim Cc: Krzysztof Kozlowski Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux...@kvack.org Cc: linux-arm-ker...@lists.infradead.org Cc: linux-samsung-...@vger.kernel.org Cc: linux-me...@vger.kernel.org -- v2: Use

Re: [PATCH v2 03/17] misc/habana: Stop using frame_vector helpers

2020-10-16 Thread John Hubbard
On 10/9/20 12:59 AM, Daniel Vetter wrote: All we need are a pages array, pin_user_pages_fast can give us that directly. Plus this avoids the entire raw pfn side of get_vaddr_frames. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc

Re: [PATCH v2 05/17] mm/frame-vector: Use FOLL_LONGTERM

2020-10-16 Thread John Hubbard
less than ret == nr_frames. And the whole partially pinned region idea turns out to be just not useful for almost everyone, from what I recall of the gup/pup call sites. So I wonder if we should just have get_vaddr_frames do the cleanup here and return -EFAULT, if ret != nr_frames ? thanks, -- John Hubbard NVIDIA

Re: [PATCH 01/13] drm/exynos: Stop using frame_vector helpers

2020-10-07 Thread John Hubbard
: Kyungmin Park Cc: Kukjin Kim Cc: Krzysztof Kozlowski Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux...@kvack.org Cc: linux-arm-ker...@lists.infradead.org Cc: linux-samsung-...@vger.kernel.org Cc: linux-me...@vger.kernel.org --- drivers/gpu/drm/exynos

Re: [PATCH 03/13] misc/habana: Stop using frame_vector helpers

2020-10-07 Thread John Hubbard
patch 1: you can further simplify by using unpin_user_pages_dirty_lock(). list_del(>job_node); thanks, -- John Hubbard NVIDIA

Re: [PATCH 02/13] drm/exynos: Use FOLL_LONGTERM for g2d cmdlists

2020-10-07 Thread John Hubbard
: Kukjin Kim Cc: Krzysztof Kozlowski Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux...@kvack.org Cc: linux-arm-ker...@lists.infradead.org Cc: linux-samsung-...@vger.kernel.org Cc: linux-me...@vger.kernel.org --- drivers/gpu/drm/exynos

Re: [PATCH 04/13] misc/habana: Use FOLL_LONGTERM for userptr

2020-10-07 Thread John Hubbard
On 10/7/20 9:44 AM, Daniel Vetter wrote: These are persistent, not just for the duration of a dma operation. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux...@kvack.org Cc: linux-arm-ker

Re: mmotm 2020-10-06-15-50 uploaded

2020-10-07 Thread John Hubbard
and -next. [1] https://lore.kernel.org/dri-devel/f5130c7f-eebe-7b21-62b8-68f08212b...@nvidia.com [2] https://lore.kernel.org/r/f3c7f005b...@google.com thanks, -- John Hubbard NVIDIA

Re: [PATCH 05/13] mm/frame-vector: Use FOLL_LONGTERM

2020-10-07 Thread John Hubbard
the pup path out from the mmap_sem critical section as suggested by Jason. Signed-off-by: Daniel Vetter Cc: Jason Gunthorpe Cc: Pawel Osciak Cc: Marek Szyprowski Cc: Kyungmin Park Cc: Tomasz Figa Cc: Mauro Carvalho Chehab Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc

Re: [PATCH 01/13] drm/exynos: Stop using frame_vector helpers

2020-10-07 Thread John Hubbard
On 10/7/20 2:32 PM, Daniel Vetter wrote: On Wed, Oct 7, 2020 at 10:33 PM John Hubbard wrote: On 10/7/20 9:44 AM, Daniel Vetter wrote: ... @@ -398,15 +399,11 @@ static void g2d_userptr_put_dma_addr(struct g2d_data *g2d, dma_unmap_sgtable(to_dma_dev(g2d->drm_dev), g2d_userptr-&

Re: [PATCH 06/13] media: videobuf2: Move frame_vector into media subsystem

2020-10-07 Thread John Hubbard
Cc: Pawel Osciak Cc: Marek Szyprowski Cc: Kyungmin Park Cc: Tomasz Figa Cc: Mauro Carvalho Chehab Cc: Andrew Morton Cc: John Hubbard Cc: Jérôme Glisse Cc: Jan Kara Cc: Dan Williams Cc: linux...@kvack.org Cc: linux-arm-ker...@lists.infradead.org Cc: linux-samsung-...@vger.kernel.org Cc

[PATCH] Documentation: filesystems: better locations for sysfs-pci, sysfs-tagging

2020-10-07 Thread John Hubbard
to be reading about filesystems in general and then come across these specific details about PCI, and tagging...and then back to general filesystems again. Move sysfs-pci and sysfs-tagging to a location under the sysfs topic. Signed-off-by: John Hubbard --- Hi, I'm not sure if the "toctree, max

Re: [PATCH 07/13] mm: close race in generic_access_phys

2020-10-07 Thread John Hubbard
le for the same pte to already be there, ever? If so, we be stuck in an infinite loop here. I'm sure that's not the case, but it's not yet obvious to me why it's impossible. Resource reservations maybe? thanks, -- John Hubbard NVIDIA

Re: [PATCH] Documentation: filesystems: better locations for sysfs-pci, sysfs-tagging

2020-10-08 Thread John Hubbard
On 10/8/20 9:04 AM, Jonathan Corbet wrote: On Wed, 7 Oct 2020 16:31:51 -0700 John Hubbard wrote: sysfs-pci and sysfs-tagging were mis-filed: their locations with Documentation/ implied that they were related to file systems. Actually, each topic is about a very specific *use* of sysfs

Re: [PATCH 1/4] mm: introduce vma_set_file function v2

2020-10-08 Thread John Hubbard
it was already done within vma_set_file(). thanks, -- John Hubbard NVIDIA

[PATCH v2] Documentation: better locations for sysfs-pci, sysfs-tagging

2020-10-09 Thread John Hubbard
-by: John Hubbard --- Changes since v1: moved sysfs-pci to PCI, and sysfs-tagging to networking, and poked around in the generated HTML via Firefox. It looks like the right place now. thanks, John Hubbard NVIDIA Documentation/PCI/index.rst | 1 + Documentation

Re: [PATCH 1/4] mm: introduce vma_set_file function v2

2020-10-09 Thread John Hubbard
On 10/9/20 12:33 AM, Christian König wrote: Am 08.10.20 um 23:49 schrieb John Hubbard: On 10/8/20 4:23 AM, Christian König wrote: ... diff --git a/drivers/gpu/drm/i915/gem/i915_gem_mman.c b/drivers/gpu/drm/i915/gem/i915_gem_mman.c index 3d69e51f3e4d..c9d5f1a38af3 100644 --- a/drivers/gpu/drm

Re: [PATCH RFC PKS/PMEM 57/58] nvdimm/pmem: Stray access protection for pmem->virt_addr

2020-10-09 Thread John Hubbard
lumsy API design to *disable*, right?). And there is no hint about the scope. And it *could* be so much more readable like this: dev_access_enable(DEV_ACCESS_THIS_THREAD); thanks, -- John Hubbard NVIDIA

[PATCH] mm/gup: protect unpin_user_pages() against npages==-ERRNO

2020-09-17 Thread John Hubbard
pinned), but probably not. More likely, gup/pup returned a hard -ERRNO error to the caller, who erroneously passed it here. Cc: Ira Weiny Cc: Souptick Joarder Signed-off-by: Dan Carpenter Signed-off-by: John Hubbard --- Hi Dan, Is is OK to use your signed-off-by here? Since you came up

Re: [linux-next PATCH] rapidio: Fix error handling path

2020-09-17 Thread John Hubbard
to the simpler API. There are more than enough gup/pup API entry points as it is, that's for sure. thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/4] mm: Trial do_wp_page() simplification

2020-09-17 Thread John Hubbard
out the need to ever clear that flag. That *seems* reasonable, based on the use cases for these kinds of pages. 2) And then rename: page_maybe_dma_pinned() --> page_likely_dma_pinned() :) thanks, -- John Hubbard NVIDIA

Re: [PATCH 1/4] mm: Trial do_wp_page() simplification

2020-09-17 Thread John Hubbard
page pinning event done on it". We'd have to give up pin_user_pages_fast() to do that as we can't fast walk and get vmas? oops, yes. I'd forgotten about that point. Basically all of the O_DIRECT callers need _fast. This is a big problem. thanks, -- John Hubbard NVIDIA Hmm, there are man

<    1   2   3   4   5   6   7   8   9   10   >