Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-07 Thread Mike Kravetz
On 07/07/2017 01:45 AM, Anshuman Khandual wrote: > On 07/06/2017 09:47 PM, Mike Kravetz wrote: >> The mremap system call has the ability to 'mirror' parts of an existing >> mapping. To do so, it creates a new mapping that maps the same pages as >> the original mapping, just a

Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-07 Thread Mike Kravetz
On 07/07/2017 03:23 AM, Kirill A. Shutemov wrote: > On Thu, Jul 06, 2017 at 09:17:26AM -0700, Mike Kravetz wrote: >> The mremap system call has the ability to 'mirror' parts of an existing >> mapping. To do so, it creates a new mapping that maps the same pages as >> the o

Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-07 Thread Mike Kravetz
On 07/07/2017 10:45 AM, Kirill A. Shutemov wrote: > On Fri, Jul 07, 2017 at 10:29:52AM -0700, Mike Kravetz wrote: >> On 07/07/2017 03:23 AM, Kirill A. Shutemov wrote: >>> On Thu, Jul 06, 2017 at 09:17:26AM -0700, Mike Kravetz wrote: >>>> The mremap system call has

Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-10 Thread Mike Kravetz
On 07/10/2017 09:22 AM, Vlastimil Babka wrote: > On 07/09/2017 09:32 AM, Anshuman Khandual wrote: >> On 07/07/2017 11:39 PM, Mike Kravetz wrote: >>> On 07/07/2017 10:45 AM, Kirill A. Shutemov wrote: >>>> On Fri, Jul 07, 2017 at 10:29:52AM -0700, Mike Kravetz wrote

Re: [PATCH] mm/mremap: Document MREMAP_FIXED dependency on MREMAP_MAYMOVE

2017-07-10 Thread Mike Kravetz
ammer, there is no need as this is clearly specified in the man page: "If MREMAP_FIXED is specified, then MREMAP_MAYMOVE must also be specified." -- Mike Kravetz > >> Signed-off-by: Anshuman Khandual >> --- >> include/uapi/linux/mman.h | 6 -- >> 1 fil

Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-11 Thread Mike Kravetz
On 07/11/2017 05:36 AM, Michal Hocko wrote: > On Thu 06-07-17 09:17:26, Mike Kravetz wrote: >> The mremap system call has the ability to 'mirror' parts of an existing >> mapping. To do so, it creates a new mapping that maps the same pages as >> the original mapping, just a

Re: [RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-11 Thread Mike Kravetz
On 07/11/2017 02:02 PM, Andrea Arcangeli wrote: > On Tue, Jul 11, 2017 at 11:23:19AM -0700, Mike Kravetz wrote: >> I was surprised as well when a JVM developer pointed this out. >> >> From the old e-mail thread, here is original use case: >> shmget(IPC_PRIVATE, 31498

[RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag

2017-07-06 Thread Mike Kravetz
of a huge page mapping, but it can not be used to expand or mirror a mapping. Such support is fairly straight forward. [1] https://lkml.org/lkml/2004/1/12/260 Mike Kravetz (1): mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality include/uapi/linux/mman.h | 5

[RFC PATCH 1/1] mm/mremap: add MREMAP_MIRROR flag for existing mirroring functionality

2017-07-06 Thread Mike Kravetz
be specified. Signed-off-by: Mike Kravetz --- include/uapi/linux/mman.h | 5 +++-- mm/mremap.c | 23 --- tools/include/uapi/linux/mman.h | 5 +++-- 3 files changed, 22 insertions(+), 11 deletions(-) diff --git a/include/uapi/linux/mman.h b/include/uapi/linux

Re: [RFC PATCH 0/1] mm/mremap: add MREMAP_MIRROR flag

2017-07-07 Thread Mike Kravetz
On 07/07/2017 01:19 AM, Anshuman Khandual wrote: > On 07/06/2017 09:47 PM, Mike Kravetz wrote: >> The mremap system call has the ability to 'mirror' parts of an existing >> mapping. To do so, it creates a new mapping that maps the same pages as >> the original mapping, just a

[RFC PATCH 1/3] mm:hugetlb: Define system call hugetlb size encodings in single file

2017-07-17 Thread Mike Kravetz
for these encodings. Put common definitions in a single header file. arch specific code can still override if desired. Signed-off-by: Mike Kravetz --- include/uapi/asm-generic/hugetlb_encode.h | 30 ++ 1 file changed, 30 insertions(+) create mode 100644 include/uapi/asm

Re: [PATCH] mm,hugetlb: compute page_size_log properly

2017-07-17 Thread Mike Kravetz
the shm user space definitions in the uapi file as previously suggested by Matthew Wilcox. I did not (yet) move the shm definitions to arch specific files as suggested by Aneesh Kumar. [1] https://lkml.org/lkml/2017/7/6/564 Mike Kravetz (3): mm:hugetlb: Define system call hugetlb size encodings

[RFC PATCH 2/3] mm: arch: Use new hugetlb size encoding definitions

2017-07-17 Thread Mike Kravetz
Use the common definitions from hugetlb_encode.h header file for encoding hugetlb size definitions in mmap system call flags. Signed-off-by: Mike Kravetz --- arch/alpha/include/uapi/asm/mman.h | 14 ++ arch/mips/include/uapi/asm/mman.h | 14 ++ arch/parisc

[RFC PATCH 3/3] mm: shm: Use new hugetlb size encoding definitions

2017-07-17 Thread Mike Kravetz
Use the common definitions from hugetlb_encode.h header file for encoding hugetlb size definitions in shmget system call flags. In addition, move these definitions to the from the internal to user (uapi) header file. Suggested-by: Matthew Wilcox Signed-off-by: Mike Kravetz --- include/linux

Re: [PATCH] mm/mremap: Fail map duplication attempts for private mappings

2017-07-19 Thread Mike Kravetz
On 07/13/2017 12:11 PM, Vlastimil Babka wrote: > [+CC linux-api] > > On 07/13/2017 05:58 PM, Mike Kravetz wrote: >> mremap will create a 'duplicate' mapping if old_size == 0 is >> specified. Such duplicate mappings make no sense for private >> mappings. If duplication

[PATCH v2] mm/mremap: Fail map duplication attempts for private mappings

2017-07-20 Thread Mike Kravetz
-by: Mike Kravetz --- mm/mremap.c | 9 + 1 file changed, 9 insertions(+) diff --git a/mm/mremap.c b/mm/mremap.c index cd8a1b1..949f6a7 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -383,6 +383,15 @@ static struct vm_area_struct *vma_to_resize(unsigned long addr, if (!vma || vma

Re: [PATCH] selftests/vm: Add test to validate mirror functionality with mremap

2017-07-20 Thread Mike Kravetz
On 07/20/2017 02:36 AM, Anshuman Khandual wrote: > This adds a test to validate mirror functionality with mremap() > system call on shared anon mappings. > > Suggested-by: Mike Kravetz > Signed-off-by: Anshuman Khandual > --- > tools/testing/selftests/vm/Makefil

Re: [PATCH v2] mm/mremap: Fail map duplication attempts for private mappings

2017-07-21 Thread Mike Kravetz
On 07/21/2017 07:36 AM, Michal Hocko wrote: > On Thu 20-07-17 13:37:59, Mike Kravetz wrote: >> mremap will create a 'duplicate' mapping if old_size == 0 is >> specified. Such duplicate mappings make no sense for private >> mappings. > > sorry for the nit picking bu

[PATCH] sparc64: mm: fix copy_tsb to correctly copy huge page TSBs

2017-06-02 Thread Mike Kravetz
-off-by: Mike Kravetz --- arch/sparc/kernel/tsb.S | 11 +++ arch/sparc/mm/tsb.c | 7 +-- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/sparc/kernel/tsb.S b/arch/sparc/kernel/tsb.S index 10689cf..07c0df9 100644 --- a/arch/sparc/kernel/tsb.S +++ b/arch/sparc

Re: [PATCH v2] mm: show total hugetlb memory consumption in /proc/meminfo

2017-11-21 Thread Mike Kravetz
that there is no consistency guarantee for the numbers with the default huge page size today. However, I am not really a fan of taking the lock for that guarantee. IMO, the above code is fine. This discussion reminds me that ideally there should be a per-hstate lock. My guess is that the globa

Re: [RFC PATCH 0/3] restructure memfd code

2017-11-21 Thread Mike Kravetz
On 11/21/2017 08:32 AM, Khalid Aziz wrote: > On Wed, 2017-11-08 at 17:41 -0800, Mike Kravetz wrote: >> With the addition of memfd hugetlbfs support, we now have the >> situation >> where memfd depends on TMPFS -or- HUGETLBFS. Previously, memfd was >> only >> sup

Re: [PATCH 1/1] mm/cma: fix alloc_contig_range ret code/potential leak

2017-11-22 Thread Mike Kravetz
On 11/22/2017 04:00 AM, Johannes Weiner wrote: > On Mon, Nov 20, 2017 at 11:39:30AM -0800, Mike Kravetz wrote: >> If the call __alloc_contig_migrate_range() in alloc_contig_range >> returns -EBUSY, processing continues so that test_pages_isolated() >> is called where

[PATCH v2] mm/cma: fix alloc_contig_range ret code/potential leak

2017-11-22 Thread Mike Kravetz
if __alloc_contig_migrate_range returns -EBUSY. Also, clear return code in this case so that it is not accidentally used or returned to caller. Fixes: 8ef5849fa8a2 ("mm/cma: always check which page caused allocation failure") Cc: Signed-off-by: Mike Kravetz --- mm/page_alloc.c | 9 ++

Re: hugetlb page migration vs. overcommit

2017-11-22 Thread Mike Kravetz
ce Naoya was originally involved in huge page migration, I would welcome his comments. -- Mike Kravetz

Re: [PATCH RFC 1/2] mm, hugetlb: unify core page allocation accounting and initialization

2017-11-29 Thread Mike Kravetz
On 11/28/2017 10:57 PM, Michal Hocko wrote: > On Tue 28-11-17 13:34:53, Mike Kravetz wrote: >> On 11/28/2017 06:12 AM, Michal Hocko wrote: > [...] >>> +/* >>> + * Allocates a fresh page to the hugetlb allocator pool in the node >>> interleaved &

Re: [PATCH RFC 2/2] mm, hugetlb: do not rely on overcommit limit during migration

2017-11-29 Thread Mike Kravetz
if (h->surplus_huge_pages_node[old_nid]) { > + h->surplus_huge_pages_node[old_nid]--; > + h->surplus_huge_pages_node[new_nid]++; > + } You need to take hugetlb_lock before adjusting the surplus counts. -- Mike Kravetz

Re: [PATCH RFC 2/2] mm, hugetlb: do not rely on overcommit limit during migration

2017-11-30 Thread Mike Kravetz
On 11/29/2017 11:57 PM, Michal Hocko wrote: > On Wed 29-11-17 11:52:53, Mike Kravetz wrote: >> On 11/29/2017 01:22 AM, Michal Hocko wrote: >>> What about this on top. I haven't tested this yet though. >> >> Yes, this would work. >> >> However, I th

Re: [PATCH v2 5/9] shmem: add sealing support to hugetlb-backed memfd

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > Adapt add_seals()/get_seals() to work with hugetbfs-backed memory. > > Teach memfd_create() to allow sealing operations on MFD_HUGETLB. > > Signed-off-by: Marc-André Lureau Looks fine to me, Reviewed-by: Mike Kravetz -

Re: [PATCH v2 6/9] memfd-tests: test hugetlbfs sealing

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > Remove most of the special-casing of hugetlbfs now that sealing > is supported. > > Signed-off-by: Marc-André Lureau Reviewed-by: Mike Kravetz -- Mike Kravetz > --- > tools/testing/selftests/memfd/

Re: [PATCH v2 7/9] memfd-test: add 'memfd-hugetlb:' prefix when testing hugetlbfs

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > Suggested-by: Mike Kravetz > Signed-off-by: Marc-André Lureau Reviewed-by: Mike Kravetz -- Mike Kravetz > --- > tools/testing/selftests/memfd/memfd_test.c | 26 -- > 1 file changed, 16 insertions(+

Re: [PATCH v2 8/9] memfd-test: move common code to a shared unit

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > The memfd & fuse tests will share more common code in the following > commits to test hugetlb support. > > Signed-off-by: Marc-André Lureau Reviewed-by: Mike Kravetz -- Mike Kravetz > --- > tools/testing/selftests/m

Re: [PATCH v2 9/9] memfd-test: run fuse test on hugetlb backend memory

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > Suggested-by: Mike Kravetz > Signed-off-by: Marc-André Lureau > --- > tools/testing/selftests/memfd/fuse_test.c | 30 > ++ > tools/testing/selftests/memfd/run_fuse_test.sh | 2 +- > tools/te

Re: [PATCH v2 0/9] memfd: add sealing to hugetlb-backed memory

2017-11-06 Thread Mike Kravetz
On 11/06/2017 06:39 AM, Marc-André Lureau wrote: > Hi, > > Recently, Mike Kravetz added hugetlbfs support to memfd. However, he > didn't add sealing support. One of the reasons to use memfd is to have > shared memory sealing when doing IPC or sharing memory with another > proce

Re: [PATCH v3 9/9] memfd-test: run fuse test on hugetlb backend memory

2017-11-07 Thread Mike Kravetz
On 11/07/2017 04:28 AM, Marc-André Lureau wrote: > Suggested-by: Mike Kravetz > Signed-off-by: Marc-André Lureau Reviewed-by: Mike Kravetz -- Mike Kravetz > --- > tools/testing/selftests/memfd/fuse_test.c | 38 > -- > tools/testi

Re: [RFC PATCH 1/5] mm, hugetlb: unify core page allocation accounting and initialization

2017-12-12 Thread Mike Kravetz
cator pool. All current callers are updated to call > put_page explicitly. Later patches will add new callers which won't > need it. > > This patch shouldn't introduce any functional change. > > Signed-off-by: Michal Hocko Re

Re: [RFC PATCH 2/5] mm, hugetlb: integrate giga hugetlb more naturally to the allocation path

2017-12-12 Thread Mike Kravetz
uge_pages > which doesn't have to care about what kind of huge page we allocate. > > Signed-off-by: Michal Hocko I agree with the analysis. Thanks for cleaning this up. There really is no need for the separate allocation paths. Reviewed-by: Mike Kravetz -- M

Re: [RFC PATCH 3/5] mm, hugetlb: do not rely on overcommit limit during migration

2017-12-13 Thread Mike Kravetz
e per-node's. > + */ > + if (PageHugeTemporary(newpage)) { > + int old_nid = page_to_nid(oldpage); > + int new_nid = page_to_nid(newpage); > + > + SetPageHugeTemporary(oldpage); > + ClearPageHugeTemporary(newpage); > + >

Re: [RFC PATCH 4/5] mm, hugetlb: get rid of surplus page accounting tricks

2017-12-13 Thread Mike Kravetz
*/ > h->nr_huge_pages_node[r_nid]++; > h->surplus_huge_pages_node[r_nid]++; > - } else { > - h->nr_huge_pages--; > - h->surplus_huge_pages--; In the case of a successful surplus allocation, the following

Re: [PATCH v2 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-20 Thread Mike Kravetz
On 1/20/21 1:30 AM, Oscar Salvador wrote: > On Tue, Jan 19, 2021 at 05:30:45PM -0800, Mike Kravetz wrote: >> + * Macros to create test, set and clear function definitions for >> + * hugetlb specific page flags. >> + */ >> +#ifdef CONFIG_HUGETLB_PAGE >> +#d

Re: [PATCH v2 4/5] hugetlb: convert PageHugeTemporary() to HPageTemporary flag

2021-01-20 Thread Mike Kravetz
On 1/20/21 2:09 AM, Oscar Salvador wrote: > On Tue, Jan 19, 2021 at 05:30:48PM -0800, Mike Kravetz wrote: >> Use new hugetlb specific HPageTemporary flag to replace the >> PageHugeTemporary() interfaces. >> >> Signed-off-by: Mike Kravetz > > I would have add

Re: [RFC PATCH 2/3] hugetlb: convert page_huge_active() to HPageMigratable flag

2021-01-15 Thread Mike Kravetz
On 1/15/21 9:43 AM, Mike Kravetz wrote: > On 1/15/21 1:17 AM, Oscar Salvador wrote: >> On Mon, Jan 11, 2021 at 01:01:51PM -0800, Mike Kravetz wrote: >>> Use the new hugetlb page specific flag to replace the page_huge_active >>> interfaces. By it's name, page_huge_acti

Re: [RFC PATCH 2/3] hugetlb: convert page_huge_active() to HPageMigratable flag

2021-01-15 Thread Mike Kravetz
to be something like: 1) allocate a fresh hugetlb page from buddy 2) free the 'migrated' free huge page back to buddy I do not think we can use the existing 'isolate-migrate' flow. Isolating a page would make it unavailable for allocation and that could cause application issues. -- Mike Kravetz

Re: [PATCH v2 2/5] hugetlb: convert page_huge_active() HPageMigratable flag

2021-01-20 Thread Mike Kravetz
On 1/20/21 2:00 AM, Oscar Salvador wrote: > On Wed, Jan 20, 2021 at 10:59:05AM +0100, Oscar Salvador wrote: >> On Tue, Jan 19, 2021 at 05:30:46PM -0800, Mike Kravetz wrote: >>> Use the new hugetlb page specific flag HPageMigratable to replace the >>> page_huge_activ

[PATCH 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-15 Thread Mike Kravetz
patches. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 12 ++-- include/linux/hugetlb.h | 61 + mm/hugetlb.c| 46 +++ 3 files changed, 87 insertions(+), 32 deletions(-) diff --git a/fs/hugetlbfs/inode.c b

[PATCH 0/5] create hugetlb flags to consolidate state

2021-01-15 Thread Mike Kravetz
set of flag manipulation routines (Oscar) Moved flags and routines to hugetlb.h (Muchun) Changed format of page flag names (Muchun) Changed subpool routine names (Matthew) More comments in code (Oscar) Based on v5.11-rc3-mmotm-2021-01-12-01-57 Mike Kravetz (5): hugetlb: use page.priv

[PATCH 2/5] hugetlb: convert page_huge_active() to HP_Migratable flag

2021-01-15 Thread Mike Kravetz
with code freeing the page. The extra check in page_huge_active shortened the race window, but did not prevent the race. Offline code calling scan_movable_pages already deals with these races, so removing the check is acceptable. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c | 2

[PATCH 4/5] hugetlb: convert PageHugeTemporary() to HP_Temporary flag

2021-01-15 Thread Mike Kravetz
Use new hugetlb specific flag HP_Temporary flag to replace the PageHugeTemporary() interfaces. Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 5 + mm/hugetlb.c| 36 +++- 2 files changed, 12 insertions(+), 29 deletions(-) diff --git

[PATCH 3/5] hugetlb: only set HP_Migratable for migratable hstates

2021-01-15 Thread Mike Kravetz
necessary. If migration is not supported for the hstate, HP_Migratable will not be set, the page will not be isolated and no attempt will be made to migrate. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 2 +- include/linux/hugetlb.h | 9 + mm/hugetlb.c| 8

[PATCH 5/5] hugetlb: convert PageHugeFreed to HP_Freed flag

2021-01-15 Thread Mike Kravetz
Use new hugetlb specific flag HP_Freed flag to replace the PageHugeFreed interfaces. Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 2 ++ mm/hugetlb.c| 23 --- 2 files changed, 6 insertions(+), 19 deletions(-) diff --git a/include/linux/hugetlb.h b

Re: [PATCH 2/5] hugetlb: convert page_huge_active() to HP_Migratable flag

2021-01-16 Thread Mike Kravetz
s need to be set/tested outside hugetlb code, so > it indeed looks nicer and more consistent to follow page-flags.h convention. > > Sorry for the noise. Thanks everyone! I was unsure about the best way to go for this. Will send out a new version in a few days using the page-flag style macros. -- Mike Kravetz

[PATCH v3 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-22 Thread Mike Kravetz
information will happen in subsequent patches. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 12 ++-- include/linux/hugetlb.h | 68 + mm/hugetlb.c| 48 +++-- 3 files changed, 96 insertions(+), 32 deletions

[PATCH v3 5/5] hugetlb: convert PageHugeFreed to HPageFreed flag

2021-01-22 Thread Mike Kravetz
Use new hugetlb specific HPageFreed flag to replace the PageHugeFreed interfaces. Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador Reviewed-by: Muchun Song --- include/linux/hugetlb.h | 3 +++ mm/hugetlb.c| 23 --- 2 files changed, 7 insertions(+), 19

[PATCH v3 4/5] hugetlb: convert PageHugeTemporary() to HPageTemporary flag

2021-01-22 Thread Mike Kravetz
. Signed-off-by: Mike Kravetz Reviewed-by: Oscar Salvador --- include/linux/hugetlb.h | 6 ++ mm/hugetlb.c| 36 +++- 2 files changed, 13 insertions(+), 29 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index cd1960541f2a

[PATCH v3 0/5] create hugetlb flags to consolidate state

2021-01-22 Thread Mike Kravetz
v5.11-rc4-mmotm-2021-01-21-20-07 Mike Kravetz (5): hugetlb: use page.private for hugetlb specific page flags hugetlb: convert page_huge_active() HPageMigratable flag hugetlb: only set HPageMigratable for migratable hstates hugetlb: convert PageHugeTemporary() to HPageTemporary flag hugetlb

Re: [PATCH v13 03/12] mm: hugetlb: free the vmemmap pages associated with each HugeTLB page

2021-01-22 Thread Mike Kravetz
_each_entry_safe(page, next, list, lru) { > + list_del(>lru); > + free_vmemmap_page(page); > + } > +} > + > +static void vmemmap_remap_pte(pte_t *pte, unsigned long addr, > + struct vmemmap_remap_walk *walk) > +{ &g

Re: [PATCH v2 2/5] hugetlb: convert page_huge_active() HPageMigratable flag

2021-01-22 Thread Mike Kravetz
On 1/21/21 10:53 PM, Miaohe Lin wrote: > Hi: > On 2021/1/20 9:30, Mike Kravetz wrote: >> Use the new hugetlb page specific flag HPageMigratable to replace the >> page_huge_active interfaces. By it's name, page_huge_active implied >> that a huge page was on t

[PATCH v3 2/5] hugetlb: convert page_huge_active() HPageMigratable flag

2021-01-22 Thread Mike Kravetz
race with code freeing the page. The extra check in page_huge_active shortened the race window, but did not prevent the race. Offline code calling scan_movable_pages already deals with these races, so removing the check is acceptable. Add comment to racy code. Signed-off-by: Mike Kravetz

[PATCH v3 3/5] hugetlb: only set HPageMigratable for migratable hstates

2021-01-22 Thread Mike Kravetz
will not be isolated and no attempt will be made to migrate. We should never get to unmap_and_move_huge_page for a page where migration is not supported, so throw a warning if we do. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 2 +- include/linux/hugetlb.h | 9 + mm/hugetlb.c

Re: [PATCH v2] Documentation/admin-guide: kernel-parameters: update CMA entries

2021-01-25 Thread Mike Kravetz
n Corbet > Cc: linux-...@vger.kernel.org > Cc: linux...@kvack.org > Cc: Andrew Morton > Cc: Mike Kravetz > --- > v2: rebase & resend > > Documentation/admin-guide/kernel-parameters.txt |8 > 1 file changed, 4 insertions(+), 4 deletions(-)

Re: [External] Re: [PATCH v13 05/12] mm: hugetlb: allocate the vmemmap pages associated with each HugeTLB page

2021-01-25 Thread Mike Kravetz
from there ... > > Yeah, you are right. So I tend to trigger OOM to kill other processes to > reclaim some memory when we allocate memory fails. IIUC, even non-gigantic hugetlb pages can exist in CMA. They can be migrated out of CMA if needed (except free pages in the pool, but that is a separate issue David H already noted in another thread). When we first started discussing this patch set, one suggestion was to force hugetlb pool pages to be allocated at boot time and never permit them to be freed back to the buddy allocator. A primary reason for the suggestion was to avoid this issue of needing to allocate memory when freeing a hugetlb page to buddy. IMO, that would be an unreasonable restriction for many existing hugetlb use cases. A simple thought is that we simply fail the 'freeing hugetlb page to buddy' if we can not allocate the required vmemmap pages. However, as David R says freeing hugetlb pages to buddy is a reasonable way to free up memory in oom situations. However, failing the operation 'might' be better than looping forever trying to allocate the pages needed? As mentioned in the previous patch, it would be better to use GFP_ATOMIC to at least dip into reserves if we can. I think using pages of the hugetlb for vmemmap to cover pages of the hugetlb is the only way we can guarantee success of freeing a hugetlb page to buddy. However, this should only only be used when there is no other option and could result in vmemmap pages residing in CMA or ZONE_MOVABLE. I'm not sure how much better this is than failing the free to buddy operation. I don't have a solution. Just wanted to share some thoughts. BTW, just thought of something else. Consider offlining a memory section that contains a free hugetlb page. The offline code will try to disolve the hugetlb page (free to buddy). So, vmemmap pages will need to be allocated. We will try to allocate vmemap pages on the same node as the hugetlb page. But, if this memory section is the last of the node all the pages will have been isolated and no allocations will succeed. Is that a possible scenario, or am I just having too many negative thoughts? -- Mike Kravetz

Re: [PATCH] hugetlbfs: make hugepage size conversion more readable

2021-01-21 Thread Mike Kravetz
On 1/21/21 5:42 PM, Miaohe Lin wrote: > Hi: > On 2021/1/22 3:00, Mike Kravetz wrote: >> On 1/20/21 1:23 AM, Miaohe Lin wrote: >>> The calculation 1U << (h->order + PAGE_SHIFT - 10) is actually equal to >>> (PAGE_SHIFT << (h->order)) >&

Re: [PATCH] hugetlbfs: make hugepage size conversion more readable

2021-01-21 Thread Mike Kravetz
the kernel, size in KB is often calculated as (size << (PAGE_SHIFT - 10)). If you change the calculation in the hugetlb code to be: huge_page_size(h) << (PAGE_SHIFT - 10) my compiler will actually reduce the size of the routine by one instruction. -- Mike Kravetz > return mnt; > } > >

Re: [PATCH v2] hugetlbfs: remove meaningless variable avoid_reserve

2021-01-21 Thread Mike Kravetz
dd a comment offered by Mike Kravetz to explain this. > > Reviewed-by: David Hildenbrand > Signed-off-by: Miaohe Lin > Cc: Mike Kravetz > --- > fs/hugetlbfs/inode.c | 12 +--- > 1 file changed, 9 insertions(+), 3 deletions(-) Reviewed-by: Mike Kravetz > >

Re: [PATCH v2] hugetlbfs: correct obsolete function name in hugetlbfs_read_iter()

2021-01-19 Thread Mike Kravetz
t; to generic_file_buffered_read(). So replace do_generic_mapping_read() with > generic_file_buffered_read() to keep comment uptodate. > > Signed-off-by: Miaohe Lin > --- > fs/hugetlbfs/inode.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Thanks, Reviewed-by: Mik

Re: [PATCH] hugetlbfs: Use helper macro default_hstate in init_hugetlbfs_fs

2021-01-19 Thread Mike Kravetz
CC Andrew On 1/19/21 9:53 AM, Mike Kravetz wrote: > On 1/16/21 1:18 AM, Miaohe Lin wrote: >> Since commit e5ff215941d5 ("hugetlb: multiple hstates for multiple page >> sizes"), we can use macro default_hstate to get the struct hstate which >> we use by default.

Re: [PATCH] hugetlbfs: remove meaningless variable avoid_reserve

2021-01-19 Thread Mike Kravetz
s with * reserves for the file at the inode level. If we fallocate * pages in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ -- Mike Kravetz > - page = alloc_hug

Re: [PATCH] hugetlbfs: Use helper macro default_hstate in init_hugetlbfs_fs

2021-01-19 Thread Mike Kravetz
f-by: Miaohe Lin Thanks, Reviewed-by: Mike Kravetz -- Mike Kravetz > --- > fs/hugetlbfs/inode.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c > index 9b221b87fbea..88751e35e69d 100644 > --- a/fs/hugetlbfs/ino

Re: [PATCH v2] hugetlbfs: Remove useless BUG_ON(!inode) in hugetlbfs_setattr()

2021-01-19 Thread Mike Kravetz
-- > fs/hugetlbfs/inode.c | 2 -- > 1 file changed, 2 deletions(-) Thanks, Reviewed-by: Mike Kravetz -- Mike Kravetz > > diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c > index 016c863b493b..79464963f95e 100644 > --- a/fs/hugetlbfs/inode.c > +++ b/fs/hugetlbfs/i

Re: [PATCH v2] mm/hugetlb: avoid unnecessary hugetlb_acct_memory() call

2021-01-19 Thread Mike Kravetz
Miaohe Lin >> >> I would avoid mentioning gbl_reserve as not all callers use it, and focus >> on what delta means: >> >> "When reservation accounting remains unchanged..", but anyway: > > Sounds good. Maybe Andrew could kindly do this if this patch is picked up ? Thank you and Andrew. Looks like Andrew updated the commit message and added to his tree. -- Mike Kravetz

[PATCH v2 1/5] hugetlb: use page.private for hugetlb specific page flags

2021-01-19 Thread Mike Kravetz
information will happen in subsequent patches. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 12 ++- include/linux/hugetlb.h | 72 + mm/hugetlb.c| 45 +- 3 files changed, 97 insertions(+), 32 deletions(-) diff

[PATCH v2 4/5] hugetlb: convert PageHugeTemporary() to HPageTemporary flag

2021-01-19 Thread Mike Kravetz
Use new hugetlb specific HPageTemporary flag to replace the PageHugeTemporary() interfaces. Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 6 ++ mm/hugetlb.c| 36 +++- 2 files changed, 13 insertions(+), 29 deletions(-) diff --git

[PATCH v2 2/5] hugetlb: convert page_huge_active() HPageMigratable flag

2021-01-19 Thread Mike Kravetz
race with code freeing the page. The extra check in page_huge_active shortened the race window, but did not prevent the race. Offline code calling scan_movable_pages already deals with these races, so removing the check is acceptable. Add comment to racy code. Signed-off-by: Mike Kravetz --- fs

[PATCH v2 3/5] hugetlb: only set HPageMigratable for migratable hstates

2021-01-19 Thread Mike Kravetz
will not be isolated and no attempt will be made to migrate. We should never get to unmap_and_move_huge_page for a page where migration is not supported, so throw a warning if we do. Signed-off-by: Mike Kravetz --- fs/hugetlbfs/inode.c| 2 +- include/linux/hugetlb.h | 9 + mm/hugetlb.c

[PATCH v2 5/5] hugetlb: convert PageHugeFreed to HPageFreed flag

2021-01-19 Thread Mike Kravetz
Use new hugetlb specific HPageFreed flag to replace the PageHugeFreed interfaces. Signed-off-by: Mike Kravetz --- include/linux/hugetlb.h | 3 +++ mm/hugetlb.c| 23 --- 2 files changed, 7 insertions(+), 19 deletions(-) diff --git a/include/linux/hugetlb.h b

[PATCH v2 0/5] create hugetlb flags to consolidate state

2021-01-19 Thread Mike Kravetz
hanged subpool routine names (Matthew) More comments in code (Oscar) Based on v5.11-rc3-mmotm-2021-01-12-01-57 Mike Kravetz (5): hugetlb: use page.private for hugetlb specific page flags hugetlb: convert page_huge_active() HPageMigratable flag hugetlb: only set HPageMigratable for migratable h

Re: [RFC PATCH 1/3] hugetlb: use page.private for hugetlb specific page flags

2021-01-13 Thread Mike Kravetz
On 1/13/21 5:54 AM, Oscar Salvador wrote: > On Mon, Jan 11, 2021 at 01:01:50PM -0800, Mike Kravetz wrote: >> As hugetlbfs evolved, state information about hugetlb pages was added. >> One 'convenient' way of doing this was to use available fields in tail >> pages. Ove

Re: [RFC PATCH 1/3] hugetlb: use page.private for hugetlb specific page flags

2021-01-13 Thread Mike Kravetz
On 1/13/21 6:45 AM, Matthew Wilcox wrote: > On Mon, Jan 11, 2021 at 01:01:50PM -0800, Mike Kravetz wrote: >> +if (hpage_spool(page)) { > > Could this be named hpage_subpool(), please? > Of course! -- Mike Kravetz

Re: [PATCH] mm/hugetlb: avoid unnecessary hugetlb_acct_memory() call

2021-01-14 Thread Mike Kravetz
this code path. However, there are other code paths where hugetlb_acct_memory is called with a delta value of 0 as well. I would rather see a simple check at the beginning of hugetlb_acct_memory like. if (!delta) return 0; -- Mike Kravetz >> >> return 0; >> } >> > >

Re: [PATCH] mm/hugetlb: Use helper huge_page_order and pages_per_huge_page

2021-01-14 Thread Mike Kravetz
; use it. > > Signed-off-by: Miaohe Lin > --- > mm/hugetlb.c | 4 ++-- > 1 file changed, 2 insertions(+), 2 deletions(-) Thanks, Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH v2 2/6] mm: hugetlbfs: fix cannot migrate the fallocated HugeTLB page

2021-01-06 Thread Mike Kravetz
tion could race with the page fault and the page could be migrated before being added to the page table of the faulting task. This was an issue when hugetlb_no_page set_page_huge_active right after allocating and clearing the huge page. Commit cb6acd01e2e4 moved the set_page_huge_active after adding the page to the page table to address this issue. -- Mike Kravetz

Re: [PATCH v2 3/6] mm: hugetlb: fix a race between freeing and dissolving the page

2021-01-06 Thread Mike Kravetz
d_gigantic_page(struct page *page, >> unsigned int order) { } >> #endif >> >> +/* >> + * Because we reuse the mapping field of some tail page structs, we should >> + * reset those mapping to initial value before @

Re: [PATCH v2 2/6] mm: hugetlbfs: fix cannot migrate the fallocated HugeTLB page

2021-01-06 Thread Mike Kravetz
On 1/6/21 12:02 PM, Michal Hocko wrote: > On Wed 06-01-21 11:30:25, Mike Kravetz wrote: >> On 1/6/21 8:35 AM, Michal Hocko wrote: >>> On Wed 06-01-21 16:47:35, Muchun Song wrote: >>>> Because we only can isolate a active page via isolate_huge_page() >>>>

Re: [PATCH] mm/hugetlb: Fix potential missing huge page size info

2021-01-07 Thread Mike Kravetz
-off-by: Miaohe Lin > Cc: > --- > mm/hugetlb.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) Thanks! Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH] mm/hugetlb: Fix potential double free in hugetlb_register_node() error path

2021-01-07 Thread Mike Kravetz
. > > Fixes: a3437870160c ("hugetlb: new sysfs interface") > Signed-off-by: Miaohe Lin > Cc: > --- > mm/hugetlb.c | 4 +++- > 1 file changed, 3 insertions(+), 1 deletion(-) Thanks, this is a potential issue that should be fixed. Reviewed-by: Mike Kravetz This

Re: [PATCH 3/6] hugetlb: add free page reporting support

2021-01-07 Thread Mike Kravetz
to_nid(page); > + > + list_move(>lru, >hugepage_freelists[nid]); > +} The above routines move pages between the free and active lists without any update to free page counts. How does that work? Will the number of entries on the free list get out of sync with the free_huge_pages counters? -- Mike Kravetz

Re: [PATCH v2 3/6] mm: hugetlb: fix a race between freeing and dissolving the page

2021-01-07 Thread Mike Kravetz
On 1/7/21 12:40 AM, Michal Hocko wrote: > On Wed 06-01-21 12:58:29, Mike Kravetz wrote: >> On 1/6/21 8:56 AM, Michal Hocko wrote: >>> On Wed 06-01-21 16:47:36, Muchun Song wrote: >>>> There is a race condition between __free_huge_page() >>>> and

Re: [External] Re: [PATCH v2 3/6] mm: hugetlb: fix a race between freeing and dissolving the page

2021-01-07 Thread Mike Kravetz
touch > the page->mapping. So we need to check PageHuge(). > The check and llist_add() should be protected by > hugetlb_lock. But we cannot do that. Right? If dissolve > happens after it is linked to the list. We also should > remove it from the list (hpage_freelist). It seems to make > the thing more complex. You are correct. This is also an issue/potential problem with this race. It seems that adding the state information might be the least complex way to address issue. -- Mike Kravetz

Re: [PATCH v3 1/6] mm: migrate: do not migrate HugeTLB page whose refcount is one

2021-01-12 Thread Mike Kravetz
of that code. Free huge pages have a ref count of 0, and this code is checking for ref count of 1. That is a long way of saying that I still agree with this patch. The only modification/suggestion would be enhancing the commit message as suggested by Michal. -- Mike Kravetz

Re: [PATCH v12 03/13] mm: Introduce VM_WARN_ON_PAGE macro

2021-01-13 Thread Mike Kravetz
e/need for this macro in the following patch. Looks like Oscar has already done that, and free_bootmem_page will now use VM_BUG_ON_PAGE. So, this patch can be dropped. -- Mike Kravetz

Re: [RFC PATCH 2/3] hugetlb: convert page_huge_active() to HPageMigratable flag

2021-01-15 Thread Mike Kravetz
On 1/15/21 1:17 AM, Oscar Salvador wrote: > On Mon, Jan 11, 2021 at 01:01:51PM -0800, Mike Kravetz wrote: >> Use the new hugetlb page specific flag to replace the page_huge_active >> interfaces. By it's name, page_huge_active implied that a huge page >> was on the

Re: [RFC PATCH 3/3] hugetlb: convert PageHugeTemporary() to HPageTempSurplus

2021-01-15 Thread Mike Kravetz
On 1/15/21 2:16 AM, Oscar Salvador wrote: > On Mon, Jan 11, 2021 at 01:01:52PM -0800, Mike Kravetz wrote: >> Use new hugetlb specific flag HPageTempSurplus to replace the >> PageHugeTemporary() interfaces. >> >> Signed-off-by: Mike Kravetz &

Re: [External] Re: [PATCH v12 04/13] mm/hugetlb: Free the vmemmap pages associated with each HugeTLB page

2021-01-13 Thread Mike Kravetz
ions. - If we see this being used for more general purpose remapping, then we should go with a more general purpose implementation. Again, just my opinion. -- Mike Kravetz

Re: [PATCH v12 12/13] mm/hugetlb: Gather discrete indexes of tail page

2021-01-13 Thread Mike Kravetz
uot; was done before I even noticed your efforts here. At least we agree the metadata could be better organized. :) IMO, using page.private of the head page to consolidate flags will be easier to manage. So, I would like to use that. The BUILD_BUG_ON in this patch makes sense. -- Mike Kravetz

Re: [v5] mm: khugepaged: recalculate min_free_kbytes after memory hotplug as expected by khugepaged

2020-10-02 Thread Mike Kravetz
On 10/2/20 4:25 AM, Michal Hocko wrote: > On Wed 30-09-20 15:03:11, Mike Kravetz wrote: >> On 9/30/20 1:47 PM, Vijay Balakrishna wrote: >>> On 9/30/2020 11:20 AM, Mike Kravetz wrote: >>>> On 9/29/20 9:49 AM, Vijay Balakrishna wrote: >>>> >>

Re: [PATCH 01/10] mm/hugetlb: not necessary to coalesce regions recursively

2020-08-10 Thread Mike Kravetz
s! It is unfortunate that the region management code is difficult to understand. Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH 02/10] mm/hugetlb: make sure to get NULL when list is empty

2020-08-10 Thread Mike Kravetz
list_first_entry_or_null(>region_cache, >>> + struct file_region, link); >>> VM_BUG_ON(!nrg); > > Or we can remove this VM_BUG_ON()? > I would prefer that we just remove the 'VM_BUG_ON(!nrg)'. Code elsewhere is responsible for making sure there is ALWAYS an entry in the cache. That is why the 'VM_BUG_ON(resv->region_cache_count <= 0)' is at the beginning of the routine. -- Mike Kravetz

Re: [PATCH 03/10] mm/hugetlb: use list_splice to merge two list at once

2020-08-10 Thread Mike Kravetz
s! Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH 04/10] mm/hugetlb: count file_region to be added when regions_needed != NULL

2020-08-10 Thread Mike Kravetz
- Removed the function parameter. - Added local variable bool count_only = regions_needed != NULL; - Left remaining code as it. I'm OK with the proposed change. Any change to readability is VERY minimal. Reviewed-by: Mike Kravetz -- Mike Kravetz

Re: [PATCH 06/10] mm/hugetlb: remove redundant huge_pte_alloc() in hugetlb_fault()

2020-08-10 Thread Mike Kravetz
https://lore.kernel.org/linux-mm/e670f327-5cf9-1959-96e4-6dc7cc30d...@oracle.com/ -- Mike Kravetz

Re: [PATCH 07/10] mm/hugetlb: a page from buddy is not on any list

2020-08-10 Thread Mike Kravetz
On 8/7/20 2:12 AM, Wei Yang wrote: > The page allocated from buddy is not on any list, so just use list_add() > is enough. > > Signed-off-by: Wei Yang Thanks! Reviewed-by: Mike Kravetz -- Mike Kravetz

<    5   6   7   8   9   10   11   12   13   14   >