Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2019-01-24 Thread Anthony Yznaga
On 11/14/18 3:15 PM, anthony.yzn...@oracle.com wrote: > > > On 11/10/2018 08:44 AM, Andrea Arcangeli wrote: >> On Sat, Nov 10, 2018 at 01:22:49PM +, Mel Gorman wrote: >>> On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: And if you're in the camp that is concerned

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-20 Thread Andrea Arcangeli
On Tue, Nov 20, 2018 at 12:11:22PM +0300, Kirill A. Shutemov wrote: > On Sat, Nov 10, 2018 at 11:44:12AM -0500, Andrea Arcangeli wrote: > > I would prefer to add intelligence to detect when COWs after fork > > should be done at 2m or 4k granularity (in the latter case by > > splitting the pmd

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-20 Thread Andrea Arcangeli
On Tue, Nov 20, 2018 at 12:11:22PM +0300, Kirill A. Shutemov wrote: > On Sat, Nov 10, 2018 at 11:44:12AM -0500, Andrea Arcangeli wrote: > > I would prefer to add intelligence to detect when COWs after fork > > should be done at 2m or 4k granularity (in the latter case by > > splitting the pmd

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-20 Thread Kirill A. Shutemov
On Sat, Nov 10, 2018 at 11:44:12AM -0500, Andrea Arcangeli wrote: > I would prefer to add intelligence to detect when COWs after fork > should be done at 2m or 4k granularity (in the latter case by > splitting the pmd before the actual COW while leaving the transhuge > pmd intact in the other mm),

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-20 Thread Kirill A. Shutemov
On Sat, Nov 10, 2018 at 11:44:12AM -0500, Andrea Arcangeli wrote: > I would prefer to add intelligence to detect when COWs after fork > should be done at 2m or 4k granularity (in the latter case by > splitting the pmd before the actual COW while leaving the transhuge > pmd intact in the other mm),

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-14 Thread anthony . yznaga
On 11/10/2018 08:44 AM, Andrea Arcangeli wrote: > On Sat, Nov 10, 2018 at 01:22:49PM +, Mel Gorman wrote: >> On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: >>> And if you're in the camp that is concerned about the use of more RAM >>> or/and about the higher latency of COW

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-14 Thread anthony . yznaga
On 11/10/2018 08:44 AM, Andrea Arcangeli wrote: > On Sat, Nov 10, 2018 at 01:22:49PM +, Mel Gorman wrote: >> On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: >>> And if you're in the camp that is concerned about the use of more RAM >>> or/and about the higher latency of COW

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Andrea Arcangeli
On Sat, Nov 10, 2018 at 01:22:49PM +, Mel Gorman wrote: > On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: > > And if you're in the camp that is concerned about the use of more RAM > > or/and about the higher latency of COW faults, I'm afraid the > > intermediate solution will

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Andrea Arcangeli
On Sat, Nov 10, 2018 at 01:22:49PM +, Mel Gorman wrote: > On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: > > And if you're in the camp that is concerned about the use of more RAM > > or/and about the higher latency of COW faults, I'm afraid the > > intermediate solution will

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Mel Gorman
On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > > I haven't yet read the patch in details, but I'm skeptical about the > > approach in general for few reasons: > > > > - PTE page table retracting to replace

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Mel Gorman
On Fri, Nov 09, 2018 at 02:51:50PM -0500, Andrea Arcangeli wrote: > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > > I haven't yet read the patch in details, but I'm skeptical about the > > approach in general for few reasons: > > > > - PTE page table retracting to replace

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Kirill A. Shutemov
On Fri, Nov 09, 2018 at 10:34:07AM -0500, Zi Yan wrote: > On 9 Nov 2018, at 8:11, Mel Gorman wrote: > > > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > >>> The basic idea as outlined by Mel Gorman in [2]

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-10 Thread Kirill A. Shutemov
On Fri, Nov 09, 2018 at 10:34:07AM -0500, Zi Yan wrote: > On 9 Nov 2018, at 8:11, Mel Gorman wrote: > > > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > >>> The basic idea as outlined by Mel Gorman in [2]

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 11:51 AM, Andrea Arcangeli wrote: > Hello, > > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >>> The basic idea as outlined by Mel Gorman in [2] is: >>> >>> 1) On first fault in a sufficiently

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 11:51 AM, Andrea Arcangeli wrote: > Hello, > > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >>> The basic idea as outlined by Mel Gorman in [2] is: >>> >>> 1) On first fault in a sufficiently

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 07:34 AM, Zi Yan wrote: > On 9 Nov 2018, at 8:11, Mel Gorman wrote: > >> On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >>> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: The basic idea as outlined by Mel Gorman in [2] is: 1) On

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 07:34 AM, Zi Yan wrote: > On 9 Nov 2018, at 8:11, Mel Gorman wrote: > >> On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >>> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: The basic idea as outlined by Mel Gorman in [2] is: 1) On

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 04:13 AM, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >> The basic idea as outlined by Mel Gorman in [2] is: >> >> 1) On first fault in a sufficiently sized range, allocate a huge page >>sized and aligned block of base pages.

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 04:13 AM, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >> The basic idea as outlined by Mel Gorman in [2] is: >> >> 1) On first fault in a sufficiently sized range, allocate a huge page >>sized and aligned block of base pages.

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 03:07 AM, Mel Gorman wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >> The basic idea as outlined by Mel Gorman in [2] is: >> >> 1) On first fault in a sufficiently sized range, allocate a huge page >>sized and aligned block of base pages. Map the

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread anthony . yznaga
On 11/09/2018 03:07 AM, Mel Gorman wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >> The basic idea as outlined by Mel Gorman in [2] is: >> >> 1) On first fault in a sufficiently sized range, allocate a huge page >>sized and aligned block of base pages. Map the

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Andrea Arcangeli
Hello, On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > > The basic idea as outlined by Mel Gorman in [2] is: > > > > 1) On first fault in a sufficiently sized range, allocate a huge page > >sized and

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Andrea Arcangeli
Hello, On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > > The basic idea as outlined by Mel Gorman in [2] is: > > > > 1) On first fault in a sufficiently sized range, allocate a huge page > >sized and

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Zi Yan
On 9 Nov 2018, at 8:11, Mel Gorman wrote: > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >>> The basic idea as outlined by Mel Gorman in [2] is: >>> >>> 1) On first fault in a sufficiently sized range,

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Zi Yan
On 9 Nov 2018, at 8:11, Mel Gorman wrote: > On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: >> On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: >>> The basic idea as outlined by Mel Gorman in [2] is: >>> >>> 1) On first fault in a sufficiently sized range,

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Mel Gorman
On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > > The basic idea as outlined by Mel Gorman in [2] is: > > > > 1) On first fault in a sufficiently sized range, allocate a huge page > >sized and aligned

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Mel Gorman
On Fri, Nov 09, 2018 at 03:13:18PM +0300, Kirill A. Shutemov wrote: > On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > > The basic idea as outlined by Mel Gorman in [2] is: > > > > 1) On first fault in a sufficiently sized range, allocate a huge page > >sized and aligned

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Kirill A. Shutemov
On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > The basic idea as outlined by Mel Gorman in [2] is: > > 1) On first fault in a sufficiently sized range, allocate a huge page >sized and aligned block of base pages. Map the base page >corresponding to the fault address

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Kirill A. Shutemov
On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > The basic idea as outlined by Mel Gorman in [2] is: > > 1) On first fault in a sufficiently sized range, allocate a huge page >sized and aligned block of base pages. Map the base page >corresponding to the fault address

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Mel Gorman
On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > The basic idea as outlined by Mel Gorman in [2] is: > > 1) On first fault in a sufficiently sized range, allocate a huge page >sized and aligned block of base pages. Map the base page >corresponding to the fault address

Re: [RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-09 Thread Mel Gorman
On Thu, Nov 08, 2018 at 10:48:58PM -0800, Anthony Yznaga wrote: > The basic idea as outlined by Mel Gorman in [2] is: > > 1) On first fault in a sufficiently sized range, allocate a huge page >sized and aligned block of base pages. Map the base page >corresponding to the fault address

[RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-08 Thread Anthony Yznaga
When the THP enabled policy is "always", or the mode is "madvise" and a region is marked as MADV_HUGEPAGE, a hugepage is allocated on a page fault if the PMD is empty. This yields the best VA translation performance but increases memory consumption if a significant part of the huge page is never

[RFC PATCH] mm: thp: implement THP reservations for anonymous memory

2018-11-08 Thread Anthony Yznaga
When the THP enabled policy is "always", or the mode is "madvise" and a region is marked as MADV_HUGEPAGE, a hugepage is allocated on a page fault if the PMD is empty. This yields the best VA translation performance but increases memory consumption if a significant part of the huge page is never