Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-06 Thread Michal Hocko
On Mon 05-10-20 14:05:17, Zi Yan wrote: > On 5 Oct 2020, at 13:39, David Hildenbrand wrote: > > consideting that 2MB THP have turned out to be quite a pain but > situation has settled over time. Maybe our current code base is prepared > for that much better. > >> > >> I am planning

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-06 Thread David Hildenbrand
On 05.10.20 21:11, Roman Gushchin wrote: > On Mon, Oct 05, 2020 at 08:33:44PM +0200, David Hildenbrand wrote: >> On 05.10.20 20:25, Roman Gushchin wrote: >>> On Mon, Oct 05, 2020 at 07:27:47PM +0200, David Hildenbrand wrote: On 05.10.20 19:16, Roman Gushchin wrote: > On Mon, Oct 05, 2020

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Matthew Wilcox
On Mon, Oct 05, 2020 at 03:12:55PM -0400, Zi Yan wrote: > On 5 Oct 2020, at 11:55, Matthew Wilcox wrote: > > One of the longer-term todo items is to support variable sized THPs for > > anonymous memory, just like I've done for the pagecache. With that in > > place, I think scaling up from PMD

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Zi Yan
On 5 Oct 2020, at 11:55, Matthew Wilcox wrote: > On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: >> On 2 Oct 2020, at 4:30, David Hildenbrand wrote: >>> Yes, I think one important feature would be that we don't end up placing >>> a gigantic page where only a handful of pages are actually

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Roman Gushchin
On Mon, Oct 05, 2020 at 08:33:44PM +0200, David Hildenbrand wrote: > On 05.10.20 20:25, Roman Gushchin wrote: > > On Mon, Oct 05, 2020 at 07:27:47PM +0200, David Hildenbrand wrote: > >> On 05.10.20 19:16, Roman Gushchin wrote: > >>> On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: > On

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread David Hildenbrand
> The real control of hugetlbfs comes from the interfaces provided by > the kernel. If kernel provides similar interfaces to control page sizes > of THPs, it should work the same as hugetlbfs. Mixing page sizes usually > comes from system memory fragmentation and hugetlbfs does not have this >

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread David Hildenbrand
On 05.10.20 20:25, Roman Gushchin wrote: > On Mon, Oct 05, 2020 at 07:27:47PM +0200, David Hildenbrand wrote: >> On 05.10.20 19:16, Roman Gushchin wrote: >>> On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > On 02.10.20 10:10,

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Roman Gushchin
On Mon, Oct 05, 2020 at 07:27:47PM +0200, David Hildenbrand wrote: > On 05.10.20 19:16, Roman Gushchin wrote: > > On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: > >> On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > >> > >>> On 02.10.20 10:10, Michal Hocko wrote: > On Fri 02-10-20

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Zi Yan
On 5 Oct 2020, at 13:39, David Hildenbrand wrote: consideting that 2MB THP have turned out to be quite a pain but situation has settled over time. Maybe our current code base is prepared for that much better. >> >> I am planning to refactor my code further to reduce the amount of

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread David Hildenbrand
>>> consideting that 2MB THP have turned out to be quite a pain but >>> situation has settled over time. Maybe our current code base is prepared >>> for that much better. > > I am planning to refactor my code further to reduce the amount of > the added code, since PUD THP is very similar to PMD

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread David Hildenbrand
>> I think gigantic pages are a sparse resource. Only selected applications >> *really* depend on them and benefit from them. Let these special >> applications handle it explicitly. >> >> Can we have a summary of use cases that would really benefit from this >> change? > > For large machine

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread David Hildenbrand
On 05.10.20 19:16, Roman Gushchin wrote: > On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: >> On 2 Oct 2020, at 4:30, David Hildenbrand wrote: >> >>> On 02.10.20 10:10, Michal Hocko wrote: On Fri 02-10-20 09:50:02, David Hildenbrand wrote: - huge page sizes controllable by

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Roman Gushchin
On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: > On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > > > On 02.10.20 10:10, Michal Hocko wrote: > >> On Fri 02-10-20 09:50:02, David Hildenbrand wrote: > >> - huge page sizes controllable by the userspace? > > > > It might be

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Roman Gushchin
On Mon, Oct 05, 2020 at 04:55:53PM +0100, Matthew Wilcox wrote: > On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: > > On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > > > Yes, I think one important feature would be that we don't end up placing > > > a gigantic page where only a handful

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Matthew Wilcox
On Mon, Oct 05, 2020 at 11:03:56AM -0400, Zi Yan wrote: > On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > > Yes, I think one important feature would be that we don't end up placing > > a gigantic page where only a handful of pages are actually populated > > without green light from the

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Zi Yan
On 2 Oct 2020, at 3:50, David Hildenbrand wrote: - huge page sizes controllable by the userspace? >>> >>> It might be good to allow advanced users to choose the page sizes, so they >>> have better control of their applications. >> >> Could you elaborate more? Those advanced users can use

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-05 Thread Zi Yan
On 2 Oct 2020, at 4:30, David Hildenbrand wrote: > On 02.10.20 10:10, Michal Hocko wrote: >> On Fri 02-10-20 09:50:02, David Hildenbrand wrote: >> - huge page sizes controllable by the userspace? > > It might be good to allow advanced users to choose the page sizes, so they > have

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-02 Thread David Hildenbrand
On 02.10.20 10:10, Michal Hocko wrote: > On Fri 02-10-20 09:50:02, David Hildenbrand wrote: > - huge page sizes controllable by the userspace? It might be good to allow advanced users to choose the page sizes, so they have better control of their applications. >>> >>> Could you

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-02 Thread Michal Hocko
On Fri 02-10-20 09:50:02, David Hildenbrand wrote: > >>> - huge page sizes controllable by the userspace? > >> > >> It might be good to allow advanced users to choose the page sizes, so they > >> have better control of their applications. > > > > Could you elaborate more? Those advanced users can

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-02 Thread David Hildenbrand
>>> - huge page sizes controllable by the userspace? >> >> It might be good to allow advanced users to choose the page sizes, so they >> have better control of their applications. > > Could you elaborate more? Those advanced users can use hugetlb, right? > They get a very good control over page

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-02 Thread Michal Hocko
On Thu 01-10-20 11:14:14, Zi Yan wrote: > On 30 Sep 2020, at 7:55, Michal Hocko wrote: > > > On Mon 28-09-20 13:53:58, Zi Yan wrote: > >> From: Zi Yan > >> > >> Hi all, > >> > >> This patchset adds support for 1GB PUD THP on x86_64. It is on top of > >> v5.9-rc5-mmots-2020-09-18-21-23. It is

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-10-01 Thread Zi Yan
On 30 Sep 2020, at 7:55, Michal Hocko wrote: > On Mon 28-09-20 13:53:58, Zi Yan wrote: >> From: Zi Yan >> >> Hi all, >> >> This patchset adds support for 1GB PUD THP on x86_64. It is on top of >> v5.9-rc5-mmots-2020-09-18-21-23. It is also available at: >>

Re: [RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-09-30 Thread Michal Hocko
On Mon 28-09-20 13:53:58, Zi Yan wrote: > From: Zi Yan > > Hi all, > > This patchset adds support for 1GB PUD THP on x86_64. It is on top of > v5.9-rc5-mmots-2020-09-18-21-23. It is also available at: > https://github.com/x-y-z/linux-1gb-thp/tree/1gb_thp_v5.9-rc5-mmots-2020-09-18-21-23 > >

[RFC PATCH v2 00/30] 1GB PUD THP support on x86_64

2020-09-28 Thread Zi Yan
From: Zi Yan Hi all, This patchset adds support for 1GB PUD THP on x86_64. It is on top of v5.9-rc5-mmots-2020-09-18-21-23. It is also available at: https://github.com/x-y-z/linux-1gb-thp/tree/1gb_thp_v5.9-rc5-mmots-2020-09-18-21-23 Other than PUD THP, we had some discussion on generating THPs