Re: [PATCH] shmem: avoid huge pages for small files

2016-11-10 Thread Kirill A. Shutemov
From f0a582888ac6dcb56c6134611c83edfb091bbcb6 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" <kirill.shute...@linux.intel.com> Date: Mon, 17 Oct 2016 14:44:47 +0300 Subject: [PATCH] shmem: avoid huge pages for small files Huge pages are detrimental for small file: they cause

Re: [PATCH] shmem: avoid huge pages for small files

2016-11-10 Thread Kirill A. Shutemov
From f0a582888ac6dcb56c6134611c83edfb091bbcb6 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" Date: Mon, 17 Oct 2016 14:44:47 +0300 Subject: [PATCH] shmem: avoid huge pages for small files Huge pages are detrimental for small file: they causes noticible overhead on both allocation perfo

Re: [PATCH] shmem: avoid huge pages for small files

2016-11-10 Thread kbuild test robot
Hi Kirill, [auto build test WARNING on linus/master] [also build test WARNING on v4.9-rc4 next-20161110] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH] shmem: avoid huge pages for small files

2016-11-10 Thread kbuild test robot
Hi Kirill, [auto build test WARNING on linus/master] [also build test WARNING on v4.9-rc4 next-20161110] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url:

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-24 Thread Dave Chinner
On Mon, Oct 24, 2016 at 01:34:53PM -0700, Dave Hansen wrote: > On 10/21/2016 03:50 PM, Dave Chinner wrote: > > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > >> On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > >> To me, most of things you're talking about is

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-24 Thread Dave Chinner
On Mon, Oct 24, 2016 at 01:34:53PM -0700, Dave Hansen wrote: > On 10/21/2016 03:50 PM, Dave Chinner wrote: > > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > >> On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > >> To me, most of things you're talking about is

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-24 Thread Dave Hansen
On 10/21/2016 03:50 PM, Dave Chinner wrote: > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: >> On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: >> To me, most of things you're talking about is highly dependent on access >> pattern generated by userspace: >> >>

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-24 Thread Dave Hansen
On 10/21/2016 03:50 PM, Dave Chinner wrote: > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: >> On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: >> To me, most of things you're talking about is highly dependent on access >> pattern generated by userspace: >> >>

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Kirill A. Shutemov
On Sat, Oct 22, 2016 at 09:50:13AM +1100, Dave Chinner wrote: > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > > > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > > > Ugh, no, please don't

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Kirill A. Shutemov
On Sat, Oct 22, 2016 at 09:50:13AM +1100, Dave Chinner wrote: > On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > > > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > > > Ugh, no, please don't

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Dave Chinner
On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > > Ugh, no, please don't use mount options for file specific behaviours > > > > in filesystems

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Dave Chinner
On Fri, Oct 21, 2016 at 06:00:07PM +0300, Kirill A. Shutemov wrote: > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > > Ugh, no, please don't use mount options for file specific behaviours > > > > in filesystems

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Michal Hocko
On Fri 21-10-16 18:00:07, Kirill A. Shutemov wrote: > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: [...] > > None of these aspects can be optimised sanely by a single threshold, > > especially when considering the combination of access patterns vs file > > layout. > > I agree. >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Michal Hocko
On Fri 21-10-16 18:00:07, Kirill A. Shutemov wrote: > On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: [...] > > None of these aspects can be optimised sanely by a single threshold, > > especially when considering the combination of access patterns vs file > > layout. > > I agree. >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Kirill A. Shutemov
On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > Ugh, no, please don't use mount options for file specific behaviours > > > in filesystems like ext4 and XFS. This is exactly the sort of > > > behaviour that should

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-21 Thread Kirill A. Shutemov
On Fri, Oct 21, 2016 at 04:01:18PM +1100, Dave Chinner wrote: > On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > > Ugh, no, please don't use mount options for file specific behaviours > > > in filesystems like ext4 and XFS. This is exactly the sort of > > > behaviour that should

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Dave Chinner
On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > Ugh, no, please don't use mount options for file specific behaviours > > in filesystems like ext4 and XFS. This is exactly the sort of > > behaviour that should either just work automatically (i.e. be > > completely controlled by the

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Dave Chinner
On Thu, Oct 20, 2016 at 07:01:16PM -0700, Andi Kleen wrote: > > Ugh, no, please don't use mount options for file specific behaviours > > in filesystems like ext4 and XFS. This is exactly the sort of > > behaviour that should either just work automatically (i.e. be > > completely controlled by the

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Andi Kleen
> Ugh, no, please don't use mount options for file specific behaviours > in filesystems like ext4 and XFS. This is exactly the sort of > behaviour that should either just work automatically (i.e. be > completely controlled by the filesystem) or only be applied to files Can you explain what you

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Andi Kleen
> Ugh, no, please don't use mount options for file specific behaviours > in filesystems like ext4 and XFS. This is exactly the sort of > behaviour that should either just work automatically (i.e. be > completely controlled by the filesystem) or only be applied to files Can you explain what you

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Dave Chinner
On Thu, Oct 20, 2016 at 01:39:46PM +0300, Kirill A. Shutemov wrote: > On Wed, Oct 19, 2016 at 11:13:54AM -0700, Hugh Dickins wrote: > > On Tue, 18 Oct 2016, Michal Hocko wrote: > > > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Dave Chinner
On Thu, Oct 20, 2016 at 01:39:46PM +0300, Kirill A. Shutemov wrote: > On Wed, Oct 19, 2016 at 11:13:54AM -0700, Hugh Dickins wrote: > > On Tue, 18 Oct 2016, Michal Hocko wrote: > > > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Kirill A. Shutemov
On Wed, Oct 19, 2016 at 11:13:54AM -0700, Hugh Dickins wrote: > On Tue, 18 Oct 2016, Michal Hocko wrote: > > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-20 Thread Kirill A. Shutemov
On Wed, Oct 19, 2016 at 11:13:54AM -0700, Hugh Dickins wrote: > On Tue, 18 Oct 2016, Michal Hocko wrote: > > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-19 Thread Hugh Dickins
On Tue, 18 Oct 2016, Michal Hocko wrote: > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-19 Thread Hugh Dickins
On Tue, 18 Oct 2016, Michal Hocko wrote: > On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Michal Hocko
On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Michal Hocko
On Tue 18-10-16 17:32:07, Kirill A. Shutemov wrote: > On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Kirill A. Shutemov
On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: > [...] > > > > We add two handle to specify minimal file

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Kirill A. Shutemov
On Tue, Oct 18, 2016 at 04:20:07PM +0200, Michal Hocko wrote: > On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: > [...] > > > > We add two handle to specify minimal file

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Michal Hocko
On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: [...] > > > We add two handle to specify minimal file size for huge pages: > > > > > > - mount option 'huge_min_size'; > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-18 Thread Michal Hocko
On Mon 17-10-16 17:55:40, Kirill A. Shutemov wrote: > On Mon, Oct 17, 2016 at 04:12:46PM +0200, Michal Hocko wrote: > > On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: [...] > > > We add two handle to specify minimal file size for huge pages: > > > > > > - mount option 'huge_min_size'; > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
> > > Date: Mon, 17 Oct 2016 14:44:47 +0300 > > Subject: [PATCH] shmem: avoid huge pages for small files > > > > Huge pages are detrimental for small file: they causes noticible > > overhead on both allocation performance and memory footprint. > > >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
16 14:44:47 +0300 > > Subject: [PATCH] shmem: avoid huge pages for small files > > > > Huge pages are detrimental for small file: they causes noticible > > overhead on both allocation performance and memory footprint. > > > > This patch aimed to address this issue

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Michal Hocko
On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: [...] > >From fd0b01b9797ddf2bef308c506c42d3dd50f11793 Mon Sep 17 00:00:00 2001 > From: "Kirill A. Shutemov" <kirill.shute...@linux.intel.com> > Date: Mon, 17 Oct 2016 14:44:47 +0300 > Subject: [PATCH] shmem: a

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Michal Hocko
On Mon 17-10-16 15:30:21, Kirill A. Shutemov wrote: [...] > >From fd0b01b9797ddf2bef308c506c42d3dd50f11793 Mon Sep 17 00:00:00 2001 > From: "Kirill A. Shutemov" > Date: Mon, 17 Oct 2016 14:44:47 +0300 > Subject: [PATCH] shmem: avoid huge pages for small files >

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
stly = HPAGE_PMD_SIZE; >From fd0b01b9797ddf2bef308c506c42d3dd50f11793 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" <kirill.shute...@linux.intel.com> Date: Mon, 17 Oct 2016 14:44:47 +0300 Subject: [PATCH] shmem: avoid huge pages for small files Huge pages are detrimental for small file:

Re: [PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
stly = HPAGE_PMD_SIZE; >From fd0b01b9797ddf2bef308c506c42d3dd50f11793 Mon Sep 17 00:00:00 2001 From: "Kirill A. Shutemov" Date: Mon, 17 Oct 2016 14:44:47 +0300 Subject: [PATCH] shmem: avoid huge pages for small files Huge pages are detrimental for small file: they causes noticible overhead on both

[PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
Huge pages are detrimental for small file: they causes noticible overhead on both allocation performance and memory footprint. This patch aimed to address this issue by avoiding huge pages until file grown to specified size. This would cover most of the cases where huge pages causes regressions

[PATCH] shmem: avoid huge pages for small files

2016-10-17 Thread Kirill A. Shutemov
Huge pages are detrimental for small file: they causes noticible overhead on both allocation performance and memory footprint. This patch aimed to address this issue by avoiding huge pages until file grown to specified size. This would cover most of the cases where huge pages causes regressions