Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-31 Thread Vladimir Davydov
On Thu, Jul 30, 2015 at 04:01:22PM +0300, Vladimir Davydov wrote: > On Thu, Jul 30, 2015 at 12:12:12PM +0300, Vladimir Davydov wrote: > > On Wed, Jul 29, 2015 at 02:30:15PM -0700, Andrew Morton wrote: > > > On Wed, 29 Jul 2015 19:29:08 +0300 Vladimir Davydov > > > wrote: > > > > > > > /proc/kpag

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-30 Thread Vladimir Davydov
On Thu, Jul 30, 2015 at 12:12:12PM +0300, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 02:30:15PM -0700, Andrew Morton wrote: > > On Wed, 29 Jul 2015 19:29:08 +0300 Vladimir Davydov > > wrote: > > > > > /proc/kpageidle should probably live somewhere in /sys/kernel/mm, but I > > > added it w

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-30 Thread Vladimir Davydov
On Thu, Jul 30, 2015 at 11:07:09AM +0200, Michal Hocko wrote: > On Wed 29-07-15 19:29:08, Vladimir Davydov wrote: > > On Wed, Jul 29, 2015 at 05:47:18PM +0200, Michal Hocko wrote: > [...] > > > If you use the low limit for isolating an important load then you do not > > > have to care about the oth

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-30 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 02:30:15PM -0700, Andrew Morton wrote: > On Wed, 29 Jul 2015 19:29:08 +0300 Vladimir Davydov > wrote: > > > /proc/kpageidle should probably live somewhere in /sys/kernel/mm, but I > > added it where similar files are located (kpagecount, kpageflags) to > > keep things con

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-30 Thread Michal Hocko
On Wed 29-07-15 19:29:08, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 05:47:18PM +0200, Michal Hocko wrote: [...] > > If you use the low limit for isolating an important load then you do not > > have to care about the others that much. All you care about is to set > > the reasonable protectio

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Andrew Morton
On Wed, 29 Jul 2015 19:29:08 +0300 Vladimir Davydov wrote: > /proc/kpageidle should probably live somewhere in /sys/kernel/mm, but I > added it where similar files are located (kpagecount, kpageflags) to > keep things consistent. I think these files should be moved elsewhere. Consistency is go

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 08:55:01AM -0700, Andres Lagar-Cavilla wrote: > On Wed, Jul 29, 2015 at 8:28 AM, Vladimir Davydov > wrote: > > On Wed, Jul 29, 2015 at 04:26:19PM +0200, Michal Hocko wrote: > >> On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: > >> > On Wed, Jul 29, 2015 at 02:36:30PM +020

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 05:47:18PM +0200, Michal Hocko wrote: > On Wed 29-07-15 18:28:17, Vladimir Davydov wrote: > > On Wed, Jul 29, 2015 at 04:26:19PM +0200, Michal Hocko wrote: > > > On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: > > > > On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michal Hocko
On Wed 29-07-15 18:36:40, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 05:08:55PM +0200, Michal Hocko wrote: > > On Wed 29-07-15 17:45:39, Vladimir Davydov wrote: [...] > > > Page table scan approach has the inherent problem - it ignores unmapped > > > page cache. If a workload does a lot of r

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Andres Lagar-Cavilla
On Wed, Jul 29, 2015 at 8:28 AM, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 04:26:19PM +0200, Michal Hocko wrote: >> On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: >> > On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko wrote: >> > > On Sun 19-07-15 15:31:09, Vladimir Davydov wrote:

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michal Hocko
On Wed 29-07-15 18:28:17, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 04:26:19PM +0200, Michal Hocko wrote: > > On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: > > > On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko wrote: > > > > On Sun 19-07-15 15:31:09, Vladimir Davydov wrote: > > >

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 05:08:55PM +0200, Michal Hocko wrote: > On Wed 29-07-15 17:45:39, Vladimir Davydov wrote: > > On Wed, Jul 29, 2015 at 07:12:13AM -0700, Michel Lespinasse wrote: > > > On Wed, Jul 29, 2015 at 6:59 AM, Vladimir Davydov > > > wrote: > > > >> I guess the primary reason to rely

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 08:08:22AM -0700, Michel Lespinasse wrote: > On Wed, Jul 29, 2015 at 7:45 AM, Vladimir Davydov > wrote: > > Page table scan approach has the inherent problem - it ignores unmapped > > page cache. If a workload does a lot of read/write or map-access-unmap > > operations, we

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 04:26:19PM +0200, Michal Hocko wrote: > On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: > > On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko wrote: > > > On Sun 19-07-15 15:31:09, Vladimir Davydov wrote: > > > [...] > > > > USER API > > > > > > > > The use

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michal Hocko
On Wed 29-07-15 17:45:39, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 07:12:13AM -0700, Michel Lespinasse wrote: > > On Wed, Jul 29, 2015 at 6:59 AM, Vladimir Davydov > > wrote: > > >> I guess the primary reason to rely on the pfn rather than the LRU walk, > > >> which would be more targeted

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 07:12:13AM -0700, Michel Lespinasse wrote: > On Wed, Jul 29, 2015 at 6:59 AM, Vladimir Davydov > wrote: > >> I guess the primary reason to rely on the pfn rather than the LRU walk, > >> which would be more targeted (especially for memcg cases), is that we > >> cannot hold l

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michal Hocko
On Wed 29-07-15 16:59:07, Vladimir Davydov wrote: > On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko wrote: > > On Sun 19-07-15 15:31:09, Vladimir Davydov wrote: > > [...] > > > USER API > > > > > > The user API consists of two new proc files: > > > > I was thinking about this for

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michel Lespinasse
(resending as text, sorry for previous post which didn't make it to the ML) On Wed, Jul 29, 2015 at 7:12 AM, Michel Lespinasse wrote: > > On Wed, Jul 29, 2015 at 6:59 AM, Vladimir Davydov > wrote: > >> I guess the primary reason to rely on the pfn rather than the LRU walk, > >> which would be m

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Vladimir Davydov
On Wed, Jul 29, 2015 at 02:36:30PM +0200, Michal Hocko wrote: > On Sun 19-07-15 15:31:09, Vladimir Davydov wrote: > [...] > > USER API > > > > The user API consists of two new proc files: > > I was thinking about this for a while. I dislike the interface. It is > quite awkward to use -

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-29 Thread Michal Hocko
On Sun 19-07-15 15:31:09, Vladimir Davydov wrote: [...] > USER API > > The user API consists of two new proc files: I was thinking about this for a while. I dislike the interface. It is quite awkward to use - e.g. you have to read the full memory to check a single memcg idleness. This

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-27 Thread Andrew Morton
On Mon, 27 Jul 2015 12:18:57 -0700 Kees Cook wrote: > > Why were these put in /proc anyway? Rather than under /sys/fs/cgroup > > somewhere? Presumably because /proc/kpageidle is useful in non-memcg > > setups. > > Do we need a /proc/vm/ for holding these kinds of things? We're > collecting a l

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-27 Thread Kees Cook
On Tue, Jul 21, 2015 at 4:34 PM, Andrew Morton wrote: > On Sun, 19 Jul 2015 15:31:09 +0300 Vladimir Davydov > wrote: >> To mark a page idle one should set the bit corresponding to the >>page by writing to the file. A value written to the file is OR-ed with the >>current bitmap value. Onl

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-25 Thread Vladimir Davydov
On Wed, Jul 22, 2015 at 07:23:53PM +0300, Vladimir Davydov wrote: > On Tue, Jul 21, 2015 at 04:34:02PM -0700, Andrew Morton wrote: > > On Sun, 19 Jul 2015 15:31:09 +0300 Vladimir Davydov > > wrote: > > > Documentation/vm/pagemap.txt | 22 ++- > > > > I think we'll need quite a lot mor

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-22 Thread Vladimir Davydov
On Tue, Jul 21, 2015 at 04:34:02PM -0700, Andrew Morton wrote: > On Sun, 19 Jul 2015 15:31:09 +0300 Vladimir Davydov > wrote: > > > Hi, > > > > This patch set introduces a new user API for tracking user memory pages > > that have not been used for a given period of time. The purpose of this > >

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-21 Thread Andrew Morton
On Sun, 19 Jul 2015 15:31:09 +0300 Vladimir Davydov wrote: > Hi, > > This patch set introduces a new user API for tracking user memory pages > that have not been used for a given period of time. The purpose of this > is to provide the userspace with the means of tracking a workload's > working

Re: [PATCH -mm v9 0/8] idle memory tracking

2015-07-19 Thread Vladimir Davydov
On Sun, Jul 19, 2015 at 03:31:09PM +0300, Vladimir Davydov wrote: > PERFORMANCE EVALUATION > > SPECjvm2008 (https://www.spec.org/jvm2008/) was used to evaluate the > performance impact introduced by this patch set. Three runs were carried > out: > > - base: kernel without the patch >

[PATCH -mm v9 0/8] idle memory tracking

2015-07-19 Thread Vladimir Davydov
Hi, This patch set introduces a new user API for tracking user memory pages that have not been used for a given period of time. The purpose of this is to provide the userspace with the means of tracking a workload's working set, i.e. the set of pages that are actively used by the workload. Knowing