Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hi Tejun, In my previous messages, I think the message "memsw improves the performance of the job" might have been conveyed. Please ignore that. The message I want to express is the "memsw provides users the ability to consistently limit their job's memory (specifically anon memory) irrespective

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Thu, Dec 21, 2017 at 07:22:20AM -0800, Shakeel Butt wrote: > I am claiming memory allocations under global pressure will be > affected by the performance of the underlying swap device. However > memory allocations under memcg memory pressure, with memsw, will not > be affected

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Thu, Dec 21, 2017 at 5:37 AM, Tejun Heo wrote: > Hello, Shakeel. > > On Wed, Dec 20, 2017 at 05:15:41PM -0800, Shakeel Butt wrote: >> Let's say we have a job that allocates 100 MiB memory and suppose 80 >> MiB is anon and 20 MiB is non-anon (file & kmem). >> >> [With memsw]

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Wed, Dec 20, 2017 at 05:15:41PM -0800, Shakeel Butt wrote: > Let's say we have a job that allocates 100 MiB memory and suppose 80 > MiB is anon and 20 MiB is non-anon (file & kmem). > > [With memsw] Scheduler sets the memsw limit of the job to 100 MiB and > memory to max. Now

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Wed, Dec 20, 2017 at 3:36 PM, Tejun Heo wrote: > Hello, Shakeel. > > On Wed, Dec 20, 2017 at 12:15:46PM -0800, Shakeel Butt wrote: >> > I don't understand how this invariant is useful across different >> > backing swap devices and availability. e.g. Our OOM decisions are >> >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Wed, Dec 20, 2017 at 12:15:46PM -0800, Shakeel Butt wrote: > > I don't understand how this invariant is useful across different > > backing swap devices and availability. e.g. Our OOM decisions are > > currently not great in that the kernel can easily thrash for a very > >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Wed, Dec 20, 2017 at 11:37 AM, Tejun Heo wrote: > Hello, Shakeel. > > On Tue, Dec 19, 2017 at 02:39:19PM -0800, Shakeel Butt wrote: >> Suppose a user wants to run multiple instances of a specific job on >> different datacenters and s/he has budget of 100MiB for each instance.

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Tue, Dec 19, 2017 at 02:39:19PM -0800, Shakeel Butt wrote: > Suppose a user wants to run multiple instances of a specific job on > different datacenters and s/he has budget of 100MiB for each instance. > The instances are schduled on the requested datacenters and the >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 1:41 PM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 10:25:12AM -0800, Shakeel Butt wrote: >> Making the runtime environment, an invariant is very critical to make >> the management of a job easier whose instances run on different >> clusters

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 10:25:12AM -0800, Shakeel Butt wrote: > Making the runtime environment, an invariant is very critical to make > the management of a job easier whose instances run on different > clusters across the world. Some clusters might have different type of > swaps installed

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 9:33 AM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 09:23:29AM -0800, Shakeel Butt wrote: >> To provide consistent memory usage history using the current >> cgroup-v2's 'swap' interface, an additional metric expressing the >> intersection of

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 09:23:29AM -0800, Shakeel Butt wrote: > To provide consistent memory usage history using the current > cgroup-v2's 'swap' interface, an additional metric expressing the > intersection of memory and swap has to be exposed. Basically memsw is > the union of memory and

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 7:24 AM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 07:12:19AM -0800, Shakeel Butt wrote: >> Yes, there are pros & cons, therefore we should give users the option >> to select the API that is better suited for their use-cases and > > Heh, that's

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 07:12:19AM -0800, Shakeel Butt wrote: > Yes, there are pros & cons, therefore we should give users the option > to select the API that is better suited for their use-cases and Heh, that's not how API decisions should be made. The long term outcome would be really

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 4:49 AM, Michal Hocko wrote: > On Mon 18-12-17 16:01:31, Shakeel Butt wrote: >> The memory controller in cgroup v1 provides the memory+swap (memsw) >> interface to account to the combined usage of memory and swap of the >> jobs. The memsw interface

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Mon 18-12-17 16:01:31, Shakeel Butt wrote: > The memory controller in cgroup v1 provides the memory+swap (memsw) > interface to account to the combined usage of memory and swap of the > jobs. The memsw interface allows the users to limit or view the > consistent memory usage of their jobs

[RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

The memory controller in cgroup v1 provides the memory+swap (memsw) interface to account to the combined usage of memory and swap of the jobs. The memsw interface allows the users to limit or view the consistent memory usage of their jobs irrespectibe of the presense of swap on the system