Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hi Tejun, In my previous messages, I think the message "memsw improves the performance of the job" might have been conveyed. Please ignore that. The message I want to express is the "memsw provides users the ability to consistently limit their job's memory (specifically anon memory) irrespective

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Thu, Dec 21, 2017 at 07:22:20AM -0800, Shakeel Butt wrote: > I am claiming memory allocations under global pressure will be > affected by the performance of the underlying swap device. However > memory allocations under memcg memory pressure, with memsw, will not > be affected

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Thu, Dec 21, 2017 at 5:37 AM, Tejun Heo wrote: > Hello, Shakeel. > > On Wed, Dec 20, 2017 at 05:15:41PM -0800, Shakeel Butt wrote: >> Let's say we have a job that allocates 100 MiB memory and suppose 80 >> MiB is anon and 20 MiB is non-anon (file & kmem). >> >> [With memsw]

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Wed, Dec 20, 2017 at 05:15:41PM -0800, Shakeel Butt wrote: > Let's say we have a job that allocates 100 MiB memory and suppose 80 > MiB is anon and 20 MiB is non-anon (file & kmem). > > [With memsw] Scheduler sets the memsw limit of the job to 100 MiB and > memory to max. Now

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Wed, Dec 20, 2017 at 3:36 PM, Tejun Heo wrote: > Hello, Shakeel. > > On Wed, Dec 20, 2017 at 12:15:46PM -0800, Shakeel Butt wrote: >> > I don't understand how this invariant is useful across different >> > backing swap devices and availability. e.g. Our OOM decisions are >> >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Wed, Dec 20, 2017 at 12:15:46PM -0800, Shakeel Butt wrote: > > I don't understand how this invariant is useful across different > > backing swap devices and availability. e.g. Our OOM decisions are > > currently not great in that the kernel can easily thrash for a very > >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Wed, Dec 20, 2017 at 11:37 AM, Tejun Heo wrote: > Hello, Shakeel. > > On Tue, Dec 19, 2017 at 02:39:19PM -0800, Shakeel Butt wrote: >> Suppose a user wants to run multiple instances of a specific job on >> different datacenters and s/he has budget of 100MiB for each instance.

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, Shakeel. On Tue, Dec 19, 2017 at 02:39:19PM -0800, Shakeel Butt wrote: > Suppose a user wants to run multiple instances of a specific job on > different datacenters and s/he has budget of 100MiB for each instance. > The instances are schduled on the requested datacenters and the >

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 1:41 PM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 10:25:12AM -0800, Shakeel Butt wrote: >> Making the runtime environment, an invariant is very critical to make >> the management of a job easier whose instances run on different >> clusters

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 10:25:12AM -0800, Shakeel Butt wrote: > Making the runtime environment, an invariant is very critical to make > the management of a job easier whose instances run on different > clusters across the world. Some clusters might have different type of > swaps installed

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 9:33 AM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 09:23:29AM -0800, Shakeel Butt wrote: >> To provide consistent memory usage history using the current >> cgroup-v2's 'swap' interface, an additional metric expressing the >> intersection of

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 09:23:29AM -0800, Shakeel Butt wrote: > To provide consistent memory usage history using the current > cgroup-v2's 'swap' interface, an additional metric expressing the > intersection of memory and swap has to be exposed. Basically memsw is > the union of memory and

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 7:24 AM, Tejun Heo wrote: > Hello, > > On Tue, Dec 19, 2017 at 07:12:19AM -0800, Shakeel Butt wrote: >> Yes, there are pros & cons, therefore we should give users the option >> to select the API that is better suited for their use-cases and > > Heh, that's

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

Hello, On Tue, Dec 19, 2017 at 07:12:19AM -0800, Shakeel Butt wrote: > Yes, there are pros & cons, therefore we should give users the option > to select the API that is better suited for their use-cases and Heh, that's not how API decisions should be made. The long term outcome would be really

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Tue, Dec 19, 2017 at 4:49 AM, Michal Hocko wrote: > On Mon 18-12-17 16:01:31, Shakeel Butt wrote: >> The memory controller in cgroup v1 provides the memory+swap (memsw) >> interface to account to the combined usage of memory and swap of the >> jobs. The memsw interface

Re: [RFC PATCH] mm: memcontrol: memory+swap accounting for cgroup-v2

On Mon 18-12-17 16:01:31, Shakeel Butt wrote: > The memory controller in cgroup v1 provides the memory+swap (memsw) > interface to account to the combined usage of memory and swap of the > jobs. The memsw interface allows the users to limit or view the > consistent memory usage of their jobs