Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-07-29 Thread Alex Thorlton
We've re-evaluated the need for a patch to support some sort of finer grained control over thp, and, based on some tests performed by our benchmarking team, we're seeing that we'd definitely still like to implement some method to support this. Here's an e-mail from John Baron (jba...@sgi.com),

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-07-29 Thread Alex Thorlton
We've re-evaluated the need for a patch to support some sort of finer grained control over thp, and, based on some tests performed by our benchmarking team, we're seeing that we'd definitely still like to implement some method to support this. Here's an e-mail from John Baron (jba...@sgi.com),

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-20 Thread David Rientjes
On Thu, 20 Jun 2013, Mike Galbraith wrote: > > I'm suspecting that you're referring to enlarged rss because of > > khugepaged's max_ptes_none and because you're abusing the purpose of > > cpusets for containerization. > > Why is containerization an abuse? What's wrong with renting out chunks

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-20 Thread David Rientjes
On Thu, 20 Jun 2013, Mike Galbraith wrote: I'm suspecting that you're referring to enlarged rss because of khugepaged's max_ptes_none and because you're abusing the purpose of cpusets for containerization. Why is containerization an abuse? What's wrong with renting out chunks of a

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Li Zefan
Cc: Tejun, and cgroup ML >> Here are the entries in the cpuset: >> cgroup.event_control mem_exclusivememory_pressure_enabled >> notify_on_release tasks >> cgroup.procs mem_hardwall memory_spread_page release_agent >> cpu_exclusive memory_migrate

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Mike Galbraith
On Wed, 2013-06-19 at 19:43 -0700, David Rientjes wrote: > > I'm suspecting that you're referring to enlarged rss because of > khugepaged's max_ptes_none and because you're abusing the purpose of > cpusets for containerization. Why is containerization an abuse? What's wrong with renting

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread David Rientjes
On Wed, 19 Jun 2013, Robin Holt wrote: > cpusets was not for NUMA. It has no preference for "nodes" or anything like > that. It was for splitting a machine into layered smaller groups. Usually, > we see one cpuset with contains the batch scheduler. The batch scheduler then > creates cpusets

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Robin Holt
On Wed, Jun 19, 2013 at 02:24:07PM -0700, David Rientjes wrote: > On Wed, 19 Jun 2013, Robin Holt wrote: > > > The convenience being that many batch schedulers have added cpuset > > support. They create the cpuset's and configure them as appropriate > > for the job as determined by a mixture of

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread David Rientjes
On Wed, 19 Jun 2013, Robin Holt wrote: > The convenience being that many batch schedulers have added cpuset > support. They create the cpuset's and configure them as appropriate > for the job as determined by a mixture of input from the submitting > user but still under the control of the

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Robin Holt
On Tue, Jun 18, 2013 at 05:01:23PM -0700, David Rientjes wrote: > On Tue, 18 Jun 2013, Alex Thorlton wrote: > > > Thanks for your input, however, I believe the method of using a malloc > > hook falls apart when it comes to static binaries, since we wont' have > > any shared libraries to hook

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Robin Holt
On Tue, Jun 18, 2013 at 05:01:23PM -0700, David Rientjes wrote: On Tue, 18 Jun 2013, Alex Thorlton wrote: Thanks for your input, however, I believe the method of using a malloc hook falls apart when it comes to static binaries, since we wont' have any shared libraries to hook into.

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread David Rientjes
On Wed, 19 Jun 2013, Robin Holt wrote: The convenience being that many batch schedulers have added cpuset support. They create the cpuset's and configure them as appropriate for the job as determined by a mixture of input from the submitting user but still under the control of the

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Robin Holt
On Wed, Jun 19, 2013 at 02:24:07PM -0700, David Rientjes wrote: On Wed, 19 Jun 2013, Robin Holt wrote: The convenience being that many batch schedulers have added cpuset support. They create the cpuset's and configure them as appropriate for the job as determined by a mixture of input

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread David Rientjes
On Wed, 19 Jun 2013, Robin Holt wrote: cpusets was not for NUMA. It has no preference for nodes or anything like that. It was for splitting a machine into layered smaller groups. Usually, we see one cpuset with contains the batch scheduler. The batch scheduler then creates cpusets for

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Mike Galbraith
On Wed, 2013-06-19 at 19:43 -0700, David Rientjes wrote: I'm suspecting that you're referring to enlarged rss because of khugepaged's max_ptes_none and because you're abusing the purpose of cpusets for containerization. Why is containerization an abuse? What's wrong with renting out

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-19 Thread Li Zefan
Cc: Tejun, and cgroup ML Here are the entries in the cpuset: cgroup.event_control mem_exclusivememory_pressure_enabled notify_on_release tasks cgroup.procs mem_hardwall memory_spread_page release_agent cpu_exclusive memory_migrate

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-18 Thread David Rientjes
On Tue, 18 Jun 2013, Alex Thorlton wrote: > Thanks for your input, however, I believe the method of using a malloc > hook falls apart when it comes to static binaries, since we wont' have > any shared libraries to hook into. Although using a malloc hook is a > perfectly suitable solution for

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-18 Thread Alex Thorlton
On Tue, Jun 11, 2013 at 03:20:09PM -0700, David Rientjes wrote: > On Tue, 11 Jun 2013, Alex Thorlton wrote: > > > This patch adds the ability to control THPs on a per cpuset basis. > > Please see > > the additions to Documentation/cgroups/cpusets.txt for more > > information. > > > > What's

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-18 Thread Alex Thorlton
On Tue, Jun 11, 2013 at 03:20:09PM -0700, David Rientjes wrote: On Tue, 11 Jun 2013, Alex Thorlton wrote: This patch adds the ability to control THPs on a per cpuset basis. Please see the additions to Documentation/cgroups/cpusets.txt for more information. What's missing from both

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-18 Thread David Rientjes
On Tue, 18 Jun 2013, Alex Thorlton wrote: Thanks for your input, however, I believe the method of using a malloc hook falls apart when it comes to static binaries, since we wont' have any shared libraries to hook into. Although using a malloc hook is a perfectly suitable solution for most

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-11 Thread David Rientjes
On Tue, 11 Jun 2013, Alex Thorlton wrote: > This patch adds the ability to control THPs on a per cpuset basis. Please see > the additions to Documentation/cgroups/cpusets.txt for more information. > What's missing from both this changelog and the documentation you point to is why this change

[PATCH v2] Make transparent hugepages cpuset aware

2013-06-11 Thread Alex Thorlton
This patch adds the ability to control THPs on a per cpuset basis. Please see the additions to Documentation/cgroups/cpusets.txt for more information. Signed-off-by: Alex Thorlton Reviewed-by: Robin Holt Cc: Li Zefan Cc: Rob Landley Cc: Andrew Morton Cc: Mel Gorman Cc: Rik van Riel Cc:

[PATCH v2] Make transparent hugepages cpuset aware

2013-06-11 Thread Alex Thorlton
This patch adds the ability to control THPs on a per cpuset basis. Please see the additions to Documentation/cgroups/cpusets.txt for more information. Signed-off-by: Alex Thorlton athorl...@sgi.com Reviewed-by: Robin Holt h...@sgi.com Cc: Li Zefan lize...@huawei.com Cc: Rob Landley

Re: [PATCH v2] Make transparent hugepages cpuset aware

2013-06-11 Thread David Rientjes
On Tue, 11 Jun 2013, Alex Thorlton wrote: This patch adds the ability to control THPs on a per cpuset basis. Please see the additions to Documentation/cgroups/cpusets.txt for more information. What's missing from both this changelog and the documentation you point to is why this change is