Re: [RFC][PATCH 2/7] RSS controller core

2007-03-20 Thread Mel Gorman
On (14/03/07 13:42), Dave Hansen didst pronounce: > On Wed, 2007-03-14 at 15:38 +, Mel Gorman wrote: > > On (13/03/07 10:05), Dave Hansen didst pronounce: > > > How do we determine what is shared, and goes into the shared zones? > > > > Assuming we had a means of creating a zone that was

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-20 Thread Mel Gorman
On (14/03/07 13:42), Dave Hansen didst pronounce: On Wed, 2007-03-14 at 15:38 +, Mel Gorman wrote: On (13/03/07 10:05), Dave Hansen didst pronounce: How do we determine what is shared, and goes into the shared zones? Assuming we had a means of creating a zone that was assigned to a

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-19 Thread Eric W. Biederman
"Paul Menage" <[EMAIL PROTECTED]> writes: > On 3/13/07, Dave Hansen <[EMAIL PROTECTED]> wrote: >> How do we determine what is shared, and goes into the shared zones? >> Once we've allocated a page, it's too late because we already picked. >> Do we just assume all page cache is shared? Base it on

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-19 Thread Herbert Poetzl
On Sun, Mar 18, 2007 at 11:42:15AM -0600, Eric W. Biederman wrote: > Dave Hansen <[EMAIL PROTECTED]> writes: > > > On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: > >> Dave Hansen <[EMAIL PROTECTED]> writes: > > > >> - Why do limits have to apply to the unmapped page cache? > > > > To

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-19 Thread Herbert Poetzl
On Sun, Mar 18, 2007 at 11:42:15AM -0600, Eric W. Biederman wrote: Dave Hansen [EMAIL PROTECTED] writes: On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: Dave Hansen [EMAIL PROTECTED] writes: - Why do limits have to apply to the unmapped page cache? To me, it is just

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-19 Thread Eric W. Biederman
Paul Menage [EMAIL PROTECTED] writes: On 3/13/07, Dave Hansen [EMAIL PROTECTED] wrote: How do we determine what is shared, and goes into the shared zones? Once we've allocated a page, it's too late because we already picked. Do we just assume all page cache is shared? Base it on filesystem,

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Paul Menage
On 3/13/07, Dave Hansen <[EMAIL PROTECTED]> wrote: How do we determine what is shared, and goes into the shared zones? Once we've allocated a page, it's too late because we already picked. Do we just assume all page cache is shared? Base it on filesystem, mount, ...? Mount seems the most

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Eric W. Biederman
Dave Hansen <[EMAIL PROTECTED]> writes: > On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: >> Dave Hansen <[EMAIL PROTECTED]> writes: > >> - Why do limits have to apply to the unmapped page cache? > > To me, it is just because it consumes memory. Unmapped cache is, of > couse, much

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Eric W. Biederman
Dave Hansen <[EMAIL PROTECTED]> writes: > On Mon, 2007-03-12 at 23:41 +0100, Herbert Poetzl wrote: >> >> let me give a real world example here: >> >> - typical guest with 600MB disk space >> - about 100MB guest specific data (not shared) >> - assumed that 80% of the libs/tools are used > > I

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Eric W. Biederman
Dave Hansen [EMAIL PROTECTED] writes: On Mon, 2007-03-12 at 23:41 +0100, Herbert Poetzl wrote: let me give a real world example here: - typical guest with 600MB disk space - about 100MB guest specific data (not shared) - assumed that 80% of the libs/tools are used I get the general

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Eric W. Biederman
Dave Hansen [EMAIL PROTECTED] writes: On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: Dave Hansen [EMAIL PROTECTED] writes: - Why do limits have to apply to the unmapped page cache? To me, it is just because it consumes memory. Unmapped cache is, of couse, much more easily

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-18 Thread Paul Menage
On 3/13/07, Dave Hansen [EMAIL PROTECTED] wrote: How do we determine what is shared, and goes into the shared zones? Once we've allocated a page, it's too late because we already picked. Do we just assume all page cache is shared? Base it on filesystem, mount, ...? Mount seems the most logical

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Dave Hansen
On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: > Dave Hansen <[EMAIL PROTECTED]> writes: > > http://linux-mm.org/SoftwareZones > Looking at your page, and I'm too lazy to figure out how to update it > I have a couple of comments. You just need to create an account by clicking the

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Eric W. Biederman
Dave Hansen <[EMAIL PROTECTED]> writes: > On Thu, 2007-03-15 at 18:55 -0600, Eric W. Biederman wrote: >> To create a DOS attack. >> >> - Allocate some memory you know your victim will want in the future, >> (shared libraries and the like). >> - Wait until your victim is using the memory you

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Dave Hansen
On Thu, 2007-03-15 at 18:55 -0600, Eric W. Biederman wrote: > To create a DOS attack. > > - Allocate some memory you know your victim will want in the future, > (shared libraries and the like). > - Wait until your victim is using the memory you allocated. > - Terminate your memory resource

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Dave Hansen
On Thu, 2007-03-15 at 18:55 -0600, Eric W. Biederman wrote: To create a DOS attack. - Allocate some memory you know your victim will want in the future, (shared libraries and the like). - Wait until your victim is using the memory you allocated. - Terminate your memory resource group. -

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Eric W. Biederman
Dave Hansen [EMAIL PROTECTED] writes: On Thu, 2007-03-15 at 18:55 -0600, Eric W. Biederman wrote: To create a DOS attack. - Allocate some memory you know your victim will want in the future, (shared libraries and the like). - Wait until your victim is using the memory you allocated. -

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-16 Thread Dave Hansen
On Fri, 2007-03-16 at 12:54 -0600, Eric W. Biederman wrote: Dave Hansen [EMAIL PROTECTED] writes: http://linux-mm.org/SoftwareZones Looking at your page, and I'm too lazy to figure out how to update it I have a couple of comments. You just need to create an account by clicking the Login

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-15 Thread Eric W. Biederman
Alan Cox <[EMAIL PROTECTED]> writes: >> stuff is happening by comparing page->count and page->_mapcount, but it >> certainly wouldn't be conclusive. But, does this kind of nonsense even >> happen in practice? > > "Is it useful for me as a bad guy to make it happen ?" To create a DOS attack.

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-15 Thread Eric W. Biederman
Alan Cox [EMAIL PROTECTED] writes: stuff is happening by comparing page-count and page-_mapcount, but it certainly wouldn't be conclusive. But, does this kind of nonsense even happen in practice? Is it useful for me as a bad guy to make it happen ? To create a DOS attack. - Allocate

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Dave Hansen
On Wed, 2007-03-14 at 15:38 +, Mel Gorman wrote: > On (13/03/07 10:05), Dave Hansen didst pronounce: > > How do we determine what is shared, and goes into the shared zones? > > Assuming we had a means of creating a zone that was assigned to a container, > a second zone for shared data between

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Mel Gorman
On (13/03/07 10:26), Dave Hansen didst pronounce: > On Mon, 2007-03-12 at 22:04 -0800, Andrew Morton wrote: > > So these mmapped pages will contiue to be shared across all guests. The > > problem boils down to "which guest(s) get charged for each shared page". > > > > A simple and obvious and

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Mel Gorman
On (13/03/07 10:05), Dave Hansen didst pronounce: > On Tue, 2007-03-13 at 03:48 -0800, Andrew Morton wrote: > > If we use a physical zone-based containment scheme: fake-numa, > > variable-sized zones, etc then it all becomes moot. You set up a container > > which has 1.5GB of physial memory then

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Mel Gorman
On (13/03/07 10:05), Dave Hansen didst pronounce: On Tue, 2007-03-13 at 03:48 -0800, Andrew Morton wrote: If we use a physical zone-based containment scheme: fake-numa, variable-sized zones, etc then it all becomes moot. You set up a container which has 1.5GB of physial memory then toss

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Mel Gorman
On (13/03/07 10:26), Dave Hansen didst pronounce: On Mon, 2007-03-12 at 22:04 -0800, Andrew Morton wrote: So these mmapped pages will contiue to be shared across all guests. The problem boils down to which guest(s) get charged for each shared page. A simple and obvious and

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-14 Thread Dave Hansen
On Wed, 2007-03-14 at 15:38 +, Mel Gorman wrote: On (13/03/07 10:05), Dave Hansen didst pronounce: How do we determine what is shared, and goes into the shared zones? Assuming we had a means of creating a zone that was assigned to a container, a second zone for shared data between a set

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Tue, 2007-03-13 at 19:09 +, Alan Cox wrote: > > stuff is happening by comparing page->count and page->_mapcount, but it > > certainly wouldn't be conclusive. But, does this kind of nonsense even > > happen in practice? > > "Is it useful for me as a bad guy to make it happen ?" A very

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Alan Cox
> stuff is happening by comparing page->count and page->_mapcount, but it > certainly wouldn't be conclusive. But, does this kind of nonsense even > happen in practice? "Is it useful for me as a bad guy to make it happen ?" Alan - To unsubscribe from this list: send the line "unsubscribe

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Mon, 2007-03-12 at 22:04 -0800, Andrew Morton wrote: > So these mmapped pages will contiue to be shared across all guests. The > problem boils down to "which guest(s) get charged for each shared page". > > A simple and obvious and easy-to-implement answer is "the guest which paged > it in".

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Tue, 2007-03-13 at 03:48 -0800, Andrew Morton wrote: > If we use a physical zone-based containment scheme: fake-numa, > variable-sized zones, etc then it all becomes moot. You set up a container > which has 1.5GB of physial memory then toss processes into it. As that > process set increases

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
Srivatsa Vaddagiri <[EMAIL PROTECTED]>, > > Balbir Singh <[EMAIL PROTECTED]> > > Cc: [EMAIL PROTECTED], > > Linux Kernel Mailing List > > Date: Tue, 06 Mar 2007 17:55:29 +0300 > > ---

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Herbert, >>Just curious why current vserver code kills arbitrary >>task in container then? > > > because it obviously lacks the finess of OpenVZ code :) > > seriously, handling the OOM kills inside a container > has never been a real world issue, as once you are > really out of memory (and OOM

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Pavel Emelianov
Herbert Poetzl wrote: > On Tue, Mar 13, 2007 at 10:17:54AM +0300, Pavel Emelianov wrote: >> Herbert Poetzl wrote: >>> On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: >>> Maybe you have some ideas how we can decide on this? >> We need to work out what the requirements are

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Eric, >>>And misses every resource sharing opportunity in sight. >> >>that was my point too. >> >> >>>Except for >>>filtering the which pages are eligible for reclaim an RSS limit should >>>not need to change the existing reclaim logic, and with things like the >>>memory zones we have had that

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 06:10:55PM +0300, Kirill Korotaev wrote: > >>So what to do when virtual physical limit is hit? > >>OOM-kill current task? > > > > > > when the RSS limit is hit, but there _are_ enough > > pages left on the physical system, there is no > > good reason to swap out the page

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 10:17:54AM +0300, Pavel Emelianov wrote: > Herbert Poetzl wrote: > > On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: > > Maybe you have some ideas how we can decide on this? > We need to work out what the requirements are before we can >

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 03:48:34AM -0800, Andrew Morton wrote: > > On Tue, 13 Mar 2007 13:19:53 +0300 Kirill Korotaev <[EMAIL PROTECTED]> > > wrote: > > Andrew Morton wrote: > > - shared mappings of 'shared' files (binaries > > and libraries) to allow for reduced memory > >

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
>>So what to do when virtual physical limit is hit? >>OOM-kill current task? > > > when the RSS limit is hit, but there _are_ enough > pages left on the physical system, there is no > good reason to swap out the page at all > > - there is no benefit in doing so (performance >wise, that is)

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Andrew Morton
> On Tue, 13 Mar 2007 13:19:53 +0300 Kirill Korotaev <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > - shared mappings of 'shared' files (binaries > and libraries) to allow for reduced memory > footprint when N identical guests are running > >>> > >>>So, it sounds like this

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Andrew Morton wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical guests are running >>> >>>So, it sounds like this can be phrased as a requirement like: >>> >>> "Guests must be able to share pages." >>>

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Eric W. Biederman
Kirill Korotaev <[EMAIL PROTECTED]> writes: > Eric, > >> And misses every resource sharing opportunity in sight. > > that was my point too. > >> Except for >> filtering the which pages are eligible for reclaim an RSS limit should >> not need to change the existing reclaim logic, and with things

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Pavel Emelianov
Herbert Poetzl wrote: > On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: > Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can settle on an implementation. >>> Linux-VServer (and probably OpenVZ): >>> >>>

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Pavel Emelianov
Herbert Poetzl wrote: On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can settle on an implementation. Linux-VServer (and probably OpenVZ): - shared mappings of

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Eric W. Biederman
Kirill Korotaev [EMAIL PROTECTED] writes: Eric, And misses every resource sharing opportunity in sight. that was my point too. Except for filtering the which pages are eligible for reclaim an RSS limit should not need to change the existing reclaim logic, and with things like the memory

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Andrew Morton wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical guests are running So, it sounds like this can be phrased as a requirement like: Guests must be able to share pages. Can you give us an idea why

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Andrew Morton
On Tue, 13 Mar 2007 13:19:53 +0300 Kirill Korotaev [EMAIL PROTECTED] wrote: Andrew Morton wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical guests are running So, it sounds like this can be phrased as a

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
So what to do when virtual physical limit is hit? OOM-kill current task? when the RSS limit is hit, but there _are_ enough pages left on the physical system, there is no good reason to swap out the page at all - there is no benefit in doing so (performance wise, that is) - it

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 10:17:54AM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can settle on an

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 06:10:55PM +0300, Kirill Korotaev wrote: So what to do when virtual physical limit is hit? OOM-kill current task? when the RSS limit is hit, but there _are_ enough pages left on the physical system, there is no good reason to swap out the page at all -

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Pavel Emelianov
Herbert Poetzl wrote: On Tue, Mar 13, 2007 at 10:17:54AM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Herbert, Just curious why current vserver code kills arbitrary task in container then? because it obviously lacks the finess of OpenVZ code :) seriously, handling the OOM kills inside a container has never been a real world issue, as once you are really out of memory (and OOM starts

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
], Linux Kernel Mailing List linux-kernel@vger.kernel.org Date: Tue, 06 Mar 2007 17:55:29 +0300 Subject: Re: [RFC][PATCH 2/7] RSS controller core From: Andrew Morton [EMAIL PROTECTED] To: Pavel Emelianov [EMAIL

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Tue, 2007-03-13 at 03:48 -0800, Andrew Morton wrote: If we use a physical zone-based containment scheme: fake-numa, variable-sized zones, etc then it all becomes moot. You set up a container which has 1.5GB of physial memory then toss processes into it. As that process set increases in

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Mon, 2007-03-12 at 22:04 -0800, Andrew Morton wrote: So these mmapped pages will contiue to be shared across all guests. The problem boils down to which guest(s) get charged for each shared page. A simple and obvious and easy-to-implement answer is the guest which paged it in. I think

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Alan Cox
stuff is happening by comparing page-count and page-_mapcount, but it certainly wouldn't be conclusive. But, does this kind of nonsense even happen in practice? Is it useful for me as a bad guy to make it happen ? Alan - To unsubscribe from this list: send the line unsubscribe linux-kernel

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Dave Hansen
On Tue, 2007-03-13 at 19:09 +, Alan Cox wrote: stuff is happening by comparing page-count and page-_mapcount, but it certainly wouldn't be conclusive. But, does this kind of nonsense even happen in practice? Is it useful for me as a bad guy to make it happen ? A very fine

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Herbert Poetzl
On Tue, Mar 13, 2007 at 03:48:34AM -0800, Andrew Morton wrote: On Tue, 13 Mar 2007 13:19:53 +0300 Kirill Korotaev [EMAIL PROTECTED] wrote: Andrew Morton wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-13 Thread Kirill Korotaev
Eric, And misses every resource sharing opportunity in sight. that was my point too. Except for filtering the which pages are eligible for reclaim an RSS limit should not need to change the existing reclaim logic, and with things like the memory zones we have had that kind of restriction in

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Andrew Morton
> On Mon, 12 Mar 2007 23:41:29 +0100 Herbert Poetzl <[EMAIL PROTECTED]> wrote: > On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: > > How about we drill down on these a bit more. > > > > On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: > > > - shared mappings of 'shared'

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Srivatsa Vaddagiri
On Tue, Mar 13, 2007 at 07:27:06AM +0530, Balbir Singh wrote: > I am not sure what went wrong. Could you please check your mail > client, cause it seemed to even change email address to smtp.osdl.org > which bounced back when I wrote to you earlier. I have a problem doing a group-reply in mutt to

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Balbir Singh
ue, 06 Mar 2007 17:55:29 +0300 ---------------- Subject: Re: [RFC][PATCH 2/7] RSS controller core From: Andrew Morton <[EMAIL PROTECTED]> To: Pavel Emelianov <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECT

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
To: Andrew Morton <[EMAIL PROTECTED]>, Paul Menage <[EMAIL PROTECTED]>, Srivatsa Vaddagiri <[EMAIL PROTECTED]>, Balbir Singh <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED], Linux Kernel Mailing List Date: Tue, 06 Mar 2007 17:55:29 +0300 --

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Dave Hansen
On Mon, 2007-03-12 at 23:41 +0100, Herbert Poetzl wrote: > On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: > > How about we drill down on these a bit more. > > > > On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: > > > - shared mappings of 'shared' files (binaries > > >

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: > How about we drill down on these a bit more. > > On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: > > - shared mappings of 'shared' files (binaries > >and libraries) to allow for reduced memory > >footprint when N

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: > >>> Maybe you have some ideas how we can decide on this? > >> We need to work out what the requirements are before we can > >> settle on an implementation. > > > > Linux-VServer (and probably OpenVZ): > > > > - shared mappings

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Dave Hansen
How about we drill down on these a bit more. On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: > - shared mappings of 'shared' files (binaries >and libraries) to allow for reduced memory >footprint when N identical guests are running So, it sounds like this can be phrased as a

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Balbir Singh
doesn't look so good for me, mainly becaus of the additional per page data and per page processing on 4GB memory, with 100 guests, 50% shared for each guest, this basically means ~1mio pages, 500k shared and 1500k x sizeof(page_container) entries, which roughly boils down to ~25MB of wasted

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Kirill Korotaev
Eric, > And misses every resource sharing opportunity in sight. that was my point too. > Except for > filtering the which pages are eligible for reclaim an RSS limit should > not need to change the existing reclaim logic, and with things like the > memory zones we have had that kind of

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Pavel Emelianov
>>> Maybe you have some ideas how we can decide on this? >> We need to work out what the requirements are before we can >> settle on an implementation. > > Linux-VServer (and probably OpenVZ): > > - shared mappings of 'shared' files (binaries >and libraries) to allow for reduced memory >

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Pavel Emelianov
[snip] >> We need to decide whether we want to do per-container memory >> limitation via these data structures, or whether we do it via >> a physical scan of some software zone, possibly based on Mel's >> patches. > why not do simple page accounting (as done currently > in

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Pavel Emelianov
[snip] We need to decide whether we want to do per-container memory limitation via these data structures, or whether we do it via a physical scan of some software zone, possibly based on Mel's patches. why not do simple page accounting (as done currently in Linux) and use that for the

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Pavel Emelianov
Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can settle on an implementation. Linux-VServer (and probably OpenVZ): - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Balbir Singh
doesn't look so good for me, mainly becaus of the additional per page data and per page processing on 4GB memory, with 100 guests, 50% shared for each guest, this basically means ~1mio pages, 500k shared and 1500k x sizeof(page_container) entries, which roughly boils down to ~25MB of wasted

Re: [Devel] Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Kirill Korotaev
Eric, And misses every resource sharing opportunity in sight. that was my point too. Except for filtering the which pages are eligible for reclaim an RSS limit should not need to change the existing reclaim logic, and with things like the memory zones we have had that kind of restriction

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Dave Hansen
How about we drill down on these a bit more. On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical guests are running So, it sounds like this can be phrased as a

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
On Mon, Mar 12, 2007 at 12:02:01PM +0300, Pavel Emelianov wrote: Maybe you have some ideas how we can decide on this? We need to work out what the requirements are before we can settle on an implementation. Linux-VServer (and probably OpenVZ): - shared mappings of 'shared' files

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: How about we drill down on these a bit more. On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: - shared mappings of 'shared' files (binaries and libraries) to allow for reduced memory footprint when N identical

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Dave Hansen
On Mon, 2007-03-12 at 23:41 +0100, Herbert Poetzl wrote: On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: How about we drill down on these a bit more. On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: - shared mappings of 'shared' files (binaries and

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Herbert Poetzl
@vger.kernel.org Date: Tue, 06 Mar 2007 17:55:29 +0300 Subject: Re: [RFC][PATCH 2/7] RSS controller core From: Andrew Morton [EMAIL PROTECTED] To: Pavel Emelianov [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Balbir Singh
2007 17:55:29 +0300 Subject: Re: [RFC][PATCH 2/7] RSS controller core From: Andrew Morton [EMAIL PROTECTED] To: Pavel Emelianov [EMAIL PROTECTED] Cc: [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED], Paul Menage [EMAIL

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Srivatsa Vaddagiri
On Tue, Mar 13, 2007 at 07:27:06AM +0530, Balbir Singh wrote: I am not sure what went wrong. Could you please check your mail client, cause it seemed to even change email address to smtp.osdl.org which bounced back when I wrote to you earlier. I have a problem doing a group-reply in mutt to

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-12 Thread Andrew Morton
On Mon, 12 Mar 2007 23:41:29 +0100 Herbert Poetzl [EMAIL PROTECTED] wrote: On Mon, Mar 12, 2007 at 11:42:59AM -0700, Dave Hansen wrote: How about we drill down on these a bit more. On Mon, 2007-03-12 at 02:00 +0100, Herbert Poetzl wrote: - shared mappings of 'shared' files (binaries

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 04:51:11AM -0800, Andrew Morton wrote: > > On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev <[EMAIL PROTECTED]> > > wrote: > > Andrew Morton wrote: > > > On Tue, 06 Mar 2007 17:55:29 +0300 > > > Pavel Emelianov <[EMAIL PROTECTED]> wrote: > > > > > > > > >>+struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 06:04:28PM +0300, Pavel Emelianov wrote: > Herbert Poetzl wrote: > > On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: > >> Herbert Poetzl wrote: > >>> On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: > On Tue, 06 Mar 2007 17:55:29 +0300 >

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Eric W. Biederman
Andrew Morton <[EMAIL PROTECTED]> writes: > Yep. Straightforward machine partitioning. An attractive thing is that it > 100% reuses existing page reclaim, unaltered. And misses every resource sharing opportunity in sight. Except for filtering the which pages are eligible for reclaim an RSS

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Balbir Singh
On 3/11/07, Andrew Morton <[EMAIL PROTECTED]> wrote: > On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > > On Tue, 06 Mar 2007 17:55:29 +0300 > > Pavel Emelianov <[EMAIL PROTECTED]> wrote: > > > > > >>+struct rss_container { > >>+ struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Pavel Emelianov
Herbert Poetzl wrote: > On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: >> Herbert Poetzl wrote: >>> On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov <[EMAIL PROTECTED]> wrote: > +struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: > Herbert Poetzl wrote: >> On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: >>> On Tue, 06 Mar 2007 17:55:29 +0300 >>> Pavel Emelianov <[EMAIL PROTECTED]> wrote: >>> +struct rss_container { + struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Andrew Morton
> On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev <[EMAIL PROTECTED]> wrote: > Andrew Morton wrote: > > On Tue, 06 Mar 2007 17:55:29 +0300 > > Pavel Emelianov <[EMAIL PROTECTED]> wrote: > > > > > >>+struct rss_container { > >>+ struct res_counter res; > >>+ struct list_head page_list; >

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Kirill Korotaev
Andrew Morton wrote: > On Tue, 06 Mar 2007 17:55:29 +0300 > Pavel Emelianov <[EMAIL PROTECTED]> wrote: > > >>+struct rss_container { >>+ struct res_counter res; >>+ struct list_head page_list; >>+ struct container_subsys_state css; >>+}; >>+ >>+struct page_container { >>+ struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Pavel Emelianov
Herbert Poetzl wrote: > On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: >> On Tue, 06 Mar 2007 17:55:29 +0300 >> Pavel Emelianov <[EMAIL PROTECTED]> wrote: >> >>> +struct rss_container { >>> + struct res_counter res; >>> + struct list_head page_list; >>> + struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Pavel Emelianov
Herbert Poetzl wrote: On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct res_counter res; + struct list_head page_list; + struct container_subsys_state css; +};

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct res_counter res; + struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Pavel Emelianov
Herbert Poetzl wrote: On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Kirill Korotaev
Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct res_counter res; + struct list_head page_list; + struct container_subsys_state css; +}; + +struct page_container { + struct page *page; +

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Andrew Morton
On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev [EMAIL PROTECTED] wrote: Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct res_counter res; + struct list_head page_list; + struct

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Balbir Singh
On 3/11/07, Andrew Morton [EMAIL PROTECTED] wrote: On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev [EMAIL PROTECTED] wrote: Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct res_counter res; +

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Eric W. Biederman
Andrew Morton [EMAIL PROTECTED] writes: Yep. Straightforward machine partitioning. An attractive thing is that it 100% reuses existing page reclaim, unaltered. And misses every resource sharing opportunity in sight. Except for filtering the which pages are eligible for reclaim an RSS limit

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 06:04:28PM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Sun, Mar 11, 2007 at 12:08:16PM +0300, Pavel Emelianov wrote: Herbert Poetzl wrote: On Tue, Mar 06, 2007 at 02:00:36PM -0800, Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov

Re: [RFC][PATCH 2/7] RSS controller core

2007-03-11 Thread Herbert Poetzl
On Sun, Mar 11, 2007 at 04:51:11AM -0800, Andrew Morton wrote: On Sun, 11 Mar 2007 15:26:41 +0300 Kirill Korotaev [EMAIL PROTECTED] wrote: Andrew Morton wrote: On Tue, 06 Mar 2007 17:55:29 +0300 Pavel Emelianov [EMAIL PROTECTED] wrote: +struct rss_container { + struct

  1   2   >