[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote:
> On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote:
> > It seems reasonable to amend this case to say:
> > 
> > 1.
> > Any process with priv_sys_resource running in the global zone's
> > system project (project 0) will not subject to project.* or zone.*
> > resource controls.  System daemons which wish to be subject to the
> > global zone's resource controls can drop priv_sys_resource.
> 
> This "feels" risky for a patch... the effect here is to "un-manage"
> things which the customer may be expecting to be resource managed,
> potentially including a workload.  Or is the point that no-one using
> RM will be relying on the system project to limit things?

Currently, if rctls are specified in project(4) for the system project, they
are never actually applied during boot.  While this could be considered a bug,
this also means that currently, by default, admins will have to take additional
manual steps to resource manage project 0.  They are more likely to put what
they want to manage in a new/different project, and use project(4) to configure
rctls.

-Steve

> 
> This implies that can't give the system project 10 (or 100) shares
> after this proposal?
> 
> -dp
> 
> -- 
> Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - 
> blogs.sun.com/dp
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

I am working on a new spec.  I have an unanswered question from the
discussion:

> > The "SIZE" column will also be changed to "SWAP" for prstat
> > options a, T, and J, for users, tasks, and projects.
>
> The reason for not changing this column in the default output would be
> helpful.

I have a seperate private interface used by prstat(1m) to get aggregate swap
reserved by users, tasks, projects, and zones.  Default prstat output is
"per-process", and the information is accessed via /proc.

Currently, per-process swap reservation is not counted or made available via
/proc.  From proc(4):

 typedef struct psinfo {
...
size_t pr_size;   /* size of process image in Kbytes */
...

"size of process image" is pretty meaningless.  If we can change "pr_size" to
be "swap reserved by process", then we could change "SIZE" to "SWAP" for all
prstat(1m) output.  Would such a change to psinfo_t be reasonable  If
not, could potentially convert pr_pad1 to pr_swap

-Steve


On Mon, Nov 06, 2006 at 12:13:41PM -0800, Gary Winiger wrote:
> At the request of the project team, I've put this case into waiting need spec.
> When the spec is updated, it will be sent out and the timer reset.
> 
> Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Gary Winiger]

At the request of the project team, I've put this case into waiting need spec.
When the spec is updated, it will be sent out and the timer reset.

Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Mon, Nov 06, 2006 at 12:40:00PM +, Darren J Moffat wrote:
> Steve Lawrence wrote:
> >On Fri, Nov 03, 2006 at 09:36:45AM +, Darren J Moffat wrote:
> >>Steve Lawrence wrote:
> >>>Given a lack of supportive feedback, I'm going to revoke the proposed 
> >>>amendment
> >>>below.  To mitigate a zone admin setting a problematic swap limit on the 
> >>>global
> >>>zone, we will enhance zonecfg to:
> >>>
> >>>   1. Print a warning when setting swap (and lwp) limits on the global
> >>>  zone.  Since the swap limit will not go into effect until reboot,
> >>>  the admin has a change to modify his setting before it takes
> >>>  affect.
> >>>
> >>>   2. Enforce a reasonable minimum when setting swap (and lwp) limits
> >>>  on the global zone.
> >>What is the definition of reasonable ?  I think that being a
> >>default should be part of the case.
> >
> >I'm not sure what your second sentence means.  As for the first, how about:
> 
> I thought that what you meant was an actual number or percentage
> of the total available system swap.  Hence why I though it was a
> default value worthy of being part of the interfaces this case
> defines.
> 
> >reasonable minimum:  The amount of resource neccessary to boot a zone that
> >is has a default installation and configuration.
> >
> >My "minimum", I do not mean the system default value.  For example, for
> >the following rctl, 128 is the system default value:
> >
> ># prctl -n project.max-shm-ids $$
> >process: 425781: -sh
> >NAMEPRIVILEGE   VALUEFLAG   ACTION   
> >RECIPIENT
> >project.max-shm-ids
> >privileged128   -   deny   
> >-
> >system  16.8M max   deny   
> >-
> >
> >zone.max-swap will not have a system default value.  By default, every zone
> >will have access to all swap available on the system.
> 
> Okay.
> 
> >By minimum, I mean the minimum that zonecfg will allow you to set.
> >
> >zonecfg> add capped memory
> >zonecfg:capped memory> set zone.max-swap=1K
> >Error, minimum value for zone.max-swap is ###
> 
> Where does the ### come from.  I think I'm missing something here.
> 

### is the "reasonable minimum" described above.  Basing it in the system
software requirements instead of an arbitrary % of total system swap seems
better, especially since the total system swap can change at any time
by "swap [ -a | -d ]'.

-Steve

> -- 
> Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Darren J Moffat]

Steve Lawrence wrote:

On Fri, Nov 03, 2006 at 09:36:45AM +, Darren J Moffat wrote:

Steve Lawrence wrote:
Given a lack of supportive feedback, I'm going to revoke the proposed 
amendment
below.  To mitigate a zone admin setting a problematic swap limit on the 
global

zone, we will enhance zonecfg to:

1. Print a warning when setting swap (and lwp) limits on the global
   zone.  Since the swap limit will not go into effect until reboot,
   the admin has a change to modify his setting before it takes
   affect.

2. Enforce a reasonable minimum when setting swap (and lwp) limits
   on the global zone.

What is the definition of reasonable ?  I think that being a
default should be part of the case.


I'm not sure what your second sentence means.  As for the first, how about:


I thought that what you meant was an actual number or percentage
of the total available system swap.  Hence why I though it was a
default value worthy of being part of the interfaces this case
defines.


reasonable minimum:  The amount of resource neccessary to boot a zone that
is has a default installation and configuration.

My "minimum", I do not mean the system default value.  For example, for
the following rctl, 128 is the system default value:

# prctl -n project.max-shm-ids $$
process: 425781: -sh
NAMEPRIVILEGE   VALUEFLAG   ACTION   RECIPIENT
project.max-shm-ids
privileged128   -   deny -
system  16.8M max   deny -

zone.max-swap will not have a system default value.  By default, every zone
will have access to all swap available on the system.


Okay.


By minimum, I mean the minimum that zonecfg will allow you to set.

zonecfg> add capped memory
zonecfg:capped memory> set zone.max-swap=1K
Error, minimum value for zone.max-swap is ###


Where does the ### come from.  I think I'm missing something here.

--
Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Fri, Nov 03, 2006 at 09:36:45AM +, Darren J Moffat wrote:
> Steve Lawrence wrote:
> >Given a lack of supportive feedback, I'm going to revoke the proposed 
> >amendment
> >below.  To mitigate a zone admin setting a problematic swap limit on the 
> >global
> >zone, we will enhance zonecfg to:
> >
> > 1. Print a warning when setting swap (and lwp) limits on the global
> >zone.  Since the swap limit will not go into effect until reboot,
> >the admin has a change to modify his setting before it takes
> >affect.
> >
> > 2. Enforce a reasonable minimum when setting swap (and lwp) limits
> >on the global zone.
> 
> What is the definition of reasonable ?  I think that being a
> default should be part of the case.

I'm not sure what your second sentence means.  As for the first, how about:

reasonable minimum:  The amount of resource neccessary to boot a zone that
is has a default installation and configuration.

My "minimum", I do not mean the system default value.  For example, for
the following rctl, 128 is the system default value:

# prctl -n project.max-shm-ids $$
process: 425781: -sh
NAMEPRIVILEGE   VALUEFLAG   ACTION   RECIPIENT
project.max-shm-ids
privileged128   -   deny -
system  16.8M max   deny -

zone.max-swap will not have a system default value.  By default, every zone
will have access to all swap available on the system.

By minimum, I mean the minimum that zonecfg will allow you to set.

zonecfg> add capped memory
zonecfg:capped memory> set zone.max-swap=1K
Error, minimum value for zone.max-swap is ###

-Steve

> 
> -- 
> Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Darren J Moffat]

Steve Lawrence wrote:

Given a lack of supportive feedback, I'm going to revoke the proposed amendment
below.  To mitigate a zone admin setting a problematic swap limit on the global
zone, we will enhance zonecfg to:

1. Print a warning when setting swap (and lwp) limits on the global
   zone.  Since the swap limit will not go into effect until reboot,
   the admin has a change to modify his setting before it takes
   affect.

2. Enforce a reasonable minimum when setting swap (and lwp) limits
   on the global zone.


What is the definition of reasonable ?  I think that being a
default should be part of the case.

--
Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

Given a lack of supportive feedback, I'm going to revoke the proposed amendment
below.  To mitigate a zone admin setting a problematic swap limit on the global
zone, we will enhance zonecfg to:

1. Print a warning when setting swap (and lwp) limits on the global
   zone.  Since the swap limit will not go into effect until reboot,
   the admin has a change to modify his setting before it takes
   affect.

2. Enforce a reasonable minimum when setting swap (and lwp) limits
   on the global zone.

Currently, the rctl framework provides many mechanisms by which the admin
can make the system difficult to manage.  For instance, setting task.max-lwps
on project "user.root" can prevent root login.  If we want to make changes to
prevent admins from resource-controlling their way out of the box, I think we
need a broader case to address the whole problem.

-Steve

On Tue, Oct 31, 2006 at 03:24:18PM -0800, Steve Lawrence wrote:
> > > > I'm looking for this case to define how to preserve the current
> > > > model of "unlimited" unless one asks for a limit model in the
> > > > global zone.  I believe it is important from a system integrity 
> > > > and
> > > > maintenance perspective.  Other's may have different opinions.
> > > > If there is a compelling reason to deliver in phases, please 
> > > > discuss
> > > > that.
> > > 
> > > The global zone will have no swap limit by default.  The default 
> > > zone.max-swap
> > > rctl delivered on the global zone is UINT64_MAX, which is essentially
> > > unlimited.  Is that what you mean?
> > 
> > My point(s) here is not so much how things get done, but that
> > the global zone is in some ways special.  IIRC, before this
> > project, the GZ doesn't have a swap limit.  After this project
> > an administrator could set swap limit on the GZ.  Granted this
> > is administrative action and they get what they deserve/ask for.
> > However, it seemed to me that part of this case "should" (my
> > judgement) include some way to override the limit in case 
> > override is really desired.  As implied, perhaps by putting
> > root into project 0 at login or as part of daemon/service start
> > is a way to bypass the administrator's choice in the GZ for
> > some processes.  What I didn't see as part of this case is
> > the architecture to allow this bypass.  Perhaps I'm off base
> > for thinking it's necessary to protect against inadvertantly
> > not being able to administer the system from the GZ.
> > 
> 
> It seems reasonable to amend this case to say:
> 
>   1.
>   Any process with priv_sys_resource running in the global zone's
>   system project (project 0) will not subject to project.* or zone.*
>   resource controls.  System daemons which wish to be subject to the
>   global zone's resource controls can drop priv_sys_resource.
> 
>   2.
>   The "user.root" project will be removed, and root's default project
>   will be set to the "system" project via /etc/user_attr.
> 
> I'm not sure if (2) can be delivered via patch.  I need some guidance here.
> I'm also not sure how implementable (1) is until I do more investigation.
> 
> -Steve
> 
> > Gary..
> > 
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

I'm not sure it is within the domain of this case to to tell admins what they
should and shouldn't use the global zone for.

In any event, we are making it easy for admins to manage swap limits for zones
via zonecfg.

On Tue, Oct 31, 2006 at 05:58:24PM -0800, Michael Barto wrote:
> After all thus juggling, let make it simple for the system admin and use 
> some sort of fair share process to assignment and manage the swap for 
> all the zones. Personally I think that the global zone should use 
> minimum resources and be considered in the IT management processes to be 
> only like a system controller on a complex server. Keep your 
> applications out of the global zone!!!
> 
> Gary Winiger wrote:
> 
> This will not help root logins directly, but could by setting:
> 
>   usermod -K project=system root
>    
> 
> >>>   Or perhaps deliver root's entry this way to start with.
> >>> 
> >>>
> >>Would that be a reasonable change to make via patch?  Perhaps this change
> >>could be delivered to nevada, but not backported.
> >>
> >>It would be confusing to deliver this change, and also deliver the 
> >>"user.root"
> >>project.  If we made root's default project "system", then the "user.root"
> >>project should be removed.  "user.root" is kind of a bug anyhow, as SMF 
> >>does
> >>not run root services in "user.root".  Currently, only root processes 
> >>spawned
> >>by login/pam run in "user.root".
> >>   
> >>
> >
> >
> > 
> >
> Perhaps this issue should be run as a seperate fasttrack?  I need to 
> investigate the implementation impact.
>    
> 
> >>>   I'm looking for this case to define how to preserve the current
> >>>   model of "unlimited" unless one asks for a limit model in the
> >>>   global zone.  I believe it is important from a system integrity and
> >>>   maintenance perspective.  Other's may have different opinions.
> >>>   If there is a compelling reason to deliver in phases, please discuss
> >>>   that.
> >>> 
> >>>
> >>The global zone will have no swap limit by default.  The default 
> >>zone.max-swap
> >>rctl delivered on the global zone is UINT64_MAX, which is essentially
> >>unlimited.  Is that what you mean?
> >>   
> >>
> >
> > My point(s) here is not so much how things get done, but that
> > the global zone is in some ways special.  IIRC, before this
> > project, the GZ doesn't have a swap limit.  After this project
> > an administrator could set swap limit on the GZ.  Granted this
> > is administrative action and they get what they deserve/ask for.
> > However, it seemed to me that part of this case "should" (my
> > judgement) include some way to override the limit in case 
> > override is really desired.  As implied, perhaps by putting
> > root into project 0 at login or as part of daemon/service start
> > is a way to bypass the administrator's choice in the GZ for
> > some processes.  What I didn't see as part of this case is
> > the architecture to allow this bypass.  Perhaps I'm off base
> > for thinking it's necessary to protect against inadvertantly
> > not being able to administer the system from the GZ.
> >
> >Gary..
> > 
> >___
> >zones-discuss mailing list
> >zones-discuss@opensolaris.org
> >
> > 
> >
> 
> -- 
> 
> *Michael Barto*
> Software Architect
> 
>   LogiQwest Circle
> LogiQwest Inc.
> 16458 Bolsa Chica Street, # 15
> Huntington Beach, CA  92649
> http://www.logiqwest.com/
> 
>   [EMAIL PROTECTED] 
> Tel:  714 377 3705
> Fax: 714 840 3937
> Cell: 714 883 1949
> 
> *'tis a gift to be simple*
> 
> This e-mail may contain LogiQwest proprietary information and should be 
> treated as confidential.
> 
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Darren J Moffat]

Steve Lawrence wrote:

Would it be reasonable to propose special treatment of the global project 0
for all project and zone rctls?  Once could argue that capping system 
daemons

can only lead some sort of undesireable system failure.

This would of course exempt all global zone system daemons from resource
management.  To mitigate this, SMF could be leveraged to run application
daemons (or leaky/bad system daemons) in other projects. 
Please don't do that in a hardcoded way.  I don't mind if it is the 
default but it can't be hard coded.  One of the main reasons we have 
nfsd (and kcfd) is so that resource controls can be placed on system 
services.


Do you mean "one of the reasons that nfs and kcfd run in the system project"??


This is getting a bit off topic for this case I think.

Certainly by default.  kcfd also puts itself into the FX scheduling
class if FSS is not the default.  The idea was that you could control
how much CPU resource the kernel software crypto takes up (either give
it more or less depending on your needs).


With the advent of SMF this is actually really easy to do since you
just need to set the project/pool the start method runs in.

Also it is perfectly reasonable to have a case where no "useful" 
customer work happens in the global zone, ie it is a service processor

really, and all the real work happens in the non global zones.


Please elaborate on this.  I'm not sure I understand what you're getting at.


I think it might be my miss understanding of what this case really does.

The idea here is that the real application work all happens in a
non global zone,  the global zone is just there to manage the machine.
Unfortunately nfsd and kcfd are bad examples of services since their
work is global zone specific in this case (kcfd does run in a local
zone but the kcf thread pool it creates is global zone only).  I was
thinking more of services that could run in the global zone and if
they are they are lower priority than the local zone ones.

Now that I think about this case a bit more I think that it doesn't
actually change things in that area.


--
Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Wed, Nov 01, 2006 at 11:31:57AM +, Darren J Moffat wrote:
> Steve Lawrence wrote:
> >On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote:
> >>>This is how we treat cpu-shares. project 0 in the global zone has 
> >>>"infinite"
> >>>shares.
> >>>
> >>>This will not help root logins directly, but could by setting:
> >>>
> >>>   usermod -K project=system root
> >>Or perhaps deliver root's entry this way to start with.
> >
> >Would that be a reasonable change to make via patch?  Perhaps this change
> >could be delivered to nevada, but not backported.
> >
> >It would be confusing to deliver this change, and also deliver the 
> >"user.root"
> >project.  If we made root's default project "system", then the "user.root"
> >project should be removed.  "user.root" is kind of a bug anyhow, as SMF 
> >does
> >not run root services in "user.root".  Currently, only root processes 
> >spawned
> >by login/pam run in "user.root".
> 
> I actually felt that that was an interesting accident that had useful
> sideeffects.  It would though be better to formalise the distinction
> between services SMF starts and logins to root.  I think it is actually
> a good thing that they are in different projects by default.
> 
> Just because they are in different projects doesn't mean the 
> configuration of those projects can't be the same.

I'm proposing special kernel treatment of projid 0.  Another project cannot
be configured to be like this.  Are you proposing a new project attribute
which means "don't enforce zone rctls"??

-Steve

> 
> -- 
> Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

> >Would it be reasonable to propose special treatment of the global project 0
> >for all project and zone rctls?  Once could argue that capping system 
> >daemons
> >can only lead some sort of undesireable system failure.
> >
> >This would of course exempt all global zone system daemons from resource
> >management.  To mitigate this, SMF could be leveraged to run application
> >daemons (or leaky/bad system daemons) in other projects. 
> 
> Please don't do that in a hardcoded way.  I don't mind if it is the 
> default but it can't be hard coded.  One of the main reasons we have 
> nfsd (and kcfd) is so that resource controls can be placed on system 
> services.

Do you mean "one of the reasons that nfs and kcfd run in the system project"??

> 
> With the advent of SMF this is actually really easy to do since you
> just need to set the project/pool the start method runs in.
> 
> Also it is perfectly reasonable to have a case where no "useful" 
> customer work happens in the global zone, ie it is a service processor
> really, and all the real work happens in the non global zones.

Please elaborate on this.  I'm not sure I understand what you're getting at.

-Steve

> 
> -- 
> Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Darren J Moffat]

Steve Lawrence wrote:

On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote:

This is how we treat cpu-shares. project 0 in the global zone has "infinite"
shares.

This will not help root logins directly, but could by setting:

usermod -K project=system root

Or perhaps deliver root's entry this way to start with.


Would that be a reasonable change to make via patch?  Perhaps this change
could be delivered to nevada, but not backported.

It would be confusing to deliver this change, and also deliver the "user.root"
project.  If we made root's default project "system", then the "user.root"
project should be removed.  "user.root" is kind of a bug anyhow, as SMF does
not run root services in "user.root".  Currently, only root processes spawned
by login/pam run in "user.root".


I actually felt that that was an interesting accident that had useful
sideeffects.  It would though be better to formalise the distinction
between services SMF starts and logins to root.  I think it is actually
a good thing that they are in different projects by default.

Just because they are in different projects doesn't mean the 
configuration of those projects can't be the same.


--
Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Darren J Moffat]

Steve Lawrence wrote:

On Tue, Oct 31, 2006 at 10:59:23AM -0800, Gary Winiger wrote:

DETAIL:

  1. "zone.max-swap" resource control.

 Limits swap consumed by user process address space mappings and
 tmpfs mounts within a zone.
 zone.max-swap will be configurable on both the global zone, and
 non-global zones.  The affect on processes in a zone reaching its
 zone.max-swap limit is the same as if all system swap is reserved.
 Callers of mmap(2) and sbrk(2) will receive EAGAIN.  Writes to
 tmpfs will return ENOSPC, which is the same errno returned when
 a tmpfs mount reaches it's "size" mount option.  The "size" mount
 option limits the quantity of swap that a tmpfs mount can reserve.

 While a low zone.max-swap setting for the global zone can lead to
 a difficult-to-administer global zone, the same problem exists
 today when configuring the zone.max-lwps resource control on the
 global zone, or when all system swap is reserved.

While other controls seem to be capped by the process/thread's
project, it seems that use of swap like max-lwps might be
important for the global zone project 0.  Has consideration been
given to allowing global zone "system" processes to not be
bound by GZ zone.max-swap?
It might be better than having to reboot or boot single to fix
a problem that could otherwise be fixed in the GZ.


This is how we treat cpu-shares. project 0 in the global zone has "infinite"
shares.

This will not help root logins directly, but could by setting:

usermod -K project=system root

Would it be reasonable to propose special treatment of the global project 0
for all project and zone rctls?  Once could argue that capping system daemons
can only lead some sort of undesireable system failure.

This would of course exempt all global zone system daemons from resource
management.  To mitigate this, SMF could be leveraged to run application
daemons (or leaky/bad system daemons) in other projects. 


Please don't do that in a hardcoded way.  I don't mind if it is the 
default but it can't be hard coded.  One of the main reasons we have 
nfsd (and kcfd) is so that resource controls can be placed on system 
services.


With the advent of SMF this is actually really easy to do since you
just need to set the project/pool the start method runs in.

Also it is perfectly reasonable to have a case where no "useful" 
customer work happens in the global zone, ie it is a service processor

really, and all the real work happens in the non global zones.

--
Darren J Moffat
___
zones-discuss mailing list
zones-discuss@opensolaris.org

Re: [zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Michael Barto]

After all thus juggling, let make it simple for the system admin and
use some sort of fair share process to assignment and manage the swap
for all the zones. Personally I think that the global zone should use
minimum resources and be considered in the IT management processes to
be only like a system controller on a complex server. Keep your
applications out of the global zone!!!

Gary Winiger wrote:

  

  
This will not help root logins directly, but could by setting:

	usermod -K project=system root

  
  	Or perhaps deliver root's entry this way to start with.
  

Would that be a reasonable change to make via patch?  Perhaps this change
could be delivered to nevada, but not backported.

It would be confusing to deliver this change, and also deliver the "user.root"
project.  If we made root's default project "system", then the "user.root"
project should be removed.  "user.root" is kind of a bug anyhow, as SMF does
not run root services in "user.root".  Currently, only root processes spawned
by login/pam run in "user.root".

  
  

  
  

  
Perhaps this issue should be run as a seperate fasttrack?  I need to 
investigate the implementation impact.

  
  	I'm looking for this case to define how to preserve the current
	model of "unlimited" unless one asks for a limit model in the
	global zone.  I believe it is important from a system integrity and
	maintenance perspective.  Other's may have different opinions.
	If there is a compelling reason to deliver in phases, please discuss
	that.
  

The global zone will have no swap limit by default.  The default zone.max-swap
rctl delivered on the global zone is UINT64_MAX, which is essentially
unlimited.  Is that what you mean?

  
  
	My point(s) here is not so much how things get done, but that
	the global zone is in some ways special.  IIRC, before this
	project, the GZ doesn't have a swap limit.  After this project
	an administrator could set swap limit on the GZ.  Granted this
	is administrative action and they get what they deserve/ask for.
	However, it seemed to me that part of this case "should" (my
	judgement) include some way to override the limit in case 
	override is really desired.  As implied, perhaps by putting
	root into project 0 at login or as part of daemon/service start
	is a way to bypass the administrator's choice in the GZ for
	some processes.  What I didn't see as part of this case is
	the architecture to allow this bypass.  Perhaps I'm off base
	for thinking it's necessary to protect against inadvertantly
	not being able to administer the system from the GZ.

Gary..
	
___
zones-discuss mailing list
zones-discuss@opensolaris.org

  


-- 





  

  
  


  Michael Barto
  Software Architect
  
  
  
  


   LogiQwest
Inc.
16458 Bolsa Chica Street, # 15
Huntington Beach, CA  92649
  http://www.logiqwest.com/
  
  
     
  [EMAIL PROTECTED]
Tel:  714 377 3705
Fax: 714 840 3937
Cell: 714 883 1949
  
  


  'tis a gift to be
simple
   


   This e-mail may contain
LogiQwest
proprietary information and should be treated as confidential. 

  






___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Dan Price]

On Tue 31 Oct 2006 at 04:10PM, Steve Lawrence wrote:
> On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote:
> > On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote:
> > > It seems reasonable to amend this case to say:
> > > 
> > >   1.
> > >   Any process with priv_sys_resource running in the global zone's
> > >   system project (project 0) will not subject to project.* or zone.*
> > >   resource controls.  System daemons which wish to be subject to the
> > >   global zone's resource controls can drop priv_sys_resource.
> > 
> > This "feels" risky for a patch... the effect here is to "un-manage"
> > things which the customer may be expecting to be resource managed,
> > potentially including a workload.  Or is the point that no-one using
> > RM will be relying on the system project to limit things?
> > 
> > This implies that can't give the system project 10 (or 100) shares
> > after this proposal?
> 
> For shares, you can't do this now anyway.  The system project implicitly has
> infinite shares, regardless of what shares you have set.  This is documented
> in FSS(7):
> 
>  By default, project system (project ID 0) includes all  sys-
>  tem  daemons  started  by  initialization scripts and has an
>  "unlimited"  amount  of  shares.  That  is,  it  is   always
>  scheduled first no matter how many shares are given to other
>  projects.
> 
> I've proposed extending this for all resource controls, not just *.cpu-shares.

My apologies... I should've done some more digging first.

-dp

-- 
Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - blogs.sun.com/dp
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Tue, Oct 31, 2006 at 03:28:31PM -0800, Dan Price wrote:
> On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote:
> > It seems reasonable to amend this case to say:
> > 
> > 1.
> > Any process with priv_sys_resource running in the global zone's
> > system project (project 0) will not subject to project.* or zone.*
> > resource controls.  System daemons which wish to be subject to the
> > global zone's resource controls can drop priv_sys_resource.
> 
> This "feels" risky for a patch... the effect here is to "un-manage"
> things which the customer may be expecting to be resource managed,
> potentially including a workload.  Or is the point that no-one using
> RM will be relying on the system project to limit things?
> 
> This implies that can't give the system project 10 (or 100) shares
> after this proposal?

For shares, you can't do this now anyway.  The system project implicitly has
infinite shares, regardless of what shares you have set.  This is documented
in FSS(7):

 By default, project system (project ID 0) includes all  sys-
 tem  daemons  started  by  initialization scripts and has an
 "unlimited"  amount  of  shares.  That  is,  it  is   always
 scheduled first no matter how many shares are given to other
 projects.

I've proposed extending this for all resource controls, not just *.cpu-shares.

-Steve

> 
> -dp
> 
> -- 
> Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - 
> blogs.sun.com/dp
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Dan Price]

On Tue 31 Oct 2006 at 03:24PM, Steve Lawrence wrote:
> It seems reasonable to amend this case to say:
> 
>   1.
>   Any process with priv_sys_resource running in the global zone's
>   system project (project 0) will not subject to project.* or zone.*
>   resource controls.  System daemons which wish to be subject to the
>   global zone's resource controls can drop priv_sys_resource.

This "feels" risky for a patch... the effect here is to "un-manage"
things which the customer may be expecting to be resource managed,
potentially including a workload.  Or is the point that no-one using
RM will be relying on the system project to limit things?

This implies that can't give the system project 10 (or 100) shares
after this proposal?

-dp

-- 
Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - blogs.sun.com/dp
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

> > >   I'm looking for this case to define how to preserve the current
> > >   model of "unlimited" unless one asks for a limit model in the
> > >   global zone.  I believe it is important from a system integrity and
> > >   maintenance perspective.  Other's may have different opinions.
> > >   If there is a compelling reason to deliver in phases, please discuss
> > >   that.
> > 
> > The global zone will have no swap limit by default.  The default 
> > zone.max-swap
> > rctl delivered on the global zone is UINT64_MAX, which is essentially
> > unlimited.  Is that what you mean?
> 
>   My point(s) here is not so much how things get done, but that
>   the global zone is in some ways special.  IIRC, before this
>   project, the GZ doesn't have a swap limit.  After this project
>   an administrator could set swap limit on the GZ.  Granted this
>   is administrative action and they get what they deserve/ask for.
>   However, it seemed to me that part of this case "should" (my
>   judgement) include some way to override the limit in case 
>   override is really desired.  As implied, perhaps by putting
>   root into project 0 at login or as part of daemon/service start
>   is a way to bypass the administrator's choice in the GZ for
>   some processes.  What I didn't see as part of this case is
>   the architecture to allow this bypass.  Perhaps I'm off base
>   for thinking it's necessary to protect against inadvertantly
>   not being able to administer the system from the GZ.
> 

It seems reasonable to amend this case to say:

1.
Any process with priv_sys_resource running in the global zone's
system project (project 0) will not subject to project.* or zone.*
resource controls.  System daemons which wish to be subject to the
global zone's resource controls can drop priv_sys_resource.

2.
The "user.root" project will be removed, and root's default project
will be set to the "system" project via /etc/user_attr.

I'm not sure if (2) can be delivered via patch.  I need some guidance here.
I'm also not sure how implementable (1) is until I do more investigation.

-Steve

> Gary..
>   
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Gary Winiger]

> > > This will not help root logins directly, but could by setting:
> > > 
> > >   usermod -K project=system root
> > 
> > Or perhaps deliver root's entry this way to start with.
> 
> Would that be a reasonable change to make via patch?  Perhaps this change
> could be delivered to nevada, but not backported.
> 
> It would be confusing to deliver this change, and also deliver the "user.root"
> project.  If we made root's default project "system", then the "user.root"
> project should be removed.  "user.root" is kind of a bug anyhow, as SMF does
> not run root services in "user.root".  Currently, only root processes spawned
> by login/pam run in "user.root".


> > > Perhaps this issue should be run as a seperate fasttrack?  I need to 
> > > investigate the implementation impact.
> > 
> > I'm looking for this case to define how to preserve the current
> > model of "unlimited" unless one asks for a limit model in the
> > global zone.  I believe it is important from a system integrity and
> > maintenance perspective.  Other's may have different opinions.
> > If there is a compelling reason to deliver in phases, please discuss
> > that.
> 
> The global zone will have no swap limit by default.  The default zone.max-swap
> rctl delivered on the global zone is UINT64_MAX, which is essentially
> unlimited.  Is that what you mean?

My point(s) here is not so much how things get done, but that
the global zone is in some ways special.  IIRC, before this
project, the GZ doesn't have a swap limit.  After this project
an administrator could set swap limit on the GZ.  Granted this
is administrative action and they get what they deserve/ask for.
However, it seemed to me that part of this case "should" (my
judgement) include some way to override the limit in case 
override is really desired.  As implied, perhaps by putting
root into project 0 at login or as part of daemon/service start
is a way to bypass the administrator's choice in the GZ for
some processes.  What I didn't see as part of this case is
the architecture to allow this bypass.  Perhaps I'm off base
for thinking it's necessary to protect against inadvertantly
not being able to administer the system from the GZ.

Gary..

___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Tue, Oct 31, 2006 at 12:02:58PM -0800, Gary Winiger wrote:
> > This is how we treat cpu-shares. project 0 in the global zone has "infinite"
> > shares.
> > 
> > This will not help root logins directly, but could by setting:
> > 
> > usermod -K project=system root
> 
>   Or perhaps deliver root's entry this way to start with.

Would that be a reasonable change to make via patch?  Perhaps this change
could be delivered to nevada, but not backported.

It would be confusing to deliver this change, and also deliver the "user.root"
project.  If we made root's default project "system", then the "user.root"
project should be removed.  "user.root" is kind of a bug anyhow, as SMF does
not run root services in "user.root".  Currently, only root processes spawned
by login/pam run in "user.root".

> 
> > Would it be reasonable to propose special treatment of the global project 0
> > for all project and zone rctls?  Once could argue that capping system 
> > daemons
> > can only lead some sort of undesireable system failure.
> > 
> > This would of course exempt all global zone system daemons from resource
> > management.  To mitigate this, SMF could be leveraged to run application
> > daemons (or leaky/bad system daemons) in other projects.
> 
>   leaky or bad system daemons should be fixed, but defining projects
>   may well be a workaround ;-}

Agreed.

>  
> > Perhaps this issue should be run as a seperate fasttrack?  I need to 
> > investigate the implementation impact.
> 
>   I'm looking for this case to define how to preserve the current
>   model of "unlimited" unless one asks for a limit model in the
>   global zone.  I believe it is important from a system integrity and
>   maintenance perspective.  Other's may have different opinions.
>   If there is a compelling reason to deliver in phases, please discuss
>   that.

The global zone will have no swap limit by default.  The default zone.max-swap
rctl delivered on the global zone is UINT64_MAX, which is essentially
unlimited.  Is that what you mean?

-Steve

> 
> Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Gary Winiger]

> This is how we treat cpu-shares. project 0 in the global zone has "infinite"
> shares.
> 
> This will not help root logins directly, but could by setting:
> 
>   usermod -K project=system root

Or perhaps deliver root's entry this way to start with.

> Would it be reasonable to propose special treatment of the global project 0
> for all project and zone rctls?  Once could argue that capping system daemons
> can only lead some sort of undesireable system failure.
> 
> This would of course exempt all global zone system daemons from resource
> management.  To mitigate this, SMF could be leveraged to run application
> daemons (or leaky/bad system daemons) in other projects.

leaky or bad system daemons should be fixed, but defining projects
may well be a workaround ;-}
 
> Perhaps this issue should be run as a seperate fasttrack?  I need to 
> investigate the implementation impact.

I'm looking for this case to define how to preserve the current
model of "unlimited" unless one asks for a limit model in the
global zone.  I believe it is important from a system integrity and
maintenance perspective.  Other's may have different opinions.
If there is a compelling reason to deliver in phases, please discuss
that.

Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Steve Lawrence]

On Tue, Oct 31, 2006 at 10:59:23AM -0800, Gary Winiger wrote:
> > DETAIL:
> > 
> >   1. "zone.max-swap" resource control.
> > 
> >  Limits swap consumed by user process address space mappings and
> >  tmpfs mounts within a zone.
> 
> >  zone.max-swap will be configurable on both the global zone, and
> >  non-global zones.  The affect on processes in a zone reaching its
> >  zone.max-swap limit is the same as if all system swap is reserved.
> >  Callers of mmap(2) and sbrk(2) will receive EAGAIN.  Writes to
> >  tmpfs will return ENOSPC, which is the same errno returned when
> >  a tmpfs mount reaches it's "size" mount option.  The "size" mount
> >  option limits the quantity of swap that a tmpfs mount can reserve.
> > 
> >  While a low zone.max-swap setting for the global zone can lead to
> >  a difficult-to-administer global zone, the same problem exists
> >  today when configuring the zone.max-lwps resource control on the
> >  global zone, or when all system swap is reserved.
> 
>   While other controls seem to be capped by the process/thread's
>   project, it seems that use of swap like max-lwps might be
>   important for the global zone project 0.  Has consideration been
>   given to allowing global zone "system" processes to not be
>   bound by GZ zone.max-swap?
>   It might be better than having to reboot or boot single to fix
>   a problem that could otherwise be fixed in the GZ.

This is how we treat cpu-shares. project 0 in the global zone has "infinite"
shares.

This will not help root logins directly, but could by setting:

usermod -K project=system root

Would it be reasonable to propose special treatment of the global project 0
for all project and zone rctls?  Once could argue that capping system daemons
can only lead some sort of undesireable system failure.

This would of course exempt all global zone system daemons from resource
management.  To mitigate this, SMF could be leveraged to run application
daemons (or leaky/bad system daemons) in other projects. 

Perhaps this issue should be run as a seperate fasttrack?  I need to 
investigate the implementation impact.

-Steve

> 
> Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org

[zones-discuss] Re: PSARC/2006/598 Swap resource control; locked memory RM improvements

[Gary Winiger]

> DETAIL:
> 
>   1. "zone.max-swap" resource control.
> 
>  Limits swap consumed by user process address space mappings and
>  tmpfs mounts within a zone.

>  zone.max-swap will be configurable on both the global zone, and
>  non-global zones.  The affect on processes in a zone reaching its
>  zone.max-swap limit is the same as if all system swap is reserved.
>  Callers of mmap(2) and sbrk(2) will receive EAGAIN.  Writes to
>  tmpfs will return ENOSPC, which is the same errno returned when
>  a tmpfs mount reaches it's "size" mount option.  The "size" mount
>  option limits the quantity of swap that a tmpfs mount can reserve.
> 
>  While a low zone.max-swap setting for the global zone can lead to
>  a difficult-to-administer global zone, the same problem exists
>  today when configuring the zone.max-lwps resource control on the
>  global zone, or when all system swap is reserved.

While other controls seem to be capped by the process/thread's
project, it seems that use of swap like max-lwps might be
important for the global zone project 0.  Has consideration been
given to allowing global zone "system" processes to not be
bound by GZ zone.max-swap?
It might be better than having to reboot or boot single to fix
a problem that could otherwise be fixed in the GZ.

Gary..
___
zones-discuss mailing list
zones-discuss@opensolaris.org