Re: [zones-discuss] Re: improved zones/RM integration
Hi Jerry,
Jerry Jelinek wrote On 08/17/06 09:20,:
Steffen,
Steffen Weiberle wrote:
Jerry Jelinek wrote On 08/16/06 18:14,:
Steffen,
Thanks for your comments. Responses in-line.
Steffen Weiberle wrote:
Hi Jerry, this is great.
I have a few comments below.
Thanks
Steffen
1) "Hard" vs. "Soft" RM configuration within zonecfg
We will enhance zonecfg(1M) so that the user can configure
basic RM
capabilities in a structured way.
Various existing and upcoming RM features can be broken down
into "hard" vs. "soft" partitioning of the system's resources.
With "hard" partitioning, resources are dedicated to the zone
using
processor sets (psets) and memory sets (msets). With "soft"
partitioning, resources are shared, but capped, with an upper
limit
on their use by the zone.
Hard|Soft
-
cpu| psets | cpu-caps
memory | msets | rcapd
Within zonecfg we will organize these various RM features into
four
basic zonecfg resources so that it is simple for a user to
understand
and configure the RM features that are to be used with their zone.
Note that zonecfg "resources" are not the same as the system's
cpu & memory resources or "resource management". Within
zonecfg, a
"resource" is the name of a top-level property group for the
zone (see
zonecfg(1M) for more information).
Are you saying just the names are different, or are there other
differences as well?
Unfortunately the word "resource" is overloaded here. zonecfg(1M) uses
it to mean a group of properties which has nothing to do with
system resources (e.g. cpu or memory) or how the word "resource" is
used under the umbrella of Solaris Resource Management.
Now I am more confused. You mean
zonecfg pool[adm,cfg]
---
dedicated-cpu!= CPU resource pool (pset and pool)
capped-cpu != cpu-caps
dedicated-memory != memory resource pool
Sorry, I misunderstood your question here. The mapping *is* to these
RM features. I actually have this shown about two paragraphs down
from here. I thought you were talking about the word 'resource'
and how we use that word to mean different things in zones and RM.
OK Thanks
The four new zonecfg resources are:
dedicated-cpu
capped-cpu (future, after cpu-caps are integrated)
dedicated-memory (future, after memory sets are integrated)
capped-memory
Each of these zonecfg resources will have properties that are
appropriate to the RM capabilities associated with that resource.
Zonecfg will only allow one instance of each these resource to be
configured and it will not allow conflicting resources to be added
(e.g. dedicated-cpu and capped-cpu are mutually exclusive).
The mapping of these new zonecfg resources to the underlying RM
feature
is:
dedicated-cpu -> temporary pset
dedicated-memory -> temporary mset
capped-cpu -> cpu-cap rctl [14]
capped-memory -> rcapd running in global zone
2) Temporary Pools.
We will implement the concept of "temporary pools" within the
pools
framework.
To improve the integration of zones and pools we are allowing the
configuration of some basic pool attributes within zonecfg, as
described above in section 1. However, we do not want to extend
zonecfg to completely and directly manage standard pool
configurations.
That would lead to confusion and inconsistency regarding which
tool to
use and where configuration data is stored. Temporary pools
sidesteps
this problem and allows zones to dynamically create a simple
pool/pset
configuration for the basic case where a sysadmin just wants a
specified number of processors dedicated to the zone (and
eventually a
dedicated amount of memory).
We believe that the ability to simply specify a fixed number of
cpus
(and eventually a mset size) meets the needs of a large
percentage of
zones users who need "hard" partitioning (e.g. to meet licensing
restrictions).
If a dedicated-cpu (and/or eventually a dedicated-memory)
resource is
configured for the zone, then when the zone boots zoneadmd will
enable
pools if necessary and create a temporary pool dedicated for
the zones
use. Zoneadmd will dynamically create a pool & pset (and/or
eventually
a mset) and assign the number of cpus specified in zonecfg to that
pset. The temporary pool & pset will be named
'SUNWtmp_{zonename}'.
Zonecfg validation will disallow an explicit 'pool' property name
beginning with 'SUNWtmp'.
Zoneadmd will set the 'pset.min' and 'pset.max' pset
properties, as
well as the 'pool.importance' pool property, based on the values
specified for dedicated-cpu's 'ncpus' and 'importance' propertie
Re: [zones-discuss] Re: improved zones/RM integration
Steffen,
Steffen Weiberle wrote:
Jerry Jelinek wrote On 08/16/06 18:14,:
Steffen,
Thanks for your comments. Responses in-line.
Steffen Weiberle wrote:
Hi Jerry, this is great.
I have a few comments below.
Thanks
Steffen
1) "Hard" vs. "Soft" RM configuration within zonecfg
We will enhance zonecfg(1M) so that the user can configure basic RM
capabilities in a structured way.
Various existing and upcoming RM features can be broken down
into "hard" vs. "soft" partitioning of the system's resources.
With "hard" partitioning, resources are dedicated to the zone using
processor sets (psets) and memory sets (msets). With "soft"
partitioning, resources are shared, but capped, with an upper limit
on their use by the zone.
Hard|Soft
-
cpu| psets | cpu-caps
memory | msets | rcapd
Within zonecfg we will organize these various RM features into four
basic zonecfg resources so that it is simple for a user to
understand
and configure the RM features that are to be used with their zone.
Note that zonecfg "resources" are not the same as the system's
cpu & memory resources or "resource management". Within zonecfg, a
"resource" is the name of a top-level property group for the
zone (see
zonecfg(1M) for more information).
Are you saying just the names are different, or are there other
differences as well?
Unfortunately the word "resource" is overloaded here. zonecfg(1M) uses
it to mean a group of properties which has nothing to do with
system resources (e.g. cpu or memory) or how the word "resource" is
used under the umbrella of Solaris Resource Management.
Now I am more confused. You mean
zonecfg pool[adm,cfg]
---
dedicated-cpu!= CPU resource pool (pset and pool)
capped-cpu != cpu-caps
dedicated-memory != memory resource pool
Sorry, I misunderstood your question here. The mapping *is* to these
RM features. I actually have this shown about two paragraphs down
from here. I thought you were talking about the word 'resource'
and how we use that word to mean different things in zones and RM.
The four new zonecfg resources are:
dedicated-cpu
capped-cpu (future, after cpu-caps are integrated)
dedicated-memory (future, after memory sets are integrated)
capped-memory
Each of these zonecfg resources will have properties that are
appropriate to the RM capabilities associated with that resource.
Zonecfg will only allow one instance of each these resource to be
configured and it will not allow conflicting resources to be added
(e.g. dedicated-cpu and capped-cpu are mutually exclusive).
The mapping of these new zonecfg resources to the underlying RM
feature
is:
dedicated-cpu -> temporary pset
dedicated-memory -> temporary mset
capped-cpu -> cpu-cap rctl [14]
capped-memory -> rcapd running in global zone
2) Temporary Pools.
We will implement the concept of "temporary pools" within the pools
framework.
To improve the integration of zones and pools we are allowing the
configuration of some basic pool attributes within zonecfg, as
described above in section 1. However, we do not want to extend
zonecfg to completely and directly manage standard pool
configurations.
That would lead to confusion and inconsistency regarding which
tool to
use and where configuration data is stored. Temporary pools
sidesteps
this problem and allows zones to dynamically create a simple
pool/pset
configuration for the basic case where a sysadmin just wants a
specified number of processors dedicated to the zone (and
eventually a
dedicated amount of memory).
We believe that the ability to simply specify a fixed number of
cpus
(and eventually a mset size) meets the needs of a large
percentage of
zones users who need "hard" partitioning (e.g. to meet licensing
restrictions).
If a dedicated-cpu (and/or eventually a dedicated-memory)
resource is
configured for the zone, then when the zone boots zoneadmd will
enable
pools if necessary and create a temporary pool dedicated for the
zones
use. Zoneadmd will dynamically create a pool & pset (and/or
eventually
a mset) and assign the number of cpus specified in zonecfg to that
pset. The temporary pool & pset will be named
'SUNWtmp_{zonename}'.
Zonecfg validation will disallow an explicit 'pool' property name
beginning with 'SUNWtmp'.
Zoneadmd will set the 'pset.min' and 'pset.max' pset properties, as
well as the 'pool.importance' pool property, based on the values
specified for dedicated-cpu's 'ncpus' and 'importance' properties
in zonecfg, as described above in section 1.
If the cpu (or memory) resou
Re: [zones-discuss] Re: improved zones/RM integration
Jerry Jelinek wrote On 08/16/06 18:14,:
Steffen,
Thanks for your comments. Responses in-line.
Steffen Weiberle wrote:
Hi Jerry, this is great.
I have a few comments below.
Thanks
Steffen
1) "Hard" vs. "Soft" RM configuration within zonecfg
We will enhance zonecfg(1M) so that the user can configure basic RM
capabilities in a structured way.
Various existing and upcoming RM features can be broken down
into "hard" vs. "soft" partitioning of the system's resources.
With "hard" partitioning, resources are dedicated to the zone using
processor sets (psets) and memory sets (msets). With "soft"
partitioning, resources are shared, but capped, with an upper limit
on their use by the zone.
Hard|Soft
-
cpu| psets | cpu-caps
memory | msets | rcapd
Within zonecfg we will organize these various RM features into four
basic zonecfg resources so that it is simple for a user to
understand
and configure the RM features that are to be used with their zone.
Note that zonecfg "resources" are not the same as the system's
cpu & memory resources or "resource management". Within zonecfg, a
"resource" is the name of a top-level property group for the zone
(see
zonecfg(1M) for more information).
Are you saying just the names are different, or are there other
differences as well?
Unfortunately the word "resource" is overloaded here. zonecfg(1M) uses
it to mean a group of properties which has nothing to do with
system resources (e.g. cpu or memory) or how the word "resource" is
used under the umbrella of Solaris Resource Management.
Now I am more confused. You mean
zonecfg pool[adm,cfg]
---
dedicated-cpu!= CPU resource pool (pset and pool)
capped-cpu != cpu-caps
dedicated-memory != memory resource pool
The four new zonecfg resources are:
dedicated-cpu
capped-cpu (future, after cpu-caps are integrated)
dedicated-memory (future, after memory sets are integrated)
capped-memory
Each of these zonecfg resources will have properties that are
appropriate to the RM capabilities associated with that resource.
Zonecfg will only allow one instance of each these resource to be
configured and it will not allow conflicting resources to be added
(e.g. dedicated-cpu and capped-cpu are mutually exclusive).
The mapping of these new zonecfg resources to the underlying RM
feature
is:
dedicated-cpu -> temporary pset
dedicated-memory -> temporary mset
capped-cpu -> cpu-cap rctl [14]
capped-memory -> rcapd running in global zone
2) Temporary Pools.
We will implement the concept of "temporary pools" within the pools
framework.
To improve the integration of zones and pools we are allowing the
configuration of some basic pool attributes within zonecfg, as
described above in section 1. However, we do not want to extend
zonecfg to completely and directly manage standard pool
configurations.
That would lead to confusion and inconsistency regarding which
tool to
use and where configuration data is stored. Temporary pools
sidesteps
this problem and allows zones to dynamically create a simple
pool/pset
configuration for the basic case where a sysadmin just wants a
specified number of processors dedicated to the zone (and
eventually a
dedicated amount of memory).
We believe that the ability to simply specify a fixed number of cpus
(and eventually a mset size) meets the needs of a large
percentage of
zones users who need "hard" partitioning (e.g. to meet licensing
restrictions).
If a dedicated-cpu (and/or eventually a dedicated-memory)
resource is
configured for the zone, then when the zone boots zoneadmd will
enable
pools if necessary and create a temporary pool dedicated for the
zones
use. Zoneadmd will dynamically create a pool & pset (and/or
eventually
a mset) and assign the number of cpus specified in zonecfg to that
pset. The temporary pool & pset will be named 'SUNWtmp_{zonename}'.
Zonecfg validation will disallow an explicit 'pool' property name
beginning with 'SUNWtmp'.
Zoneadmd will set the 'pset.min' and 'pset.max' pset properties, as
well as the 'pool.importance' pool property, based on the values
specified for dedicated-cpu's 'ncpus' and 'importance' properties
in zonecfg, as described above in section 1.
If the cpu (or memory) resources needed to create the temporary pool
are unavailable, zoneadmd will issue an error and the zone won't
boot.
When the zone is halted, the temporary pool & pset will be
destroyed.
We will add a new boolean libpool(3LIB) property ('temporary')
that can
exist on pools and any pool res
Re: [zones-discuss] Re: improved zones/RM integration
Steffen, Thanks for your comments. Responses in-line. Steffen Weiberle wrote: Hi Jerry, this is great. I have a few comments below. Thanks Steffen 1) "Hard" vs. "Soft" RM configuration within zonecfg We will enhance zonecfg(1M) so that the user can configure basic RM capabilities in a structured way. Various existing and upcoming RM features can be broken down into "hard" vs. "soft" partitioning of the system's resources. With "hard" partitioning, resources are dedicated to the zone using processor sets (psets) and memory sets (msets). With "soft" partitioning, resources are shared, but capped, with an upper limit on their use by the zone. Hard|Soft - cpu| psets | cpu-caps memory | msets | rcapd Within zonecfg we will organize these various RM features into four basic zonecfg resources so that it is simple for a user to understand and configure the RM features that are to be used with their zone. Note that zonecfg "resources" are not the same as the system's cpu & memory resources or "resource management". Within zonecfg, a "resource" is the name of a top-level property group for the zone (see zonecfg(1M) for more information). Are you saying just the names are different, or are there other differences as well? Unfortunately the word "resource" is overloaded here. zonecfg(1M) uses it to mean a group of properties which has nothing to do with system resources (e.g. cpu or memory) or how the word "resource" is used under the umbrella of Solaris Resource Management. The four new zonecfg resources are: dedicated-cpu capped-cpu (future, after cpu-caps are integrated) dedicated-memory (future, after memory sets are integrated) capped-memory Each of these zonecfg resources will have properties that are appropriate to the RM capabilities associated with that resource. Zonecfg will only allow one instance of each these resource to be configured and it will not allow conflicting resources to be added (e.g. dedicated-cpu and capped-cpu are mutually exclusive). The mapping of these new zonecfg resources to the underlying RM feature is: dedicated-cpu -> temporary pset dedicated-memory -> temporary mset capped-cpu -> cpu-cap rctl [14] capped-memory -> rcapd running in global zone Temporary psets and msets are described below, in section 2. Rcapd enhancements for running in the global zone are described below in section 4. The valid properties for each of these new zonecfg resources will be: dedicated-cpu ncpus importance capped-cpu ncpus dedicated-memory physical virtual importance capped-memory physical virtual The meaning of each of these properties is as follows: dedicated-cpu ncpus:This can be a positive integer or range. A value of '2' means two cpus, a value of '2-4' means a range of two to four cpus. This sets the 'pset.min' and 'pset.max' properties on the temporary pset. importance: This property is optional. It can be a positive integer. It sets the 'pool.importance' property on the temporary pool. capped-cpu This resource group and its property will not be delivered as part of this project since cpu-caps are still under development. However, our thinking on this is described here for completeness. ncpus:This is a positive decimal. The 'ncpus' property actually maps to the zone.cpu-cap rctl. This property will be implemented as a special case of the new zones rctl aliases which are described below in section 3. The special case handling of this property will normalize the value so that it corresponds to units of cpus and is similar to the 'ncpus' property under the dedicated-cpu resource group. However, it won't accept a range and it will accept a decimal number. For example, when using 'ncpus' in the dedicated-cpu resource group, a value of 1 means one dedicated cpu. When using 'ncpus' in the capped-cpu resource group, a value of 1 means 100% of a cpu is the cap setting. A value of 1.25 means 125%, since 100% corresponds to one full cpu on the system when using cpu caps. The idea here is to align the 'ncpus' units as closely as possible in these two cases (dedicated-cpu vs. capped-cpu), given the limitations and capabilities of the two underlying mechanisms (pset vs. rctl). The 'ncpus'
Re: [zones-discuss] Re: improved zones/RM integration
Hi Jerry, this is great. I have a few comments below. Thanks Steffen 1) "Hard" vs. "Soft" RM configuration within zonecfg We will enhance zonecfg(1M) so that the user can configure basic RM capabilities in a structured way. Various existing and upcoming RM features can be broken down into "hard" vs. "soft" partitioning of the system's resources. With "hard" partitioning, resources are dedicated to the zone using processor sets (psets) and memory sets (msets). With "soft" partitioning, resources are shared, but capped, with an upper limit on their use by the zone. Hard|Soft - cpu| psets | cpu-caps memory | msets | rcapd Within zonecfg we will organize these various RM features into four basic zonecfg resources so that it is simple for a user to understand and configure the RM features that are to be used with their zone. Note that zonecfg "resources" are not the same as the system's cpu & memory resources or "resource management". Within zonecfg, a "resource" is the name of a top-level property group for the zone (see zonecfg(1M) for more information). Are you saying just the names are different, or are there other differences as well? The four new zonecfg resources are: dedicated-cpu capped-cpu (future, after cpu-caps are integrated) dedicated-memory (future, after memory sets are integrated) capped-memory Each of these zonecfg resources will have properties that are appropriate to the RM capabilities associated with that resource. Zonecfg will only allow one instance of each these resource to be configured and it will not allow conflicting resources to be added (e.g. dedicated-cpu and capped-cpu are mutually exclusive). The mapping of these new zonecfg resources to the underlying RM feature is: dedicated-cpu -> temporary pset dedicated-memory -> temporary mset capped-cpu -> cpu-cap rctl [14] capped-memory -> rcapd running in global zone Temporary psets and msets are described below, in section 2. Rcapd enhancements for running in the global zone are described below in section 4. The valid properties for each of these new zonecfg resources will be: dedicated-cpu ncpus importance capped-cpu ncpus dedicated-memory physical virtual importance capped-memory physical virtual The meaning of each of these properties is as follows: dedicated-cpu ncpus: This can be a positive integer or range. A value of '2' means two cpus, a value of '2-4' means a range of two to four cpus. This sets the 'pset.min' and 'pset.max' properties on the temporary pset. importance: This property is optional. It can be a positive integer. It sets the 'pool.importance' property on the temporary pool. capped-cpu This resource group and its property will not be delivered as part of this project since cpu-caps are still under development. However, our thinking on this is described here for completeness. ncpus: This is a positive decimal. The 'ncpus' property actually maps to the zone.cpu-cap rctl. This property will be implemented as a special case of the new zones rctl aliases which are described below in section 3. The special case handling of this property will normalize the value so that it corresponds to units of cpus and is similar to the 'ncpus' property under the dedicated-cpu resource group. However, it won't accept a range and it will accept a decimal number. For example, when using 'ncpus' in the dedicated-cpu resource group, a value of 1 means one dedicated cpu. When using 'ncpus' in the capped-cpu resource group, a value of 1 means 100% of a cpu is the cap setting. A value of 1.25 means 125%, since 100% corresponds to one full cpu on the system when using cpu caps. The idea here is to align
Re: [zones-discuss] Re: improved zones/RM integration
Nils, Thanks for looking this over. Nils Nieuwejaar wrote: On Wed 08/16/06 at 09:10 AM, [EMAIL PROTECTED] wrote: [...] dedicated-memory [...] physical: A positive decimal number or a range with a required [...] virtual: This accepts the same numbers as the 'physical' property. This will set the 'mset.minswap' and 'mset.maxswap' properties on the temporary mset. Why not just call this 'swap'? 'virtual' memory has an established definition that does not match this usage. With this naming, I guarantee we are going to get calls from confused customers wanting to know why firefox has something mapped at 0xC00 when they set a 'virtual memory' limit of 1GB. We can do that. I think we picked virtual when Steve and I were talking about this. This is not in phase 1 of the project so we haven't spent as much time on it. Let me run this past Steve but we'll probably go ahead and update the proposal to use swap. 4) Enable rcapd to limit zone memory while running in the global zone [9] Currently, to use rcapd(1M) to limit zone memory consumption, the rcapd process must be run within the zone. While useful in some configurations, in situations where the zone administrator is untrusted, this is ineffective, since the zone administrator could simply change the rcapd limit. We will enhance rcapd so that it can limit each zone's memory consumption while it is running in the global zone. This closes the rcapd loophole described above and allows the global zone administrator to set memory caps that can be enforced by a single, trusted process. Currently rcapd will not run in a Linux zone. This change will also allow you to apply rcapd limits to branded zones. Good point. I will add that to the description. Thanks again, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Re: improved zones/RM integration
On Wed 08/16/06 at 09:10 AM, [EMAIL PROTECTED] wrote: [...] > dedicated-memory [...] > physical: A positive decimal number or a range with a required [...] > virtual: This accepts the same numbers as the 'physical' > property. This will set the 'mset.minswap' and > 'mset.maxswap' properties on the temporary mset. Why not just call this 'swap'? 'virtual' memory has an established definition that does not match this usage. With this naming, I guarantee we are going to get calls from confused customers wanting to know why firefox has something mapped at 0xC00 when they set a 'virtual memory' limit of 1GB. > 4) Enable rcapd to limit zone memory while running in the global zone [9] > > Currently, to use rcapd(1M) to limit zone memory consumption, the > rcapd process must be run within the zone. While useful in some > configurations, in situations where the zone administrator is > untrusted, this is ineffective, since the zone administrator could > simply change the rcapd limit. > > We will enhance rcapd so that it can limit each zone's memory > consumption while it is running in the global zone. This closes the > rcapd loophole described above and allows the global zone administrator > to set memory caps that can be enforced by a single, trusted process. Currently rcapd will not run in a Linux zone. This change will also allow you to apply rcapd limits to branded zones. ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] Re: improved zones/RM integration
I received a lot of good feedback from the community when I sent out the first draft of the proposal for improved zones/RM integration. Based on this feedback I have made several changes. Attached is a new draft of the proposal we will be going forward with. The major changes we made are: * rctl aliases will only be top-level properties in zonecfg * enhance zones to manage the global zones RM configuration * new zonecfg clear subcommand and updated remove behavior * new zonecfg scheduling-class property * new options for rcapadm and rcapstat The details of these changes are in the attached proposal. I also added more details throughout the proposal. Please send me any comments or questions, Jerry This message posted from opensolaris.orgSUMMARY: This project enhances zones[1], pools[2-4] and resource caps[5,6] to improve the integration of zones with resource management (RM). It addresses existing RFEs[7-12] in this area and lays the groundwork for simplified, coherent management of the various RM features exposed through zones. These enhancements are targeted at "less experienced" users who are unfamiliar with all of the details and capabilities of Solaris RM. For users who need more complex RM configurations, the existing commands and procedures must still be used. However, we do feel that these enhancements will meet the needs of a large percentage of zones users. We will integrate some basic pool configuration with zones, implement the concept of "temporary pools" that are dynamically created/destroyed when a zone boots/halts and we will simplify the setting of resource controls within zonecfg. We will enhance rcapd so that it can cap a zone's memory while rcapd is running in the global zone. We will enable persistent RM configuration for the global zone. We will also make a few other changes to provide a better overall experience when using zones with RM. Patch binding is requested for these new interfaces and the stability of most of these interfaces is "Committed" (see interface table for complete list). PROBLEM: Although zones are fairly easy to configure and install, it appears that many users have difficulty setting up a good RM configuration to accompany their zone configuration. Understanding RM involves many new terms and concepts along with lots of documentation to understand. This leads to the problem that many customers either do not configure RM with their zones, or configure it incorrectly, leading them to be disappointed when zones, by themselves, do not provide all of the containment that they expect. This problem will just get worse in the near future with the additional RM features that are coming, such as cpu-caps[14], memory sets[15] and swap sets[16]. PROPOSAL: There are a set of relatively independent enhancements outlined below which will, taken together, address these problems and provide a simple, tightly integrated experience for configuring containers (zones with RM). This proposal is complicated by the fact that it tries to unify the RM concepts within zones, but not all of these RM features are available yet. Specifically, cpu-caps[14], memory sets[15], swap sets[16], and new rctls [17, 18] are under development but not yet integrated. This proposal will describe a framework for how current and future RM features will be integrated with zones. However, we do not want to make this project dependent on these future projects since there is a lot of value to be added as is. Instead, we will submit fast-tracks to incorporate future RM features as they integrate. Items related to these future enhancements are noted within the body of the proposal. The future enhancements are also noted in the interface table for follow-on phases of this overall project. 1) "Hard" vs. "Soft" RM configuration within zonecfg We will enhance zonecfg(1M) so that the user can configure basic RM capabilities in a structured way. Various existing and upcoming RM features can be broken down into "hard" vs. "soft" partitioning of the system's resources. With "hard" partitioning, resources are dedicated to the zone using processor sets (psets) and memory sets (msets). With "soft" partitioning, resources are shared, but capped, with an upper limit on their use by the zone. Hard|Soft - cpu| psets | cpu-caps memory | msets | rcapd Within zonecfg we will organize these various RM features into four
Re: [zones-discuss] Re: improved zones/RM integration
Dan Price wrote: Just FYI, please thoroughly digest PSARC/2004/471 before recommending the use of -u or considering documenting it. In short, your steps should work without needing the -u option, since -d causes an update to both the persistent file-based setting and to the kernel: # dispadmin -d FSS # priocntl -s -c FSS -i all # priocntl -s -c FSS -i pid 1 Dan, Yes, I had looked at this a bit more after I sent the email and realized that -u was unnecessary. Thanks, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Re: improved zones/RM integration
On Tue 27 Jun 2006 at 08:27AM, Jerry Jelinek wrote: > Mike Gerdts wrote: > >>I also like the idea of activating FSS if the zone.cpu-shares > >>parameter is set. Given that activating FSS requires a reboot of the > >>global zone, you should add a check to see if FSS is already active > >>and if it is not, put out a warning message indicating that a reboot > >>of the global is required to make FSS active. For that matter, there > >>should be a warning message on zone boot that FSS is not active and > >>the configured zone.cpu-shares value will be ignored. > > > >No reboot is required. See poolbind in this example: > >http://users.tpg.com.au/adsln4yb/zones.html#resource_cpu1 > > Phil, > > Mike as already responded with a bunch of useful information. > However, I looked at this url and on this one point regarding setting > FSS as the default, it doesn't look like it actually shows you how to > do this. > > You don't have to reboot to make FSS the default and have all > of the global zones processes running under FSS. Instead, you can > do something like this: > > # dispadmin -d FSS > # dispadmin -u > # priocntl -s -c FSS -i all > # priocntl -s -c FSS -i pid 1 > > I see the -u option on dispadmin is undocumented. I'll have to > look at that and see if it might make sense to raise its stability. Jerry, Just FYI, please thoroughly digest PSARC/2004/471 before recommending the use of -u or considering documenting it. In short, your steps should work without needing the -u option, since -d causes an update to both the persistent file-based setting and to the kernel: # dispadmin -d FSS # priocntl -s -c FSS -i all # priocntl -s -c FSS -i pid 1 -dp -- Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - blogs.sun.com/dp ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Re: improved zones/RM integration
Jerry Jelinek wrote: You don't have to reboot to make FSS the default and have all of the global zones processes running under FSS. Instead, you can do something like this: # dispadmin -d FSS # dispadmin -u # priocntl -s -c FSS -i all # priocntl -s -c FSS -i pid 1 Since snv_29, you can also restart the scheduler service # dispadmin -d FSS # svcadm restart system/scheduler -- Renaud ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Re: improved zones/RM integration
Mike Gerdts wrote: I also like the idea of activating FSS if the zone.cpu-shares parameter is set. Given that activating FSS requires a reboot of the global zone, you should add a check to see if FSS is already active and if it is not, put out a warning message indicating that a reboot of the global is required to make FSS active. For that matter, there should be a warning message on zone boot that FSS is not active and the configured zone.cpu-shares value will be ignored. No reboot is required. See poolbind in this example: http://users.tpg.com.au/adsln4yb/zones.html#resource_cpu1 Phil, Mike as already responded with a bunch of useful information. However, I looked at this url and on this one point regarding setting FSS as the default, it doesn't look like it actually shows you how to do this. You don't have to reboot to make FSS the default and have all of the global zones processes running under FSS. Instead, you can do something like this: # dispadmin -d FSS # dispadmin -u # priocntl -s -c FSS -i all # priocntl -s -c FSS -i pid 1 I see the -u option on dispadmin is undocumented. I'll have to look at that and see if it might make sense to raise its stability. In the meantime, be aware that it could change at any time, although that is not very likely. Also, just to clarify things a bit in regards to our proposal, if you have a zone with cpu-shares set, we won't be changing the global zones scheduling class to be FSS. While that would be useful for many scenarios, it will change the behavior of the system and it might not be what you want in all cases. Instead, we will be setting up the zone so that it is using FSS for all of its processes but the processes in the global zone will continue to run with whatever scheduling class they are using. For a lot of cases with cpu-shares, it would probably be a good idea to run with FSS as the default but we can't just make this change to the system automatically. Thanks, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Re: improved zones/RM integration
On 6/26/06, Phil Freund <[EMAIL PROTECTED]> wrote: I'm just trying to get my head around how to setup the whole pool/resource capping arena for my server/zone configurations and am finding it a bit confusing and arcane. Your proposal looks pretty good to me since most of what I need would be covered by the defaults. It's certainly more understandable. Unfortunately, I have to go ahead with what's available now and will have to retrofit the changes based on this proposal if/when they get implemented. I found Brendan Gregg's page http://users.tpg.com.au/adsln4yb/zones.html very helpful. He condensed dozens (hundreds?) of pages down to something digestable in a few minutes. One suggestion that may only be tangential to this effort is that a mechanism be put in place to allow the global zone sysadmin to set the default pool and resource values (such as cpu shares) for the global zone without having to write a script to set them on boot. It doesn't look particularly hard but it is one more thing to maintain that really should be handled by the OS. I have created a replacement for svc:/system/zones and its associated /lib/svc/method/* script to activate the appropriate pools during zone booting. It does other things too - adds a default router if necessary, allows an override for zone shutdown timeout, makes sure that the zone's IP address isn't pingable, etc. You may want to consider going this route if you have a similar set of concerns. I also like the idea of activating FSS if the zone.cpu-shares parameter is set. Given that activating FSS requires a reboot of the global zone, you should add a check to see if FSS is already active and if it is not, put out a warning message indicating that a reboot of the global is required to make FSS active. For that matter, there should be a warning message on zone boot that FSS is not active and the configured zone.cpu-shares value will be ignored. No reboot is required. See poolbind in this example: http://users.tpg.com.au/adsln4yb/zones.html#resource_cpu1 Mike -- Mike Gerdts http://mgerdts.blogspot.com/ ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] Re: improved zones/RM integration
I'm just trying to get my head around how to setup the whole pool/resource capping arena for my server/zone configurations and am finding it a bit confusing and arcane. Your proposal looks pretty good to me since most of what I need would be covered by the defaults. It's certainly more understandable. Unfortunately, I have to go ahead with what's available now and will have to retrofit the changes based on this proposal if/when they get implemented. Like many others, my biggest current need is to meet licensing restrictions on the number of CPUs. At the moment this is mostly limited to being needed in the global zone since I have to use Oracle 8i with Quick I/O on a licensed CPU limited basis from there; in the future though, I will be able to move to Oracle 10g hosted in a nonglobal zone so the concepts of the temporary pools in the proposal look pretty good. I can also see a big use for the swap sets and memory sets on a zone basis. Overall, simplicity is good. Yet another GUI certainly would not be good. That being said, management of the proposed changes should be possible in Container Manager in SMC. One suggestion that may only be tangential to this effort is that a mechanism be put in place to allow the global zone sysadmin to set the default pool and resource values (such as cpu shares) for the global zone without having to write a script to set them on boot. It doesn't look particularly hard but it is one more thing to maintain that really should be handled by the OS. I also like the idea of activating FSS if the zone.cpu-shares parameter is set. Given that activating FSS requires a reboot of the global zone, you should add a check to see if FSS is already active and if it is not, put out a warning message indicating that a reboot of the global is required to make FSS active. For that matter, there should be a warning message on zone boot that FSS is not active and the configured zone.cpu-shares value will be ignored. Phil Phil Freund Lead Systems and Storage Administrator Kichler Lighting This message posted from opensolaris.org ___ zones-discuss mailing list zones-discuss@opensolaris.org
