Re: [zones-discuss] Default RM controls for Containers?
Mads Toftum wrote: On Fri, May 11, 2007 at 01:44:42PM -0400, Jeff Victor wrote: I would choose 50%. For 3 zones, 75% doesn't accomplish enough. At 50%, they will (hopefully) investigate the performance issue and be happily surprised when they learn they've been using a default value... I'm not too keen to have defaults that could affect performance on systems running a normal load. As long as it only gets enabled when you ask for default RM, then I'm not too worried. Here we have a difficult non-technical decision to make. Which is 'better': 1) No out-of-the-box controls - the current situation. The unsuspecting zone creator will unwittingly allow DoS attacks by zones until it becomes clear that RM controls should be used, either through education or a negative experience. Possible solutions to this include A) One enable-RM knob which applies defaults that can be overridden B) Templates that have default RM controls C) Others 2) Out-of-the-box controls: all zones have default RM controls unless the creator overrides those controls. These values would be generous enough to prevent DoS attacks and the effects of very badly written software, but not affect most workloads, as Mads suggests. Templates could also be added to enable simple RM tuning. To me, (1) doesn't solve the problem that I think we need to solve. I am trying to protect the first-time and occasional zones creators. As Jerry said, RM is a requirement for zones. At the same time, I am trying to minimize any impact on normal workloads. By default zones provide an extremely robust security boundary to protect them from each other. Why do they not also provide some default minimum RM isolation, for the same reason? If we have consensus on the basic idea of out-of-the-box defaults, I think I have seen enough on this thread to draft specifics - modifications to the UI, what the defaults would be, etc. Are we ready for that yet? Further, if that concept gets far enough, I would like to implement the changes as an OpenSolaris community member. I would need some guidance on the process, but can handle the code work. -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Menno Lageman wrote: Another option for RM templates would be that the template is a pointer to a set of RM defaults instead of being used directly during zone creation. This way, changing RM settings of existing zones would simply entail changing the template in one place. Or, when moving a zone to another class of RM defaults, by changing the template reference in that zone (i.e. from SUNWsmall to SUNWmedium). Also, regardless of the model we use, any defaults should not be based on the hardware configuration of the system when the zone is first installed. After the zone is moved to a different system, defaults should be calculated when the zone boots. If default caps are in use, maybe the zone should re-calculate caps at boot time. Otherwise, caps chosen when the zone was first created could be very inappropriate, perhaps completely ineffective, after the zone is moved. This has the potential to surprise someone who doesn't expect re-calculation, but is better than incapacitating the entire concept of default caps. -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Jeff Victor wrote:
Here we have a difficult non-technical decision to make. Which is
'better':
1) No out-of-the-box controls - the current situation. The
unsuspecting zone creator will unwittingly allow DoS attacks by zones
until it becomes clear that RM controls should be used, either through
education or a negative experience. Possible solutions to this include
A) One enable-RM knob which applies defaults that can be overridden
B) Templates that have default RM controls
C) Others
2) Out-of-the-box controls: all zones have default RM controls unless
the creator overrides those controls. These values would be generous
enough to prevent DoS attacks and the effects of very badly written
software, but not affect most workloads, as Mads suggests. Templates
could also be added to enable simple RM tuning.
On the premise that we're trying to give the regular[1] Zones user a
good, default RM setup, I'd vote for option 2 ('safe' OOB controls).
Experienced users that have more insight into what good values for their
zones should be, can override these defaults if needed. Which of course
leads to the question what the default out-of-the-box values should be.
This might be the hardest part.
Menno
[1] someone who has no in-depth knowledge of/experience with Zones and
Resource Management and just needs a zone to run his applications in.
--
Menno Lageman - Sun Microsystems - http://blogs.sun.com/menno
___
zones-discuss mailing list
zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Menno Lageman wrote:
Jeff Victor wrote:
Here we have a difficult non-technical decision to make. Which is
'better':
1) No out-of-the-box controls - the current situation. The
unsuspecting zone creator will unwittingly allow DoS attacks by zones
until it becomes clear that RM controls should be used, either through
education or a negative experience. Possible solutions to this include
A) One enable-RM knob which applies defaults that can be overridden
B) Templates that have default RM controls
C) Others
2) Out-of-the-box controls: all zones have default RM controls unless
the creator overrides those controls. These values would be generous
enough to prevent DoS attacks and the effects of very badly written
software, but not affect most workloads, as Mads suggests. Templates
could also be added to enable simple RM tuning.
On the premise that we're trying to give the regular[1] Zones user a
good, default RM setup, I'd vote for option 2 ('safe' OOB controls).
Experienced users that have more insight into what good values for their
zones should be, can override these defaults if needed. Which of course
leads to the question what the default out-of-the-box values should be.
This might be the hardest part.
I think that an appropriate fraction of the resource which exists at zone-boot
time is a good start. Agreeing to a specific fraction will, undoubtedly,
involve a great deal of arm-wrestling among us... ;-)
Menno
[1] someone who has no in-depth knowledge of/experience with Zones and
Resource Management and just needs a zone to run his applications in.
--
Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com
OS AmbassadorSr. Technical Specialist
Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq
--
___
zones-discuss mailing list
zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Fri, May 11, 2007 at 11:37:03AM -0600, Jerry Jelinek wrote: Can you explain your concern? What if we fixed FSS so it works when you are running the windowing system (like IA)? That's not the point here. FSS shares being relative to the total number of shares. So, if you were to have 2 zones and give them 1 share each, it would be the same effect as handing them 10 each. My point is that if you only give them one share each and later want to add a low priority zone, then you'd effectively have to hand more shares to the original zones. If you were to default to 10 shares each and later want to add a low priority zone, then you just give the new zone one share and leave the others as they were. vh Mads Toftum -- http://soulfood.dk ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Fri, May 11, 2007 at 01:44:42PM -0400, Jeff Victor wrote: I would choose 50%. For 3 zones, 75% doesn't accomplish enough. At 50%, they will (hopefully) investigate the performance issue and be happily surprised when they learn they've been using a default value... I'm not too keen to have defaults that could affect performance on systems running a normal load. As long as it only gets enabled when you ask for default RM, then I'm not too worried. vh Mads Toftum -- http://soulfood.dk ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Jerry Jelinek wrote: Dan Price wrote: On Thu 10 May 2007 at 04:21PM, Jerry Jelinek wrote: of the other controls is trickier although I think Dan's idea of scaling these based on the system makes it easier. We might also want to think about scaling based on the number of running zones. Another way to look at it (and I think what you are saying) would be to broaden the notion of shares a bit to include more of the system resources-- for example, memory. What's tough there, though, is that our notion of shares today represent an entitlement, and the case of memory, we're talking about a cap on utilization. I think fundamentally we hear from two camps: those who want to proportionally partition whatever resources are available, and those who want to see the system as virtual 512MB Ultra-2's or virtual 1GB, 1ghz PCs. Dan, Yes, something like shares for memory would be nice because you don't have to know ahead of time what your maximum will be and as long as the system is not overcommitted you can use what you need. I was thinking down the same path - I was calling it Fair Memory Shares - but we must advise caution in usage of such a feature. CPU cycles are infinite, memory is not. When using zones with default cpu-shares and at least a few zones, adding another zone will decrease the minimum per-zone CPU utilization only slightly. Regardless of RM usage, booting an additional zone may increase RAM usage sufficiently to cause enough paging that performance of all apps will suffer noticeably. Using FMS won't help that, and we shouldn't let users think that it will. Another difference between CPU and memory RM is that workloads need a minimum amount of VM to operate at all, a minimum number of threads to operate, and a minimum amount of RAM to operate with acceptable performance. Those are all very different from CPU mgmt. With all of that, should default values be minima or maxima? The goal I have in mind is default values that will protect a zone from DoS attacks, or the equivalent symptom, caused by bad software. Although we could assign default values to caps, they would be arbitrary, and would need to be so large that they would often be largely ineffective. On the other hand, they would be easy to achieve. Even *I* could implement them... ;-) Perhaps it would be more effective to have default minima, similar to the default of 1 FSS share that the global zone has had all along. Starting with Mads' Enable-RM knob, we could probably assign reasonable default minima, maybe: * RAM: 200MB? 500MB? * VM: 200MB? 500MB? * lwps: 1000 The last one tripped up the Clarkson team - they started with max-lwps=50, and then didn't understand why the zone wouldn't boot. I gave them a couple of pointers, and suggested 175, which worked for them. There's little sense in setting that to a low number - Solaris will create more than 70,000 threads before it begins to run out of a resource. What we really want is an lwp-creation-rate meter. But max-lwps is much simpler, and still effective. Another idea: when you Enable-RM, suggest a default value and ask for confirmation or a different value. A wizard would be very useful here, because it could do a quick analysis of the current system and its workloads, and in some cases be *very* helpful in telling the user if you boot this zone in addition to the currently running zones, performance degradation will result. That's the polite version of excuse me, but you can't stuff 10 kilos of dirt in a 5-kilo bag. Back to default minima vs. default maxima: I doubt that implementing default minima would be simple, and my hope is to get something in place before the end of 2008, perhaps as early as S10 U6. Although simple default caps for existing controls could be ready for U5, if allowed... Apologies for all the rambling. More below. I agree that there are multiple ways people want to slice things up and we are actually pretty good with the capped and dedicated stuff now. It is the full sharing with a guaranteed minimum that we might want to think about improving (for memory). I'm not sure how hard that will be though. Just thinking out loud here, I wonder if there is any way we could come close to this behavior by dynamically adjusting the physical and swap controls we already have, based upon how many zones are running? This carries the danger of pulling the rug out from under the zone. Well-behaving zone one minute, waste of memory the next. :-) It wouldn't be as good as fair shares for memory but it would be a lot easier to implement. Or, maybe we could use rcapd to watch the dynamic behavior of each zone and adjust the physical cap as needed. That would be really easy to implement. I like dynamically tunable systems - if the heuristics are good enough. The 'importance' value in resource pools is simple but works. Something similar could be used. I need to think about that one... It is
Re: [zones-discuss] Default RM controls for Containers?
Jeff Victor wrote: With all of that, should default values be minima or maxima? The goal I have in mind is default values that will protect a zone from DoS attacks, or the equivalent symptom, caused by bad software. Although we could assign default values to caps, they would be arbitrary, and would need to be so large that they would often be largely ineffective. On the other hand, they would be easy to achieve. Even *I* could implement them... ;-) The requirement for the RM defaults should be that a misbehaving zone can't effectively bring down the whole system. You want to be able to get on the global zone and clean up the misbehaving zone and any other well behaved non-global zones should still be able to do work. Given that, having FSS on by default makes sense. Each zone will have 1 share by default, so thats fine. What if max-lwps defaulted to a fairly large number (5000)? How often would this be an issue for a well-behaved zone? If we implement Dan's idea of a percentage for some of the resource controls we could have physical memory and swap caps default to something like 50%-75% of the system total. Again, well-behaved zones shouldn't get close to this (if they do, the system is probably undersized to begin with) but we can keep a misbehaving zone in check. If we had these 4 things we should be pretty solid right out of the box. Would this be a good start on this problem? Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Fri, May 11, 2007 at 10:48:04AM -0600, Jerry Jelinek wrote: The requirement for the RM defaults should be that a misbehaving zone can't effectively bring down the whole system. You want to be able to get on the global zone and clean up the misbehaving zone and any other well behaved non-global zones should still be able to do work. Given that, having FSS on by default makes sense. Each zone will have 1 share by default, so thats fine. I'm not too keen to have it on by default, but if you're going to then 1 share each makes sense. Or perhaps something like 10 shares each making it simple to sneak in a low priority zone without having to edit all zones. What if max-lwps defaulted to a fairly large number (5000)? How often would this be an issue for a well-behaved zone? Sounds fairly reasonable and as something where people won't often hit the limit - in fact I'd almost be tempted to go lower and small configurations. If we implement Dan's idea of a percentage for some of the resource controls we could have physical memory and swap caps default to something like 50%-75% of the system total. Again, well-behaved zones shouldn't get close to this (if they do, the system is probably undersized to begin with) but we can keep a misbehaving zone in check. Wouldn't this lead to a waste of resources on systems with only one non-global zone? It may not be the most common setup, but still makes a lot of sense for a higher level of security. vh Mads Toftum -- http://soulfood.dk ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Mads Toftum wrote: On Fri, May 11, 2007 at 10:48:04AM -0600, Jerry Jelinek wrote: The requirement for the RM defaults should be that a misbehaving zone can't effectively bring down the whole system. You want to be able to get on the global zone and clean up the misbehaving zone and any other well behaved non-global zones should still be able to do work. Given that, having FSS on by default makes sense. Each zone will have 1 share by default, so thats fine. I'm not too keen to have it on by default, but if you're going to then 1 share each makes sense. Or perhaps something like 10 shares each making it simple to sneak in a low priority zone without having to edit all zones. Except for Jerry's comment earlier about impacting GUIs that currently use IA, I agree. What if max-lwps defaulted to a fairly large number (5000)? How often would this be an issue for a well-behaved zone? Sounds fairly reasonable and as something where people won't often hit the limit - in fact I'd almost be tempted to go lower and small configurations. Perhaps the default would be one of two or three values, depending on number of CPUs on the system (as Solaris counts them, not necessarily sockets or cores). The values could be chosen to meet a very rough quantity of threads per CPU which is similar across all size. This would be similar to the model used for memory (below): Small = 1-2 CPUs: 750 Medium = 3-8 CPUs: 2500 Large = 9+ CPUs: 5000 Those are in the range 400-500 lwps per CPU. we could have physical memory and swap caps default to something like 50%-75% of the system total. Again, well-behaved zones shouldn't get close to this (if they do, the system is probably undersized to begin with) but we can keep a misbehaving zone in check. Wouldn't this lead to a waste of resources on systems with only one non-global zone? It may not be the most common setup, but still makes a lot of sense for a higher level of security. The model sounds good. It *is* a default, meant for people who don't know (or don't want to know) about these things. Other than prompting the zone creator will this be the only zone on the system) I don't see a way around this. I would choose 50%. For 3 zones, 75% doesn't accomplish enough. At 50%, they will (hopefully) investigate the performance issue and be happily surprised when they learn they've been using a default value... -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Mads Toftum wrote: On Fri, May 11, 2007 at 10:48:04AM -0600, Jerry Jelinek wrote: The requirement for the RM defaults should be that a misbehaving zone can't effectively bring down the whole system. You want to be able to get on the global zone and clean up the misbehaving zone and any other well behaved non-global zones should still be able to do work. Given that, having FSS on by default makes sense. Each zone will have 1 share by default, so thats fine. I'm not too keen to have it on by default, but if you're going to then 1 share each makes sense. Or perhaps something like 10 shares each making it simple to sneak in a low priority zone without having to edit all zones. Can you explain your concern? What if we fixed FSS so it works when you are running the windowing system (like IA)? What if max-lwps defaulted to a fairly large number (5000)? How often would this be an issue for a well-behaved zone? Sounds fairly reasonable and as something where people won't often hit the limit - in fact I'd almost be tempted to go lower and small configurations. You could always do that, this would just be a default. If we implement Dan's idea of a percentage for some of the resource controls we could have physical memory and swap caps default to something like 50%-75% of the system total. Again, well-behaved zones shouldn't get close to this (if they do, the system is probably undersized to begin with) but we can keep a misbehaving zone in check. Wouldn't this lead to a waste of resources on systems with only one non-global zone? It may not be the most common setup, but still makes a lot of sense for a higher level of security. No, since this is only a cap, not a partitioning of resources, so everything is still shared. Thanks, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Jerry Jelinek wrote: Mads Toftum wrote: If we implement Dan's idea of a percentage for some of the resource controls we could have physical memory and swap caps default to something like 50%-75% of the system total. Again, well-behaved zones shouldn't get close to this (if they do, the system is probably undersized to begin with) but we can keep a misbehaving zone in check. Wouldn't this lead to a waste of resources on systems with only one non-global zone? It may not be the most common setup, but still makes a lot of sense for a higher level of security. No, since this is only a cap, not a partitioning of resources, so everything is still shared. Because it's a cap, if it's 50% then a single zone that wants to use 6GB of an 8GB system can't. Even if the global zone (incl kernel) uses 1GB, 1GB is wasted. -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Jeff Victor wrote: Wouldn't this lead to a waste of resources on systems with only one non-global zone? It may not be the most common setup, but still makes a lot of sense for a higher level of security. No, since this is only a cap, not a partitioning of resources, so everything is still shared. Because it's a cap, if it's 50% then a single zone that wants to use 6GB of an 8GB system can't. Even if the global zone (incl kernel) uses 1GB, 1GB is wasted. Got it. I misunderstood the first email. Yes, in this case you would need to change the default. Sorry, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
[zones-discuss] Default RM controls for Containers?
By default, Solaris Containers do not have resource controls. Up through S10 11/06 you could add many resource controls to Containers, directly or indirectly, but some of them were... 'challenging' to use. ;-) S10 7/07 improves the situation greatly, moving many of the 'indirect' controls (e.g. physical memory capping) into the 'direct' category. In doing that, it also made them much easier to use. But default settings are still absent. This was clearly demonstrated in a recent research paper at Clarkson University. They compared resource isolation of 4 different v12n solutions: Vmware Workstation, Xen, OpenVZ and Containers. I did a quick summary of the Containers conclusions: http://blogs.sun.com/JeffV/date/20070510 . That blog has a link to the paper, too. I would like to gather thoughts and opinions on this omission: should Containers have default RM settings? Is there a better method to solve this problem? If not, which settings should have defaults? It might make sense to use FSS for all zones, but some work may be necessary to avoid creating new problems. If that can be done, assigning a default of 1 share per zone would make sense. A reasonably large default value for physical capped-memory might be valuable, but might cause its own problems, e.g. more support calls: I have 1GB of freemem, but the system is paging! Why?!? Etc. Thoughts? -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Jeff Victor wrote: By default, Solaris Containers do not have resource controls. Up through S10 11/06 you could add many resource controls to Containers, directly or indirectly, but some of them were... 'challenging' to use. ;-) S10 7/07 improves the situation greatly, moving many of the 'indirect' controls (e.g. physical memory capping) into the 'direct' category. In doing that, it also made them much easier to use. But default settings are still absent. This was clearly demonstrated in a recent research paper at Clarkson University. They compared resource isolation of 4 different v12n solutions: Vmware Workstation, Xen, OpenVZ and Containers. I did a quick summary of the Containers conclusions: http://blogs.sun.com/JeffV/date/20070510 . That blog has a link to the paper, too. I would like to gather thoughts and opinions on this omission: should Containers have default RM settings? Is there a better method to solve this problem? If not, which settings should have defaults? It might make sense to use FSS for all zones, but some work may be necessary to avoid creating new problems. If that can be done, assigning a default of 1 share per zone would make sense. A reasonably large default value for physical capped-memory might be valuable, but might cause its own problems, e.g. more support calls: I have 1GB of freemem, but the system is paging! Why?!? Etc. Thoughts? When we were talking about this problem a year or so ago we first thought that the idea of zone templates would be a good way to solve this problem. This is: 6409152 RFE: template support for better RM integration The idea we had was that when you initially create your zone you would do so from one of a set of pre-configured templates. We would deliver various templates that had good settings for the various RM controls. At the time this idea didn't fly because we didn't have enough capabilities in zonecfg to do much real work with resource management (we had two rctls and the pool property). Things are much better now that we have the duckhorn project integrated. Templates would be very easy to implement but maybe there are some other, better ideas where you don't even need to make a choice. After we did duckhorn Dan had the idea of being able to set some of the controls as percentages. This is: 6543082 zonecfg should allow specification of some resources as percentage of system total I think if we implemented that then maybe we could just have some defaults for those controls that would automatically be set when you first create your zone. For FSS, zones already default to 1 share if you don't explicitly set the rctl. I would really like to make FSS the system default scheduling class if you are running zones. However, we do need to add some of the IA support into FSS if we want to do that. I don't think we have a RFE open for that. I'm curious to hear what other folks think, Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Thu, May 10, 2007 at 11:23:18AM -0400, Jeff Victor wrote: I would like to gather thoughts and opinions on this omission: should Containers have default RM settings? Is there a better method to solve this problem? If not, which settings should have defaults? I really wouldn't like having RM enabled by default on zones as I think it would create more confusion and annoyance than is fair. That being said, I think that once you enable RM on your zone, it should choose sensible settings for you until the point where you choose to override them. Perhaps it could even be as simple as set rctl=FSS to enable RM. At that point it would be fine to pull in reasonable defaults. To make matters simple, I think populating defaults is only worth doing for FSS. vh Mads Toftum -- http://soulfood.dk ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Mads Toftum wrote: On Thu, May 10, 2007 at 11:23:18AM -0400, Jeff Victor wrote: I would like to gather thoughts and opinions on this omission: should Containers have default RM settings? Is there a better method to solve this problem? If not, which settings should have defaults? I really wouldn't like having RM enabled by default on zones as I think it would create more confusion and annoyance than is fair. That being said, I think that once you enable RM on your zone, it should choose sensible settings for you until the point where you choose to override them. Currently there isn't a setting which enables (or disables) RM. Are you suggesting that there should be one 'knob' which enables RM, and chooses sufficiently large default values until you override them? I think that such a situation would be much better than what we currently have. The default values could be ones that no reasonable workload should exceed, and could be based on the hardware resources available. For example, the physical memory cap could be set to 50% (or 70%) of available RAM, perhaps after subtracting a reasonable amount for the kernel. A similar method could be used to determine a default for a cap on VM. However, this model does not solve the problem that is documented in Clarkson's paper: the out-of-the-box experience does not protect well-behaved zones from poorly-behaved zones, or a DoS attack. Perhaps it could even be as simple as set rctl=FSS to enable RM. At that point it would be fine to pull in reasonable defaults. To make matters simple, I think populating defaults is only worth doing for FSS. I am not certain I understand: are you saying that it doesn't make sense to cap the amount of swap space that a zone can use unless you are also using FSS? That is not true in some situations. For example, each container might be in a separate resource pool, using its own CPUs. FSS would not accomplish anything. -- Jeff VICTOR Sun Microsystemsjeff.victor @ sun.com OS AmbassadorSr. Technical Specialist Solaris 10 Zones FAQ:http://www.opensolaris.org/os/community/zones/faq -- ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Thu, May 10, 2007 at 02:11:12PM -0400, Jeff Victor wrote: Currently there isn't a setting which enables (or disables) RM. Are you suggesting that there should be one 'knob' which enables RM, and chooses sufficiently large default values until you override them? Yes. Perhaps it could even be as simple as set rctl=FSS to enable RM. At that point it would be fine to pull in reasonable defaults. To make matters simple, I think populating defaults is only worth doing for FSS. I am not certain I understand: are you saying that it doesn't make sense to cap the amount of swap space that a zone can use unless you are also using FSS? No, I was just thinking that while handing out cpu shares with FSS is very simple because you could easily just give 10 shares to each new zone and never worry about it, but if you have to give a specific percentage of system resources, then it becomes much harder because you'd have to either know the total number of zones up front or adjust over time. That is not true in some situations. For example, each container might be in a separate resource pool, using its own CPUs. FSS would not accomplish anything. This is a situation where someone has had to take the time to decide a need for a more complex RM setup - in that case I think it would be fair to let them define the full RM setting rather than trying to second guess their reasoning. As in - if you want to play with the advanced features, then you're on your own. vh Mads Toftum -- http://soulfood.dk ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Thu, 2007-05-10 at 14:11 -0400, Jeff Victor wrote: However, this model does not solve the problem that is documented in Clarkson's paper: the out-of-the-box experience does not protect well-behaved zones from poorly-behaved zones, or a DoS attack. I see where you are going with this Jeff, and there are some good ideas behind all of this. I have a great desire to rephrase your question without the reference to zones - how well is Solaris itself protected against the various forms of DoS attack ? Do the controls here suggest rational defaults for zones (ie, should we just inherit the limits/protections from the Solaris parent) ? One area where I struggle on this issue - you have to decide between two different corner cases (both from situations where the person isn't committed to the documentation): would I rather deal with a problem that an application dies for no apparent reason or that DoS situations can happen ? They are both corner cases right out of the Clarkson paper. In the first case, setting default limits could cause apps to throttle or perhaps fail when reaching their resource cap limits. In the next Clarkson paper :-) this will lead to the assumption that Solaris is either slow or unstable - of which neither is true. So we have to explain where the resource controls are, how to tune them, etc. Reminds me of when we used to play with lotsfree and handspread. In the second case, unmanaged workloads (which are simple to administer) can become unmanageable in the presence of hostile attacks. And I'm assuming here that about a billion buzzers and sirens are going to be going off from the log scrapers (you do at least scrape logs, don't you) which indicates there is a trouble in the neighborhood. So it's not like this is happening in a vacuum and once diagnosed should be relatively easy to restore proper equilibrium. Perhaps this is a case where the unintended consequences of simplicity may have profound implications ? Said another way - I have customers running web servers, simple network daemons, and Oracle in zones and I have no earthly idea how to suggest a rational set of defaults, other than inheriting those of the Solaris parent (which takes me back to my original thought fragment - is this really a zones issue???). Bob ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
Bob Netherton wrote: I see where you are going with this Jeff, and there are some good ideas behind all of this. I have a great desire to rephrase your question without the reference to zones - how well is Solaris itself protected against the various forms of DoS attack ? Do the controls here suggest rational defaults for zones (ie, should we just inherit the limits/protections from the Solaris parent) ? ... Perhaps this is a case where the unintended consequences of simplicity may have profound implications ? Said another way - I have customers running web servers, simple network daemons, and Oracle in zones and I have no earthly idea how to suggest a rational set of defaults, other than inheriting those of the Solaris parent (which takes me back to my original thought fragment - is this really a zones issue???). There are certainly uses for resource management outside of zones but RM is a requirement for zones. The problem for people doing consolidation is they want to create a predictable environment. Before consolidation the various stacks ran on separate systems so a problem on the stack was contained to that machine. When you are consolidating you want the same predictable behavior for the overall system. Without some form of resource management a single zone can unpredictably consume all available resources. This is true for any virtualization scheme. This is why, with a few exceptions, you should always use FSS with zones. This allows any zone to use all of the available CPU resources if it has enough work to do and no other zone is busy, but at the same time it ensures that each zone will get its share of the CPU if it needs it. Setting good values for some of the other controls is trickier although I think Dan's idea of scaling these based on the system makes it easier. We might also want to think about scaling based on the number of running zones. Jerry ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On Thu 10 May 2007 at 04:21PM, Jerry Jelinek wrote: of the other controls is trickier although I think Dan's idea of scaling these based on the system makes it easier. We might also want to think about scaling based on the number of running zones. Another way to look at it (and I think what you are saying) would be to broaden the notion of shares a bit to include more of the system resources-- for example, memory. What's tough there, though, is that our notion of shares today represent an entitlement, and the case of memory, we're talking about a cap on utilization. I think fundamentally we hear from two camps: those who want to proportionally partition whatever resources are available, and those who want to see the system as virtual 512MB Ultra-2's or virtual 1GB, 1ghz PCs. -dp -- Daniel Price - Solaris Kernel Engineering - [EMAIL PROTECTED] - blogs.sun.com/dp ___ zones-discuss mailing list zones-discuss@opensolaris.org
Re: [zones-discuss] Default RM controls for Containers?
On 5/10/07, Dan Price [EMAIL PROTECTED] wrote: I think fundamentally we hear from two camps: those who want to proportionally partition whatever resources are available, and those who want to see the system as virtual 512MB Ultra-2's or virtual 1GB, 1ghz PCs. The typical scenario I see is that an ISV gives a recommendation like V120 with 1 GB of RAM or better. It is then up to the end user to figure how big of a slice of a T2000 or x4600 that is. Using NDA information from Sun I can do this translation accurately enough for my needs. Each machine in my environment is capable of handling somewhere between 4 and several hundred Zone Power Units - ZPUs. It makes the communication of relative server compute power very easy among those familiar with the scheme used. Providing open access to this information across Sun's product line and opening up the computation methods to allow others to benchmark other systems would be very helpful. Perhaps in the future ISV's would say more meaningful things like 1 - 8 threads with at least 17 ZPUs and 6 GB RAM. Mike -- Mike Gerdts http://mgerdts.blogspot.com/ ___ zones-discuss mailing list zones-discuss@opensolaris.org
