On Mon, Nov 10, 2008 at 9:07 AM, Jeff Victor <[EMAIL PROTECTED]> wrote:
> On Mon, Nov 10, 2008 at 12:30 AM, Mike Gerdts <[EMAIL PROTECTED]> wrote:
>> FWIW, I tend to use the term "reserved memory" instead of "swap"
>> because that is less confusing to most people.
> That's a useful perspective. If you choose the swap cap - which is
> really a VM cap - so that the sum of the swap caps is less than RAM,
> you have effectively implemented 'reserved memory.' (I'm ignoring RAM
> usage of the global zone, which shouldn't be ignored in practice.)
> But you must be careful: nothing prevents you from 'over-reserving'
> memory. If you have 'reserved' all of system memory in this way, and
> add a new zone with its own 'reserve,' you will have over-subscribed
> memory. That might be a good thing, as long as no one is surprised if
> the system starts paging.

Very true.  Same goes for FSS and CPU caps.  There are plenty of ways
to shoot yourself in the foot.  :)

> However, the entire concept of reserved memory limits the scalability
> of the system. Imagine 4 zones with swap caps of 4GB, on a system with
> 16GB of RAM. (Again, I'm ignoring the GZ.) Unless you allow yourself
> to over-subscribe RAM, you can't add more zones, even if those 4 zones
> are only using 1GB each during normal conditions.

I'm actually looking forward to SSD to minimize the impact.  Paging to
it should give a relatively small amount of performance impact while
giving some useful stats to let the administrator know that there is
an upcoming capacity problem.  If I pretend that a 64 GB SSD drive
will cost about $1K and I should mirror them, that gives me the
ability to have a "fast swap device" for about $31 / GB.  If swap is
on zfs with zil + l2arc, cost may be even less.  Assuming performance
meets expectations, this is pretty cheap peace of mind.  My only
performance expectation is that the workloads that aren't paging but
may be doing some I/O have response time similar to what they would
when the rogue app starts paging.  With traditional drives, seek times
punish everyone.

> Balance is needed. When paging must be avoided at all costs,
> 'reserving' memory by setting proper swap-caps makes a great deal of
> sense. When paging is unlikely because the workload is well
> understood, and a small amount of paging would not be horrible, and
> zone 'density' is important, reserving memory would not make sense.
> Many situations would call for memory 'reservations' on some zones,
> and RAM caps on others.

The tricky part is that you have to set the swap caps on all zones to
reserve the memory for those that you really want to protect.

>> For CPU related stats, take a look at a discussion I started a while back:
>> http://mail.opensolaris.org/pipermail/perf-discuss/2005-November/002048.html
> Cool. Also, Jim Fiori had a simple idea for counting CPU time per zone
> with almost no perf impact: use DTrace to implement a probe which
> fires every M microseconds, and increments a per-zone counter. But
> that's a short-term solution. We need a per-zone counter in the kernel
> that tallies CPU time per zone.
>> One project I would like to kick off sometime is doing per user, per
>> project, and per zone microstate accounting.
> Excellent idea. I'll watch for it! :-)
> <snip>
>> I didn't have a chance to check logic closely or run it on a test
>> system.  I'll offer more feedback if needed when I get a chance to
>> test it.  It is a great start and I can't wait to see it progress.
> Thanks!

   224  open (RELEASE, "/etc/release");
   225  $rel=<RELEASE>;
   226  close RELEASE;
   227  if ($rel =~ /3\/05/) { $update=1; }
   228  if ($rel =~ /6\/06/) { $update=2; }
   229  if ($rel =~ /11\/06/) { $update=3; }

3/05 == GA (update 0?)
1/06 == U1

   492  # Note that swap(1M) doesn't report memory pages that the
kernel has locked.
   493  open (SWAP, "/usr/sbin/swap -sh|");
   494    while (<SWAP>) {

The -h option to swap doesn't exist in S10.

Mike Gerdts
zones-discuss mailing list

Reply via email to