[SoC] Jail Resource Limits

2006-08-21 Thread Chris Jones


Hi, folks: I've completed the Summer of Code work on the jail  
resource limits (http://wikitest.freebsd.org/JailResourceLimits);  
jails can now have soft-ish limits placed on their memory and CPU usage.


Briefly, when a jail uses more memory than its limit, pages are  
clawed back by a new kernel process, jpager.  The clawback is at a  
configurable rate, though the proportion is currently fixed.  When a  
jail uses more CPU time in proportion to total CPU time used than the  
number of its CPU usage shares to the total CPU usage shares, its  
processes are dropped in priority until it's had its fair share of  
the total CPU time; if there are no other processes that want to run,  
they'll use up as much CPU time as they otherwise would.


We have a new command, jtune(8), which allows you to change the  
memory limit and CPU shares for a jail while it's running.  You can  
also set these when a jail's created.  Note that some sysctls need to  
be set (see jail(8) for details) before the limits will be enforced.


A tarball with everything you need to build the experimental kernel  
can be at http://www.ualberta.ca/~cdjones/cdjones_jail_soc2006.tgz  
--- please read the INSTALL document in docs/ carefully, as it's  
pretty easy to trash your libc.so.  The tarball is against RELENG_6  
as of a few hours ago; I expect to port it to CURRENT sometime this  
week, once I build CURRENT on the desktop whose libc.so I trashed  
while writing docs/INSTALL. :)


I look forward to your feedback.

Cheers,

Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]


Re: [SoC] Jail Resource Limits

2006-08-21 Thread Chris Jones


On 21-Aug-06, at 9:37 AM, Remko Lodder wrote:

Before my little question, i would like to take the opportunity to
thank you for the hardwork, this is a much requested feature and
will greatly appriciated by the crowd :-) Cheers for that!.


Thanks!  I do some web hosting with jails on the side, and've often  
wished this sort of thing were around.  I suppose necessity truly is  
the mother of invention. ;)



My little question: How does this impact on performance? I noticed
all your p4 commits and saw some pieces of code that constantly
monitor the usage of the jail (wrt CPU and Memory); doesn't that
reduce our performance? Is that performance impact (if any) also
growing when using more jails?


The changes can be broken out into three major groups: the memory- 
limiter (kern_jail:jpager_td), the CPU usage monitor  
(sched_4bsd:schedcpu), and the process priority resetter  
(sched_4bsd:resetpriority).


The memory-limiter essentially idles if the sysctl for memory limits  
is turned off or the jail doesn't have a memory limit; otherwise, we  
need to go over all processes and check whether they're in the jail  
and sum their RSS (in prison_memory).  This could really use having a  
quick way of getting a list of processes in a jail (perhaps a new  
entry in the proc struct for next-process-in-this-jail?).


If we're over the limit, we then have to go over all processes again  
and check whether they're in the jail and amenable to being pushed  
out.  If so, we then go and push them out in the same way that the  
pagedaemon does.  This bit's going to be relatively slow, and also  
could use that quick get-processes-in-jail method.


The CPU usage monitor keeps track of the number of CPU usage shares  
allocated in schedcpu_thread, which runs every second and then calls  
schedcpu (see below).  This one's linear with the number of jails and  
again could probably benefit from a quick check to see whether we're  
enforcing CPU shares at all.  An even better method would be to just  
recalculate the total outstanding shares when the jail or  
jail_set_resource_limits syscalls are called or the sysctl for  
unjailed shares changes, rather than all the time.  The amount of CPU  
time each KSE uses is already collected by the system, so we just re- 
use that.


schedcpu needs to lock the allprison mutex and reset each prison's  
estimated CPU time at line 466, so that'll be about linear with the  
number of jails.  Hmm, looking at it, there's a really obvious  
optimization that I forgot to put in (skip the lock  reset if we  
have no jails), which I'll want to add.


Next, it will increment the CPU time if the sysctl is set and the  
process is jailed.  We're looking at about eight loads if the  
compiler's smart, three compares and two ors, an if, and maybe an  
add.  Unfortunately, this time's linear with the number of processes.


The big hit comes in the process priority resetter, which runs when  
we consider what priority to run a process at (this runs  
intermittently for each active process).  If the sysctl is set and  
the process is jailed, we do some reasonably hefty (some multiplies)  
calculations to change it.  This'll be roughly linear with the number  
of jailed processes.


Overall, I don't think there'll be a noticeable performance impact  
for non-pathological numbers of processes or jails, but I don't have  
any stats to back that up.



I think these things are simple, but important to know; I do not
have a full featured setup (sadly) to measure the performance
impact of this, If you are able to find someone, that would be
really great imo.


Consider this a general appeal for anyone who can help me do some  
analysis on the performance testing, or give me some pointers on how  
to collect the relevant data. :)


Cheers,

Chris


___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to [EMAIL PROTECTED]