Firstly, thank you Rich for wading in here with your substantial skills on
the subject at hand. I was just thumbing and fumbling through Solaris
Systems Programming looking for the correct chapter.
I also have "The Magic Garden Explained" by Goodheart & Cox here in my hands
and I note that the opening preface leads with the following quote :
I take the view, and always have done, that if you cannot say what
you have to say in twenty minutes, you should go away and write a
book about it. - Derek Charles Moore-Brabazon
Well, having repeated that I now wonder what I will find in my freshly
ordered copy of "Solaris Internals" 2nd Ed.
> On Wed, 8 Nov 2006, Dennis Clarke wrote:
>
>> How does Solaris load up its tasks and know when to say "stop, no more
>> please"?
>
> Provided the kernel doesn't run out of VM, I don't think the
> Solaris kernel ever says that (I'm thinking here of pathalogical
> fork-bomb experiments, where the load has gone >> 1000).
Right.
The "pathological" ( caused by or evidencing a mentally disturbed condition
) fork-bomb situation is exactly the sort of thing I try to prepare for with
the following /etc/system entries : ( these are verbatim from my system )
**************************************************************************
*
* Description : reserved_procs
*
* Specifies number of system process slots to be reserved
* in the process table for processes with a UID of root (0)
*
* Consider increasing to 10 + normal number of UID 0 (root)
* processes on system. This setting provides some cushion
* should it be necessary to obtain a root shell during a time
* when the system is otherwise unable to create user-level
* processes.
*
set reserved_procs = 254
***************************************************************************
*
* Description : max_nprocs
*
* Maximum number of processes that can be created on a system.
* Includes system and user processes. Any value entered in
* /etc/system is used in the computation of maxuprc.
*
* This value is also used in determining the size of several
* other system data structures. Other data structures where
* this variable plays a role are:
*
* - Determining the size of the directory name lookup
* cache (if ncsize is not specified).
*
* - Allocating disk quota structures for UFS
* (if ndquot is not specified).
*
* - Verifying that the amount of memory used by
* configured system V semaphores does not exceed
* system limits.
*
* - Configuring Hardware Address Translation resources
* for the sun4m and Intel platforms.
*
set max_nprocs = 1024
******************************************************************************
*
* Description : maxuprc
*
* Maximum number of processes that can be created on a system
* by any one user.
*
* Data Type : Signed integer
*
* Default : max_nprocs - reserved_procs
*
* Range : 1 to max_nprocs - reserved_procs
*
* Validation : Compared to max_nprocs - reserved_procs and
* then set to the smaller of the two.
*
* When you want to specify a hard limit for the number of
* processes a user can create that is less than the default
* value of however many processes the system can create.
* Attempting to exceed this limit generates the following
* warning messages on the console or in the messages file:
*
* out of per-user processes for uid N
*
set maxuprc = 384
EOF
These are just numbers that I toss in and adjust based on the system
resources at hand. They do a very fine job of totally protecting me from
such obscenities as :
:(){ :|:& }; :
When presented to the bash shell that seems to really upset the system. IT
will completely crash a Red Hat Enterprise 4 server but my Solaris server
here, left with stock bland /etc/system merely gets busy for a while. With
those /etc/system entries above I get promptly told that I am a bad person
and to not do such silly things. More or less. :-)
>> And I write a script that has hundreds of files to unzip. Each file is
>> variable in length and I unzip each file thus :
>>
>> priocntl -e -c FX -m 0 -p 0 unzip -q foo.zip &
>>
>> So there is a script that has hundreds of lines of that for hundreds of
>> little files called foo_N.zip
>
> [...]
>
>> # vmstat 5
>> kthr memory page disk faults cpu
>> r b w swap free re mf pi po fr de sr f0 s2 -- -- in sy cs us
>> sy id
>> 0 0 0 1800088 841272 9 39 57 17 17 0 13 0 11 0 0 603 646 324 3
>> 4 93
>> 8 0 0 1782680 765176 741 2387 0 2548 2548 0 0 0 38 0 0 700 4385 126 51
>> 49 0
>> 5 0 0 1764040 762392 784 2839 6 3285 3285 0 0 0 86 0 0 950 5074 247 45
>> 55 1
>> 29 0 0 1752320 767168 513 65 2 4645 4645 0 0 0 93 0 0 1003 3400 197 59
>> 34 7
>> 26 0 0 1755160 775216 300 6 0 3567 3567 0 0 0 143 0 0 1310 4104 333 46
>> 38 16
>> 34 0 0 1761392 772416 363 3 2 4899 4899 0 0 0 106 0 0 1082 2448 229 60
>> 30 10
>> 29 0 0 1765464 771760 304 2 0 4941 4941 0 0 0 135 0 0 1272 4571 253 47
>> 37 15
<< snip >>
>> You see that? I had a maximum of 34 threads in the run queue.
>>
>> But I have hundreds of tasks to run and they are all low priority fixed
>> class processes.
>
> Conjecture follows; I'm sure others who are more intimate with the kernel's
> inner workings than I will correct me if I'm wrong...
Hopefully we can drag them in here also.
> You have hundreds of unzip operations to do, but observe a maximum
> of only 34 at any one time. I guess it's possible that the number
> of processes went higher between vmstat samples, but I think what
> you're seeing here is the effect of some of those unzips finishing.
Possibly. I also wondered if the issue was that unzip is an IO bound
process and not really limited to CPU and RAM. If these were IO bound
processes then I would think that there was some logical point at which the
kernel would say "oops, I can't drive any more IO, so let's stop serviceing
new processes".
So really I was seeing 34 unzips happening simply because no more IO could
be pushed to the filesystem.
Its a weak line of thought.
> So, unzip1 starts, then unzip2, and so on. But by the time unzip35
> starts, the first one has finished, hence the value of the r column.
>
>> There .. I actually laid that all out. It was what was in my head and I
>> was
>> even going to say that a multi-core person could have a ladel in each
>> hand.
>> Possibly eight hands in the case of an UltraSparc T1.
>
> Visions of the Hindu goddess Durga ladeling water abound!
Precisely what I was thinking. Except that third column is "swapped out
processes" and how does one pull them back from the brink of never never
land? What ladel reaches into that bucket and when?
I don't rightly know.
>> At what point does Solaris push back and say "no more, I'm busy" ? I am
>> now
>> going to go to amazon.com and order a copy of "Solaris Internals".
>
> Good idea on the latter;
done !
> as for the former, I don't think it does,
> VM resources notwithstanding.
This is the sort of theory that begs to be tested a bit eh ?
Dennis
_______________________________________________
opensolaris-discuss mailing list
[email protected]
