Hi List,
just for the sake of completeness, here is a basic outline of how we
solved this issue for us (e.g. how to determine how many of the
allocated threads are actually busy):
Basically, we run multiple pound instances on a single server that deal
with multiple backends on separate servers. As the script itself is
closely tied to our setup, I will try to explain what we are doing,
rather than just pasting some random code.
We created a simple bash script, that
- knows a way to enumerate all pound instances on the local system
- checks if each one is running
- if so, it checks the syscall info from /proc for all sub-threads of
the given pound:
/proc/POUND_PROCESS_ID/task/*/syscall
- it seems that you can determine idle threads because their last
syscall was 202 (which equals getegid() )
- active threads usually show 7 (waitpid()) or 35 (ftime())
- with these information, you can count (or rather guesstimate) active +
inactive pounds
In our case, we use the bash script to write the numbers out to a
temporary file, which is then read by a snmpd perl extension upon
request that exports all numbers as some sort of snmp table. This table
is being read by cacti plugins which in turn create simple graphs for
each instance that is running on the system. You can always use the
cacti GUI to rediscover all running pound instances if there are new
ones and have cacti create new graphs for those as well (without doing
any changes to the bash script, the snmp extension or the cacti templates).
The SNMP approach is somewhat generic - it just needs a file with all
the relevant information in it. How those numbers are actually
calculated, that is up to the bash script running as a cronjob in the
background. That way, we can always change the way of calculating the
numbers without breaking the snmp extension or the cacti graphs/templates.
This is an example line from our stats file:
[POUND]: some-pound-instance-name, [DEFINED_THREADS]: 128, [THREADS]:
130, [ADDITIONAL_THREADS]: 2, [ACTIVE]: 2, [INACTIVE]: 128, [UNKNOWN]:
0, [USAGE]: 1
You see an idle pound instance that has 128 threads configured in its
config file. 130 Threads are actually running (these two additional
threads are not connection-worker-threads but pound-internal threads).
"Unknown" threads are those that have a syscall other than the ones
mentioned above. This should always be zero or can otherwise be used to
improve the accuracy of the script.
This is not in production yet - we have only used it on our testbed.
Cacti uses these numbers to produce nice graphs, Icinga checks if any
pound instance reaches a critical amount of active threads. That way we
can always increase our threads limit before running into trouble (or
decrease, if necessary).
Please let us know what you think. If anyone is interested, we might be
able to publish the snmp extension along with the cacti configuration
and the icinga check command and some code snippets that might help
implementing the process/thread checker on the server itself.
On 10/16/2013 06:38 AM, Leo wrote:
Hi,
I have already complained about the new threading model in version 2.6
(http://www.apsis.ch/pound/pound_list/archive/2012/2012-01/1326878271000#1326878271000)
and some of my suggestions have been added to the feature list for Pound 2.7
(http://www.apsis.ch/pound/pound_list/archive/2012/2012-02/1328374222000#1328374222000).
Current beta version of 2.7 shows the length of the request queue
(http://www.apsis.ch/pound/pound_list/archive/2012/2012-04/1333979077000#1333979077000).
But I suspect that the old (dynamic) threading model will not come back.
Regarding a release date for version 2.7 you have to ask Robert Segall ;-)
Hope this helps ...
Regards,
Leo
On 10/15/2013 04:27 PM, Rudolph Bott wrote:
Hello List,
we are currently switching from pound 2.5 to 2.6, which introduces us to the
static thread model. Of course it is pretty easy to count the current threads
on our pound processes to find a suitable number of threads to configure. But
how would you actually monitor this in the future so that you can
readjust the number of threads if needed? Since there are always N threads
allocated and there is no information regarding threads available via the pound
socket - how would you actually count the usage?
Besides that, is there a release date for pound 2.7? Comparing the debian
changelog to the 2.7 feature list that was discussed here earlier, some of the
new features seem to be already integrated by debian. However, we would welcome
a way to use dynamic threads again OR a way to monitor threads so
that we know when to readjust settings.
--
To unsubscribe send an email with subject unsubscribe to [email protected].
Please contact [email protected] for questions.
--
Mit freundlichen Grüßen / With kind regards
Rudolph Bott
--
To unsubscribe send an email with subject unsubscribe to [email protected].
Please contact [email protected] for questions.
