Dain Sundstrom wrote: > Michael Bartmann wrote: > >> Everybody heathy? >> >> If you operate a 24*7 production plant with a server like JBoss, >> you shurely care about the "health" of your server. Now in theory >> this is nearly trivial: we have JMX and friends. >> Any kind of monitoring can be done with it, but do we do? Of course >> a lot of aspects are application specific, and for the rest of it >> you may like to pull myserver:8080/jmx-console. >> >> I still have some - trivial - thoughts about what could be done on the >> server side to faciliate things; and I post my thoughts here to >> collect some of your opinions on this (you care about health, do you?). >> >> 1) Threadcount >> For me this is more or less a linux problem, allthough I found >> questionable things even on NT4. The problem is that the current >> thread count reported by the JVM is garbage. The best thing i can think >> of is traversing the threaddump and counting threads on the fly. > > > Why is the JVM thread count garbage?
I meant the thread count of sun jvms. IBM jvm thread count works fine, but recent JBoss 3.2 releases don't run smooth on IBM. On one of my sytems I had an active thread count of above 900 when there were only about 70 processes running (which you could see under in the JBoss thread dump _and_ using "ps aux | grep java". I searched the sun bug database and found lots of bugs, e.g.: Bug ID: 4526761 Shutdown Hook doubles Thread count on Linux Bug ID: 4089701 java.lang.ThreadGroup.activeCount() counts non-started threads. > >> 2) Thread usage >> If you are not an expert on every single part of JBoss, it might be >> a detectives work to find out where all those nice threads in the >> threaddump come from. >> This is even more criticall, as we migrate to centrally managed thread >> pools, where its more easy to do it wrong (not naming them appropriate). >> Thread creation (and even pooling) doesn't come at no cost. I think >> it might be affordable to do some more bookkeeping (factory?) >> about how and when and from which part of the system (stacktrace?) >> threads get allocated. This information - if accessible via JMX - should >> give you a warm feeling. > > > I agree. This is why I want a centrally managed thread factory (pool). > >> 3) Memory usage >> I don't understand much of the JVM heap structure but I know of tools >> (e.g. JProbe) which which can provide you with information about each >> and every living object on the heap. These tools do at least use the >> JVM debugging API, I don't know if they need modified native classes >> to do so. If you do know, please let me know. > > > With JDK 1.4 we should be able to inspect the VM using the debugging > APIs. Do you want to figure out how? I fear I only have time for some simple tasks right now. The next thing I plan is a reliable thread count and a server maintained history thereof. Unless Rhett Aultman comes first :-) > > http://java.sun.com/j2se/1.4/docs/guide/jpda/enhancements.html > >> 4) History >> The current JBoss measures for counting threads and memory usage don't >> keep a history about thread usage in the past. With a monitoring tool >> online all the time we could solve the problem on the client side. >> It should be trivial to support this through a collection MBean on the >> server side. The interesting part comes with the persitence of this >> information? What to do after some crash (out of memory)? >> Perhaps log4j is the solution? Or do you favor database storage? >> (Even that could be done with log4j and appropriate Logger/Appender >> setup. >> >> 5) Caches, pools and such >> I would prefer more control on caches, pools and containers. If >> everything >> is "done right" (with SoftReferences?) we should not fear out of >> memory due >> to bad configuration. >> But as I noticed on this list these aspects are on the TODO for Matrix2 >> anyway. > > > Caches and pools are planed for a full rewrite in 4.0. > > So do you want to start working on this stuff? At the moment my spare time is approx. zero. But the things I have to do on my job might force me to do something on this; at least I need a reliable graph of thread and memory usage. The reason I posted the whole blurb was to get warned if something was already done (or considered obsolete). It shouldn't mean "I'll lean back and wait till someone else did it" either :-) > > -dain > > > Michael Bartmann ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Jboss-development mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/jboss-development
