Hi Rainer, Think this is not related to virtual threads I think and way older (2010?). Since some years we use AsyncContext and therefore thread dumps don't match pending requests but just active computation (you can get 8K requests with 8 threads for ex). Most of the sync instrumentation (assumptions) are broken since Servlet 3.0 and must not be used to conclude anything but generally other metrics and data enable to monitor the server properly (request counter for example is decoralated from threads, active connections for database - generic term - etc). Thread stacks stay useful for performance related work but not for hanging investigation (for good since it enables a better scalability in general) so guess it is mainly a habit switch in terms of tools, no?
Hope it makes sense. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> Le mer. 20 mars 2024 à 13:11, Rainer Jung <rainer.j...@kippdata.de> a écrit : > Hi hi, > > I wanted to share an observation and I hope the things are correct how I > am describing them. Maybe things have already improved and I am not > aware of it, hints welcome. > > Part of JEP 425 (Project Loom, Java virtual threads) discusses how to > handle observability of virtual threads from inside the JVM and tooling. > > The final outcome is, that virtual threads are not included in the > typical JVM APIs which one can use to observe threads, like enumerating > them or accessing the stacks of individual threads. As a consequence, > also jstack and "kill -QUIT" do not show virtual threads at all, not > even when they are attached to a native thread and executing code. > > There is one single method to help with observability of virtual threads > > > https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat) > > which dumps a full thread dump to a file. Of course that is by no means > appropriate, if you want to do a fine grained observation. At least you > can choose to write a json structure instead of a hard to parse text > format, but that's it. > > For instance I am often using a tool, that inspects our > RequestProcessors to find long running requests and then retrieves the > list of Java threads as ThreadInfo objects to find the one executing the > request and finally retrieves the stack of that request from the JVM. > Such an approach is no longer possible. Almost all of JMX does not show > any info about virtual threads. > > It also seems Tomcat no longer uses a pool when a connector is > configured to use virtual threads. So there are no metrics like > currentThreadCount or currentThreadsBusy. > > I guess this also limits the use of some manager features and probably > also the StuckThreadsDetectionValve when combined with virtual threads. > > Of course Tomcat has solved most of the problems, that virtual threads > want to solve, by doing the hard work of using the NIO and NIO2 APIs. So > virtual threads are probably not that important for us. But we should be > aware of the observability deficiencies whenever we would discuss > whether we could switch from NIO or NIO2 to virtual threads to simplify > connector code in some distant future. > > Best regards, > > Rainer > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org > For additional commands, e-mail: dev-h...@tomcat.apache.org > >
