Rainer,
Thanks for writing this up.
On 3/20/24 07:22, Rainer Jung wrote:
I wanted to share an observation and I hope the things are correct how I
am describing them. Maybe things have already improved and I am not
aware of it, hints welcome.
Part of JEP 425 (Project Loom, Java virtual threads) discusses how to
handle observability of virtual threads from inside the JVM and tooling.
The final outcome is, that virtual threads are not included in the
typical JVM APIs which one can use to observe threads, like enumerating
them or accessing the stacks of individual threads. As a consequence,
also jstack and "kill -QUIT" do not show virtual threads at all, not
even when they are attached to a native thread and executing code.
O_O
There is one single method to help with observability of virtual threads
https://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)
which dumps a full thread dump to a file. Of course that is by no means
appropriate, if you want to do a fine grained observation. At least you
can choose to write a json structure instead of a hard to parse text
format, but that's it.
Boy, that's a real miss from the JVM team. I'm surprised that they have
overlooked this. I don't see a real reason that a Virtual Thread would
be treated any differently than a regular thread.
For instance I am often using a tool, that inspects our
RequestProcessors to find long running requests and then retrieves the
list of Java threads as ThreadInfo objects to find the one executing the
request and finally retrieves the stack of that request from the JVM.
Such an approach is no longer possible. Almost all of JMX does not show
any info about virtual threads.
It also seems Tomcat no longer uses a pool when a connector is
configured to use virtual threads. So there are no metrics like
currentThreadCount or currentThreadsBusy.
When using virtual threads, the number of requests is the same as the
number of threads, since each request -> new Virtual Thread.... though I
think the request-count is only incremented when the request has
completed, so maybe there are threads running that can't be counted (yet).
But I understand that you were hoping to get some information about
these threads and though maybe Tomcat's metrics could help. I guess not
really.
But thread-busy count is the same as in-flight-request count. And
current thread count is the same as in-flight-request count as well.
I guess this also limits the use of some manager features and probably
also the StuckThreadsDetectionValve when combined with virtual threads.
Most likely. I'm fairly sure that uses the "usual Thread-walker things"
which you are reporting do not work any more with VTs.
What JVM are you using for your investigations?
Of course Tomcat has solved most of the problems, that virtual threads
want to solve, by doing the hard work of using the NIO and NIO2 APIs. So
virtual threads are probably not that important for us. But we should be
aware of the observability deficiencies whenever we would discuss
whether we could switch from NIO or NIO2 to virtual threads to simplify
connector code in some distant future.
+1
I've been enthusiastically talking with markt about dropping all that
nasty NIO stuff and "going back to BIO with virtual threads" which, at
this point, is mostly a threat and not a promise. But if VTs really
deliver everything they are claiming to deliver, then it may be possible
to go back to BIO as long as servlet-async is retired. And I'm not
holding my breath on that one.
-chris
