Romain, On 3/20/24 13:34, Romain Manni-Bucau wrote:
Thread dumps being dump of threads - literally os threads - and virtual threads not being threads at all - they are runnables in a dedicated thread pool - it is quite fair to not make them the same and have their scheduler - pool - only in the thread dump but not themselves no?
Maybe.If you take a thread dump today (with a "real" thread), you only get a Java stack trace, you get no "native stack trace" or anything like that. So from that perspective, the "thread" is really the instance of java.lang.Thread which could just as easily be a Virtual Thread.
You also get no scheduling information, other than what the thread's "priority" is... but you can't get any real-time data about where it sits in the scheduling queue, etc.
I'm much less interested in what the "native thread" is doing _below_ the Java part. Presumably, it's always running Thread.cpp::runTheJavaCode and that's not useful information to anybody.
If the Virtual Thread is not mounted on an OS thread, then it's "suspended" or "blocked" or whatever-it-is. If it's on an OS thread, it had better be running: that's the whole point of the scheme. (I suppose it could be BLOCKED-yet-on-an-OS-thread -- one of the current problems which hopefully will be less of a problem in the future, but I'd like to ignore that for now).
I don't know what's wrong with having millions of threads, and still being able to walk through them using existing APIs. Nobody walks through threads for no good reason... if you want to see the threads, you should be able to see the threads.
If I wanted to walk-through every instance of java.lang.String in a JVM (¡millions of them!), then I should be able to do it even if (a) it's a weird thing to do and (b) it will take a while. Well, I want to do it, so let me do it(!).
If you have only one "real" Thread (e.g. the main thread) and everything else is Virtual... when you ask for a thread dump or walk all the threads, do you only see "main" and not the mounted-on-OS-threads Virtual Threads? If the JVM were willing to consider Virtual Threads "visible" and therefore dumpable, etc. through existing interfaces _while they were mounted on an OS thread_ I could almost agree that makes some sense. But if I have a unit of work not making any progress, I'm gonna want to be able to see that (Virtual) Thread in a thread dump or any other kind of analysis tool, including its stack trace.
Similarly to why threadlocal are not recommended for virtual thread this would probably make an useless pressure on the JVM IMHO - why there is an option to see it but it is mainly for debug purposes. See virtual threads as continuations (suspendable/resumable "Runnable") in a dedicated and not programmatically configurable nor selectable/proviable thread pool, they are not in thread dumps and this doesnt bother you ;) - ultimately if you want it you want all java objects to be monitored and in the thread dump which would be weird - but I agree the semantic is misleading.
I'm not sure why ThreadLocals are not recommended for Virtual Threads. Honestly, the presence ThreadLocal is a gigantic hack for badly-written APIs but if you accept that fact, there doesn't seem to be anything really bad about it. It "doesn't make sense" for Virtual Threads because it's wasteful: you probably use something once or twice and don't get the benefit of it when the thread dies -- because you aren't supposed to use thread pools anymore. But it's no less wasteful and dumb than it was in the first place.
IIRC, Tomcat uses ThreadLocal to store date-formatting objects for access logs. That's a great use for ThreadLocal. If the threads are no longer pooled, it's "wasteful" in that we create and discard a SimpleDateFormat for every log, but what's the alternative? Use a shared SimpleDateFormat and synchronize across threads? Yuck. So it's up to Tomcat to decide if ThreadLocal makes sense in a Virtual Thread world.
(Honestly, we need to start using java.time. Now that Tomcat 8.5 no longer needs to be supported, we should look at our options for moving-on.)
From my tests virtual threads do their job but they stay slower than a proper reactive/async impl mainly due to the overhead they add *everywhere* compare to reactive programming plus this single thread pool issue which adds a lot of contention when all the app/lot of tasks is/are done using it - vs bulkhead pattern for ex. But if you come from a plain sync application it can be very interesting if it stays compatible and you don't need to add throttling to control the memory - often in the old style you throttle with the number of threads, now you need a semaphore or alike. Will not be a free lunch ;).
Agreed, free lunches never exist. But.If an application exists that never bothered to convert to using non-blocking I/O but uses many threads, it could be very easily changed (slightly) to use Virtual Threads and get a performance boost out of the deal. Or at least be able to schedule many more tasks without running out of stack space.
The price you pay for that is CPU usage. And most of us have CPU to spare. -chris
Rainer, Thanks for writing this up. On 3/20/24 07:22, Rainer Jung wrote:I wanted to share an observation and I hope the things are correct how I am describing them. Maybe things have already improved and I am not aware of it, hints welcome. Part of JEP 425 (Project Loom, Java virtual threads) discusses how to handle observability of virtual threads from inside the JVM and tooling. The final outcome is, that virtual threads are not included in the typical JVM APIs which one can use to observe threads, like enumerating them or accessing the stacks of individual threads. As a consequence, also jstack and "kill -QUIT" do not show virtual threads at all, not even when they are attached to a native thread and executing code.O_OThere is one single method to help with observability of virtual threadshttps://docs.oracle.com/en/java/javase/21/docs/api/jdk.management/com/sun/management/HotSpotDiagnosticMXBean.html#dumpThreads(java.lang.String,com.sun.management.HotSpotDiagnosticMXBean.ThreadDumpFormat)which dumps a full thread dump to a file. Of course that is by no means appropriate, if you want to do a fine grained observation. At least you can choose to write a json structure instead of a hard to parse text format, but that's it.Boy, that's a real miss from the JVM team. I'm surprised that they have overlooked this. I don't see a real reason that a Virtual Thread would be treated any differently than a regular thread.For instance I am often using a tool, that inspects our RequestProcessors to find long running requests and then retrieves the list of Java threads as ThreadInfo objects to find the one executing the request and finally retrieves the stack of that request from the JVM. Such an approach is no longer possible. Almost all of JMX does not show any info about virtual threads. It also seems Tomcat no longer uses a pool when a connector is configured to use virtual threads. So there are no metrics like currentThreadCount or currentThreadsBusy.When using virtual threads, the number of requests is the same as the number of threads, since each request -> new Virtual Thread.... though I think the request-count is only incremented when the request has completed, so maybe there are threads running that can't be counted (yet). But I understand that you were hoping to get some information about these threads and though maybe Tomcat's metrics could help. I guess not really. But thread-busy count is the same as in-flight-request count. And current thread count is the same as in-flight-request count as well.I guess this also limits the use of some manager features and probably also the StuckThreadsDetectionValve when combined with virtual threads.Most likely. I'm fairly sure that uses the "usual Thread-walker things" which you are reporting do not work any more with VTs. What JVM are you using for your investigations?Of course Tomcat has solved most of the problems, that virtual threads want to solve, by doing the hard work of using the NIO and NIO2 APIs. So virtual threads are probably not that important for us. But we should be aware of the observability deficiencies whenever we would discuss whether we could switch from NIO or NIO2 to virtual threads to simplify connector code in some distant future.+1 I've been enthusiastically talking with markt about dropping all that nasty NIO stuff and "going back to BIO with virtual threads" which, at this point, is mostly a threat and not a promise. But if VTs really deliver everything they are claiming to deliver, then it may be possible to go back to BIO as long as servlet-async is retired. And I'm not holding my breath on that one. -chris --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
--------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@tomcat.apache.org For additional commands, e-mail: dev-h...@tomcat.apache.org
