joewitt commented on PR #11164: URL: https://github.com/apache/nifi/pull/11164#issuecomment-5639769379
Virtual threads do not each get an OS thread. They share a small pool of carrier platform threads (ForkJoin, ~core count). When a virtual thread does a JDK-managed blocking call (Socket.read, Lock.lock, Semaphore.acquire, most java.net/java.nio), the JVM unmounts it: the carrier is free to run someone else. That’s the win. Pinning means the virtual thread cannot unmount. The carrier sits idle in that blocking call as if it were a normal platform thread. If all carriers are pinned, no other virtual thread in the JVM can run — including unrelated Processors, reporting tasks, and (in this PR) framework tasks that also use virtual threads. On JDK 21 (what NiFi 2 still supports and CI still runs), a virtual thread is pinned if it blocks: inside synchronized / while waiting to enter synchronized / in Object.wait in native/JNI / Panama in a few leftover JDK paths (class init, some file I/O) Concrete NiFi case: Processor holds synchronized (this) or a connection monitor and then does JDBC/HttpClient I/O. That is extremely common in older NARs and in libraries we did not write. Eight such concurrent tasks on an 8-core box can pin every carrier. Then a ListenTCP / ConsumeKafka / funnel that would have unmounted is stuck behind those eight. JDK 24+ (JEP 491) largely fixes monitor pinning. Native pinning remains. Hadoop/HDFS, some crypto, some compression, some Python/JNI bridges still pin. Why this is different from the current engine: today’s timer pool threads are OS threads. Blocking in synchronized just blocks that one worker. The other 9 pool threads keep running. There is no “whole scheduler stalls because 8 monitors pinned 8 carriers.” ReportingTaskWrapper dropping synchronized is the right framework hygiene. It does not make customer NARs safe. What to tell Mark: if default stays on, JDK 21 is a product risk, not a theoretical Loom footnote. Either default off until 24/25 is the floor, or ship with JFR jdk.VirtualThreadPinned guidance and a known-bad list. I would not block the code on this; I would block default-on on JDK 21. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
