We are using only one EC2 (no horizontal scaling) with virtual threads to output video streaming, and we needed to tweak the OS configuration to make it work to support a lot of concurrent connections, we have seen 10K concurrent connections no issue using Spring Boot and JDK 24. Something to take a look https://docs.gatling.io/concepts/operations/
El jue, 27 nov 2025 a las 13:54, Robert Engels (<[email protected]>) escribió: > This is it. The OS out of the box does not expect to deal with that many > connection requests in a short period - especially OSX. It often appears to > be a DoS attack if not reconfigured. > > > On Nov 27, 2025, at 12:46 PM, Alan Bateman <[email protected]> > wrote: > > > > On 27/11/2025 16:38, Cay Horstmann wrote: > >> I did both with David's benchmark. Of course, the semaphore is the way > to go. With default Tomcat settings, 200 or 2000 permits work fine, and > then virtual threads give significantly better throughput than a small > number of platform threads. As one would expect. > >> > >> But back to those 10000 simultaneous connections. As David observed, > left unthrottled, virtual threads do really poorly. In my experiment, > *much* more poorly than platform threads, when using a > Executors.newCachedThreadPool(). > > > > 10k or 50k+ concurrent connections shouldn't be an issue but it may > involve tweaking a number of sysctl.conf/equivalent settings to get there. > Elliot Barlas published a fun project at one point where he had several > million TCP connections with 2 virtual threads per connection. > > > > I think the issue we are discussing here is partly about storming the > server with SYN packets to establish the TCP connections. Servers typically > have rate limiting and other mitigations to avoid SYN flood attacks. This > is why we were asking about the connection backlog and other configuration. > It's possible the thread pool bench isn't going to see this because it's > using synchronous/blocking APIs to establish the TCP connection and so the > number of outstanding connects in progress is limited by the size of the > thread pool. > > > > The other part that seems relevant is that the protocol in the > benchmarks is HTTP/1.1 which supports persistent connections, essentially > re-use of a TCP connection if possible to avoid creating a new TCP > connection for each HTTP server. There's another layer of configuration > here. I don't know Tomcat's defaults but it is likely that it keep 100 or > 200 connections alive for future HTTP requests. Once the concurrency > exceeds this then it's likely there is a new TCP connection established for > each request. The thread pool bench is 4 threads and likely well below the > maximum number of persistent connections. > > > > So yes, moving from thread pools to virtual threads may shine light on > the "next" concurrency bottleneck, the bottle hidden by limiting the number > of threads. The guidance that we've given at conferences and in the JEPs is > to use a semaphore/equivalent to limit concurrent at the right level of > granularity, be that number of concurrent database connections or whatever > it might be. > > > >> > >> I am wondering why that would be, and whether it is something that is > worth addressing, because it seems like something that people could run > into in practice. FWIW, I stumbled upon > https://bugs.openjdk.org/browse/JDK-8360046 which addresses a somewhat > similar scenario. > > > > That one is an over subscription issue that arises with producer > starting a lot of virtual threads that "do nothing". It helps us to reduce > the gap between ForkJoinTasks that "do nothing" with virtual threads that > "do nothing". > > > > -Alan > > > > > -- Daniel Andrés Pelaez López
