On Tue, 2019-10-01 at 19:08 +0900, Yasufumi Mizoguchi wrote: > * The number of Firing hosts : 6 > * [Each host]ThreadGroup.num_threads : 200 > * [Each host]ThreadGroup.ramp_time : 600 > * [Each host]ThreadGroup.duration: 1800 > * [Each host]ThreadGroup.delay: 0 > * The number of sample queries: 20,000,000 > > And we confirmed that Jetty threads was increasing and reached the > limit (10000). > Therefore, we raised MaxThreads value.
You have a server with 16 CPU cores and you're running more than 10K threads on it: That's a sure way to get lower throughput as the system struggles to switch between the many threads. You would probably get higher throughput by limiting the number of concurrent threads (for once it seems that Erick and I are in a disagreement). It's easy to try: Just set num_threads for each ThreadGroup to 20 and see if your QPS rises above the 60 you have now. Still, there's something off: 6*200 threads = 1200. That should not blow Jetty thread allocation beyond 10K. Maybe your index is sharded? that would explain extra thread allocation. If so, how many shards do you have and have you tries running with a single shard? Single shard indexes maximizes throughput at the possible cost of latency, so that seems fitting for your requirements. - Toke Eskildsen, Royal Danish Library
