wu-sheng commented on PR #13703: URL: https://github.com/apache/skywalking/pull/13703#issuecomment-3941046785
## Thread Count Benchmark — v10.3.0 vs v10.4.0 (JDK 25) **Environment**: 2-node OAP cluster, BanyanDB storage, JDK 25 (`eclipse-temurin:25-jre`), Istio bookinfo traffic, 10-core Kind node. **Method**: 5 thread dump rounds × 60s apart via `kill -3`, JVM-internal threads excluded. ### Results | Pool | v10.3.0 threads | v10.4.0 threads | Notes | |------|-----------------|-----------------|-------| | L1 Aggregation (OAL + MAL) | 26 (DataCarrier) | 10 (BatchQueue) | Unified OAL + MAL | | L2 Persistence (OAL + MAL) | 3 (DataCarrier) | 4 (BatchQueue) | Unified OAL + MAL | | TopN Persistence | 4 (DataCarrier) | 1 (BatchQueue) | | | gRPC Remote Client | 1 (DataCarrier) | 1 (BatchQueue) | Per peer | | Armeria HTTP event loop | 20 | 5 | `max(5, cores/4)` shared group | | Armeria HTTP handler | on-demand platform | - | Virtual threads on JDK 25+ | | gRPC event loop | 10 | 10 | Unchanged | | gRPC handler | ~9 (platform) | - | Virtual threads on JDK 25+ | | ForkJoinPool (Virtual Thread carrier) | 0 | ~10 | JDK 25+ virtual thread scheduler | | HttpClient-SelectorManager | 4 | 2 | SharedKubernetesClient | | Schedulers + others | ~24 | ~24 | Mostly unchanged | | **Total (OAP threads)** | **150+** | **~72** | **~50% reduction, stable under load** | ### Key Changes 1. **DataCarrier → BatchQueue**: L1+L2+TopN+Remote reduced from 34 → 16 threads with unified OAL+MAL queues 2. **Armeria event loop**: 20 → 5 threads via `max(5, cores/4)` shared `EventLoopGroup` across all 4 HTTP servers 3. **Virtual threads (JDK 25+)**: gRPC and Armeria HTTP handlers now use virtual threads — platform handler threads eliminated, all share ~10 ForkJoinPool carrier threads 4. **SharedKubernetesClient**: HttpClient-SelectorManager 4 → 2 threads by replacing 9 separate `KubernetesClientBuilder().build()` calls with a singleton Thread counts are stable across all 5 dump rounds and consistent between both OAP pods (only transient pools like `grpc-default-executor` and `ForkJoinPool.commonPool` fluctuate by 1-2). Report: `benchmarks/reports/istio-cluster_oap-banyandb/thread-analysis/20260222-214827/` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
