beauli opened a new issue, #60108: URL: https://github.com/apache/doris/issues/60108
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.0.2 ### What's Wrong? This bug happens after upgrade doris from 4.0.0 to 4.0.2. We observed abnormal thread growth in Doris FE. The number of threads continuously increases until it reaches more than 130,000, which eventually exhausts system memory and leads to os::commit_memory failed errors. Most of these threads are named sdk-ScheduledExecutor-* and are in WAITING (parking) state. They are created by ScheduledThreadPoolExecutor and remain idle, waiting on DelayedWorkQueue.take(). ### What You Expected? Thread pools should be reused and limited in size. Idle ScheduledThreadPoolExecutor threads should not grow indefinitely. ### How to Reproduce? Start Doris FE with JDK 17. Monitor thread count using jstack or jcmd. Observe continuous growth of threads named sdk-ScheduledExecutor-*. Eventually, memory usage exceeds physical RAM and Doris FE crashes. ### Anything Else? Example thread dump: "sdk-ScheduledExecutor-3296-3" #13890 daemon prio=5 os_prio=0 cpu=0.12ms elapsed=278.37s tid=0x00007fa6055380b0 nid=0x3e51 waiting on condition [0x00007fa2959d2000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park([email protected]/Native Method) - parking to wait for <0x0000000213a14f60> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506) at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3463) at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3434) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1623) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:1177) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take([email protected]/ScheduledThreadPoolExecutor.java:899) at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062) at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122) at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635) at java.lang.Thread.run([email protected]/Thread.java:833) ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
