wangyum opened a new pull request, #47826:
URL: https://github.com/apache/spark/pull/47826
### What changes were proposed in this pull request?
Shutdown timeout thread while cleaning up `SparkExecuteStatementOperation`.
### Why are the changes needed?
Avoid driver memory leak if query timeout is configured. For example, there
are 4127 `SparkExecuteStatementOperation` instances in the Driver:
```
jmap -histo 398 | grep SparkExecuteStatementOperation
308: 4127 1122544
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation
563: 4127 363176
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$ErrRowCountType$
876: 4127 132064
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$1
2101: 333 7992
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$5
3106: 32 1024
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2
3303: 32 768
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$3755/0x00000008021fe800
3304: 32 768
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$anon$2$$anon$3
3961: 9 360
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$5398/0x0000000802523900
3962: 9 360
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$5399/0x0000000802523bd8
20239: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$
20240: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$$Lambda$5397/0x000000080251e180
20241: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$11228/0x000000080306ba38
20242: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$11230/0x00000008032962d8
20243: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$11231/0x00000008032966b8
20244: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$5363/0x0000000802509470
20245: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$5367/0x000000080250a618
20246: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$6475/0x00000008026fda40
20247: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$7355/0x00000008028aa180
20248: 1 16
org.apache.spark.sql.hive.thriftserver.SparkExecuteStatementOperation$$Lambda$7356/0x00000008028aa560
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Manual test.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
