tianshuang opened a new pull request, #36741: URL: https://github.com/apache/spark/pull/36741
### What changes were proposed in this pull request? * Fixed memory leak caused `RawStore` cleanup not to take effect due to different `threadLocalMS` instances being manipulated * Fixed memory leak caused by `SparkExecuteStatementOperation` not calling `ThreadWithGarbageCleanup#cacheThreadLocalRawStore`. ### Why are the changes needed? We need to manipulate the same `threadLocalMS` instance for the `RawStore` cleanup mechanism to take effect, I have tried other fixes, manipulating the `threadLocalMS` instance in `HMSHandler`(loaded by IsolatedClassLoader$$anon$1) may be a suitable fix , I compiled Spark 2.4.4 by myself using this fix and verified it in the production environment, confirming that the memory leak is fixed. Reference: [SPARK-39357 pmCache memory leak caused by IsolatedClassLoader - ASF JIRA](https://issues.apache.org/jira/browse/SPARK-39357) ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? I have added `SparkSQLEnv.sqlContext = sqlContext` to `SharedThriftServer#startThriftServer`, I don't know if this is the right way, in order to pass the unit test, please give suggestions, thank you. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
