This is an automated email from the ASF dual-hosted git repository.
yumwang pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new 4e4eb6f1ed5 [SPARK-39400][SQL] spark-sql should remove hive resource
dir in all case
4e4eb6f1ed5 is described below
commit 4e4eb6f1ed5ff0d3caa7f424d2df23f186bf32a2
Author: Angerszhuuuu <[email protected]>
AuthorDate: Thu Jun 9 08:06:13 2022 +0800
[SPARK-39400][SQL] spark-sql should remove hive resource dir in all case
### What changes were proposed in this pull request?
In current code, when we use `spark-sql` `-e` , `-f` or use `ctrl + c` to
close `spark-sql` session, will remain hive session resource dir under `/tmp`
path, this pr help to clean this files
### Why are the changes needed?
Clean remained files
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Manuel tested
Closes #36786 from AngersZhuuuu/SPARK-39400.
Lead-authored-by: Angerszhuuuu <[email protected]>
Co-authored-by: AngersZhuuuu <[email protected]>
Signed-off-by: Yuming Wang <[email protected]>
---
.../apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git
a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
index fccb2a65273..d40cf73be63 100644
---
a/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
+++
b/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLCLIDriver.scala
@@ -103,10 +103,13 @@ private[hive] object SparkSQLCLIDriver extends Logging {
sessionState.info = new PrintStream(System.err, true, UTF_8.name())
sessionState.err = new PrintStream(System.err, true, UTF_8.name())
} catch {
- case e: UnsupportedEncodingException => System.exit(ERROR_PATH_NOT_FOUND)
+ case e: UnsupportedEncodingException =>
+ sessionState.close()
+ System.exit(ERROR_PATH_NOT_FOUND)
}
if (!oproc.process_stage2(sessionState)) {
+ sessionState.close()
System.exit(ERROR_MISUSE_SHELL_BUILTIN)
}
@@ -140,7 +143,10 @@ private[hive] object SparkSQLCLIDriver extends Logging {
SessionState.setCurrentSessionState(sessionState)
// Clean up after we exit
- ShutdownHookManager.addShutdownHook { () => SparkSQLEnv.stop() }
+ ShutdownHookManager.addShutdownHook { () =>
+ sessionState.close()
+ SparkSQLEnv.stop()
+ }
if (isRemoteMode(sessionState)) {
// Hive 1.2 + not supported in CLI
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
