Github user gaborgsomogyi commented on a diff in the pull request:
https://github.com/apache/spark/pull/19035#discussion_r158050733
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala
---
@@ -435,6 +435,18 @@ case class InsertIntoHiveTable(
logWarning(s"Unable to delete staging directory: $stagingDir.\n" +
e)
}
+ //delete the tmpLocation dir
+ try {
+ val fs = tmpLocation.getFileSystem(hadoopConf)
+ if (fs.delete(tmpLocation, true)) {
+ // If we successfully delete the tmpLocation dir, remove it from
FileSystem's cache.
+ fs.cancelDeleteOnExit(tmpLocation)
+ }
+ } catch {
+ case NonFatal(e) =>
+ logWarning(s"Unable to delete tmpLocation directory:" +
tmpLocation.toString + "\n" + e)
--- End diff --
This can be compressed a bit:
`
logWarning(s"Unable to delete tmpLocation directory: $tmpLocation\n$e")
`
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
