sunchao commented on a change in pull request #29387:
URL: https://github.com/apache/spark/pull/29387#discussion_r475113156
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala
##########
@@ -2712,6 +2712,16 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val TRUNCATE_TRASH_ENABLED =
+ buildConf("spark.sql.truncate.trash.enabled")
+ .doc("This configuration decides when truncating table, whether data
files will be moved " +
+ "to trash directory or deleted permanently. The trash retention time
is controlled by " +
+ "fs.trash.interval, and in default, the server side configuration
value takes " +
+ "precedence over the client-side one. Note that if fs.trash.interval
is non-positive, " +
+ "this will be a no-op and log a warning message.")
Review comment:
Yes. This is because when `fs.trash.interval` is not positive, Hadoop
side will consider trash as disabled and will not delete the data. See
[here](https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/TrashPolicyDefault.java#L125).
Currently this just logs a warning but we could consider another flag to hard
delete the data instead.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
