veghlaci05 commented on code in PR #4293:
URL: https://github.com/apache/hive/pull/4293#discussion_r1218041617
##########
ql/src/java/org/apache/hadoop/hive/ql/session/ClearDanglingScratchDir.java:
##########
@@ -236,4 +243,29 @@ static Options createOptions() {
return result;
}
+
+ /**
+ * While deleting dangling scratch dirs from hdfs, we can clean
corresponding local files as well
+ * @param sessionName prefix to determine removable tmp files
+ * @param localTmpdir local tmp file location
+ */
+ public void removeLocalTmpFiles(String sessionName, String localTmpdir) {
+ File[] files = new File(localTmpdir).listFiles();
+ boolean success;
+ if (files != null) {
+ for (File file : files) {
+ success = false;
+ if (file.getName().startsWith(sessionName)) {
+ success = file.delete();
Review Comment:
A file without write permission can permanently prevent subsequent files
from being cleaned up. Either handle SecurityException, or check access with
`File.canWrite()` method.
##########
ql/src/java/org/apache/hadoop/hive/ql/session/ClearDanglingScratchDir.java:
##########
@@ -236,4 +243,29 @@ static Options createOptions() {
return result;
}
+
+ /**
+ * While deleting dangling scratch dirs from hdfs, we can clean
corresponding local files as well
+ * @param sessionName prefix to determine removable tmp files
+ * @param localTmpdir local tmp file location
+ */
+ public void removeLocalTmpFiles(String sessionName, String localTmpdir) {
Review Comment:
Should be private
##########
ql/src/java/org/apache/hadoop/hive/ql/session/ClearDanglingScratchDir.java:
##########
@@ -236,4 +243,29 @@ static Options createOptions() {
return result;
}
+
+ /**
+ * While deleting dangling scratch dirs from hdfs, we can clean
corresponding local files as well
+ * @param sessionName prefix to determine removable tmp files
+ * @param localTmpdir local tmp file location
+ */
+ public void removeLocalTmpFiles(String sessionName, String localTmpdir) {
+ File[] files = new File(localTmpdir).listFiles();
+ boolean success;
+ if (files != null) {
+ for (File file : files) {
+ success = false;
+ if (file.getName().startsWith(sessionName)) {
Review Comment:
You may use File.listFiles() method with a filter instead of this construct.
It's not necessary to iterate through all the files again and again inside the
outer loop.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
