[
https://issues.apache.org/jira/browse/HIVE-27317?focusedWorklogId=860598&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-860598
]
ASF GitHub Bot logged work on HIVE-27317:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 04/May/23 17:21
Start Date: 04/May/23 17:21
Worklog Time Spent: 10m
Work Description: sercanCyberVision opened a new pull request, #4293:
URL: https://github.com/apache/hive/pull/4293
### What changes were proposed in this pull request?
When `ClearDanglingScratchDir` service identifies the dangling sessions to
clean HDFS FS, we will be cleaning files/dirs in
`HiveConf.ConfVars.LOCALSCRATCHDIR` (local FS) as well.
### Why are the changes needed?
When Hive session is killed, no chance for shutdown hook to clean-up tmp
files. This causes accumulation of tmp files/dirs in local FS as below;
```
> ll /tmp/user/97c4ef50-5e80-480e-a6f0-4f779050852b*
drwx----
Issue Time Tracking
-------------------
Worklog Id: (was: 860598)
Remaining Estimate: 0h
Time Spent: 10m
> Temporary (local) session files cleanup improvements
> ----------------------------------------------------
>
> Key: HIVE-27317
> URL: https://issues.apache.org/jira/browse/HIVE-27317
> Project: Hive
> Issue Type: Improvement
> Reporter: Sercan Tekin
> Assignee: Sercan Tekin
> Priority: Major
> Attachments: HIVE-27317.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When Hive session is killed, no chance for shutdown hook to clean-up tmp
> files.
> There is a Hive service to clean residual files
> https://issues.apache.org/jira/browse/HIVE-13429, and later on its execution
> is scheduled inside HS2 https://issues.apache.org/jira/browse/HIVE-15068 to
> make sure not to leave any temp file behind. But this service cleans up only
> HDFS temp files, there are still residual files/dirs in
> *HiveConf.ConfVars.LOCALSCRATCHDIR* location as follows;
> {code:java}
> > ll /tmp/user/97c4ef50-5e80-480e-a6f0-4f779050852b*
> drwx------ 2 user user 4096 Oct 29 10:09 97c4ef50-5e80-480e-a6f0-4f779050852b
> -rw------- 1 user user 0 Oct 29 10:09
> 97c4ef50-5e80-480e-a6f0-4f779050852b10571819313894728966.pipeout
> -rw------- 1 user user 0 Oct 29 10:09
> 97c4ef50-5e80-480e-a6f0-4f779050852b16013956055489853961.pipeout
> -rw------- 1 user user 0 Oct 29 10:09
> 97c4ef50-5e80-480e-a6f0-4f779050852b4383913570068173450.pipeout
> -rw------- 1 user user 0 Oct 29 10:09
> 97c4ef50-5e80-480e-a6f0-4f779050852b889740171428672108.pipeout {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
