[
https://issues.apache.org/jira/browse/DRILL-2100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15251832#comment-15251832
]
ASF GitHub Bot commented on DRILL-2100:
---------------------------------------
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/454#discussion_r60572146
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/physical/impl/xsort/ExternalSortBatch.java
---
@@ -550,7 +565,15 @@ public BatchGroup mergeAndSpill(LinkedList<BatchGroup>
batchGroups) throws Schem
c1.buildSchema(BatchSchema.SelectionVectorMode.NONE);
c1.setRecordCount(count);
- String outputFile = Joiner.on("/").join(dirs.next(), fileName,
spillCount++);
+ String spillDir = dirs.next();
+ Path currSpillPath = new Path(Joiner.on("/").join(spillDir, fileName));
+ currSpillDirs.add(currSpillPath);
+ String outputFile = Joiner.on("/").join(currSpillPath, spillCount++);
+ try {
+ fs.deleteOnExit(currSpillPath);
+ } catch (IOException e) {
+ throw new RuntimeException(e);
--- End diff --
Agree. An exception throw was replaced with:
```
// since this is meant to be used in a batches's spilling, we don't
propagate the exception
final String message = "Unable to mark spill directory \"" + currSpillPath
+ "\" to delete on exit";
System.err.println(message + '.');
logger.warn(message, e);
```
> Drill not deleting spooling files
> ---------------------------------
>
> Key: DRILL-2100
> URL: https://issues.apache.org/jira/browse/DRILL-2100
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Relational Operators
> Affects Versions: 0.8.0
> Reporter: Abhishek Girish
> Assignee: Deneche A. Hakim
> Fix For: 1.7.0
>
>
> Currently, after forcing queries to use an external sort by switching off
> hash join/agg causes spill-to-disk files accumulating.
> This causes issues with disk space availability when the spill is configured
> to be on the local file system (/tmp/drill). Also not optimal when configured
> to use DFS (custom).
> Drill must clean up all temporary files created after a query completes or
> after a drillbit restart.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
