Yida Wu has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/20852 )
Change subject: IMPALA-12681: Release file descriptors for partially written temporary files ...................................................................... IMPALA-12681: Release file descriptors for partially written temporary files This patch fixes a bug where partially written temporary files are removed without releasing the file descriptors. This patch fixes the bug by adding a call to Close() of the local file writer during the Delete() of the DiskFile class, which could be called when the local buffer file is being evicted or the query ends, ensuring proper release of the file handle. Testing: Manual testing of remote spilling, configure a local path as the local buffer and a S3 remote path for remote storage in scratch dir. Ran tpcds q78 with a low buffer_pool_limit in the query option to trigger spilling, and checked the path of local buffer path of the scratch dir that the temporary files did exist during running the query. Without the fix, after running the query, we can find references of deleted files in /proc/x/fd. With the patch that there are no references to the deleted files in the /proc/x/fd/. Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 --- M be/src/runtime/io/disk-file.cc M be/src/runtime/io/local-file-writer.cc M be/src/runtime/io/local-file-writer.h 3 files changed, 11 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/52/20852/4 -- To view, visit http://gerrit.cloudera.org:8080/20852 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I58a2bac419ced806d6f5a32bcdf24d79e078ab14 Gerrit-Change-Number: 20852 Gerrit-PatchSet: 4 Gerrit-Owner: Yida Wu <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Yida Wu <[email protected]>
