Yida Wu has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/22549 )
Change subject: IMPALA-13798: Clear of host-level remote scratch dir on shutdown ...................................................................... IMPALA-13798: Clear of host-level remote scratch dir on shutdown IMPALA-13677 introduces cleanup remote scratch files on startup, and also introduces a host-level directory to do easier cleanup. An empty host-level directory in remote storage does not use resources, but it can stay there forever if a host goes offline permanently. To improve this, this patch removed the remote host-level temporary directory on shutdown. This helps prevent too many empty directories left in the remote scratch path, especially for hosts that never restart. Changed the flag remote_scratch_cleanup_on_startup to remote_scratch_cleanup_on_startstop, so that this flag also controls to cleanup on shutdown as well. Tests: Passed exhaustive tests. Added an e2e testcase test_scratch_dirs_remote_dir_removal_on_shutdown. Change-Id: Ic8f446894afdf975630aef80a9d964a9a78d3b46 --- M be/src/runtime/tmp-file-mgr-internal.h M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h M be/src/service/impala-server.cc M tests/custom_cluster/test_scratch_disk.py 6 files changed, 87 insertions(+), 10 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/22549/2 -- To view, visit http://gerrit.cloudera.org:8080/22549 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic8f446894afdf975630aef80a9d964a9a78d3b46 Gerrit-Change-Number: 22549 Gerrit-PatchSet: 2 Gerrit-Owner: Yida Wu <wydbaggio...@gmail.com> Gerrit-Reviewer: Abhishek Rawat <ara...@cloudera.com> Gerrit-Reviewer: Daniel Becker <daniel.bec...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
