Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/23109 )
Change subject: IMPALA-14189: Cleanup subdirectories in truncate/insert overwrite ...................................................................... Patch Set 7: (7 comments) http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc File be/src/runtime/dml-exec-state.cc: http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@189 PS7, Line 189: DeleteDirData This isn't used in case of partition dir - the name could reflect this, e.g. DeleteUnpartitionedDirData http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@198 PS7, Line 198: hdfsListDirectory(fs_connection, path.c_str(), &num_files); nit: +2 indentation http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@202 PS7, Line 202: hdfsListDirectory(fs_connection, path.c_str(), &num_files); nit: +2 indentation http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@217 PS7, Line 217: boost::filesystem::path(existing_files[i].mName).filename().string(); nit: +2 indentation http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@217 PS7, Line 217: : if (!IsHiddenFile(filename)) { The hidden file prefixes are included in FLAGS_ignored_dir_prefix_list, so this is redundant. I would check the prefix for each path, regardless of being dir or file http://gerrit.cloudera.org:8080/#/c/23109/7/be/src/runtime/dml-exec-state.cc@287 PS7, Line 287: partition_fs_connection, part_path, &partition_create_ops)); nit: -2 indentation http://gerrit.cloudera.org:8080/#/c/23109/7/tests/metadata/test_recursive_listing.py File tests/metadata/test_recursive_listing.py: http://gerrit.cloudera.org:8080/#/c/23109/7/tests/metadata/test_recursive_listing.py@158 PS7, Line 158: self.filesystem_client.create_file("{0}/dir1/file1.txt".format(part_path), "file1") can you also add a hidden dir and a file in it? in the partitioned case it should be deleted, in the unpartitioned it should be kept -- To view, visit http://gerrit.cloudera.org:8080/23109 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ib3ee6cba3a4f41ad9997d0d4f45e1d28af36b72b Gerrit-Change-Number: 23109 Gerrit-PatchSet: 7 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Comment-Date: Thu, 10 Jul 2025 15:20:52 +0000 Gerrit-HasComments: Yes
