Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/22189 )
Change subject: IMPALA-13501: Clean up uncommitted Iceberg files after validation check failure ...................................................................... Patch Set 5: (2 comments) http://gerrit.cloudera.org:8080/#/c/22189/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/22189/5/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@7588 PS5, Line 7588: throw new ImpalaRuntimeException(e.getMessage(), e); I was wondering whether it would be useful to log the names of the files that were cleaned up - this could be good for example in case the table is corrupted because of missing files: we could rule out that this cleanup was the one that (incorrectly) deleted the file, or we could confirm if there is a bug with the cleanup. The argument against it is that it could potentially be hundreds of files. Do you know if it is possible to recover this info from filesystem logs as well? Do they show that it was Impala that deleted these files? http://gerrit.cloudera.org:8080/#/c/22189/5/tests/stress/test_update_stress.py File tests/stress/test_update_stress.py: http://gerrit.cloudera.org:8080/#/c/22189/5/tests/stress/test_update_stress.py@315 PS5, Line 315: assert len(data_files_in_tbl.data) == len(data_files_on_fs_rows) - 1 Would it be difficult to check that the files are actually the same, not only the number of files? -- To view, visit http://gerrit.cloudera.org:8080/22189 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibe59546ebf3c639b75b53dfa1daba37cef50eb21 Gerrit-Change-Number: 22189 Gerrit-PatchSet: 5 Gerrit-Owner: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Noemi Pap-Takacs <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 02 Jan 2025 12:27:15 +0000 Gerrit-HasComments: Yes
