Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/17857 )
Change subject: IMPALA-10914: Consistently schedule scan ranges for Iceberg tables ...................................................................... Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/17857/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/17857/1//COMMIT_MSG@11 PS1, Line 11: and the HDFS : block locations were not consistent across the reload > In this case, if we sort the block locations, would it make things better? Sorting is one possibility, but I think we shouldn't reload file descriptors if we don't have to. And since we should always reuse file descriptors, sorting becomes unnecessary and could potentially cause perf regression. With this patch coordinators access the Iceberg snapshot that is loaded by CatalogD. When CatalogD refreshes the Iceberg table, the coordinators will use the new version. This behavior is analog to what we do for other tables. http://gerrit.cloudera.org:8080/#/c/17857/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/17857/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1295 PS1, Line 1295: if (!needsToUpdateHms) { : loadTableMetadata(tbl, newCatalogVersion, true, true, null, "ALTER Iceberg TABLE " + : params.getAlter_type().name()); : catalog_.addVersionsForInflightEvents(false, tbl, newCatalogVersion); : addTableToCatalogUpdate(tbl, wantMinimalResult, response.result); : return false; : } > I see. Maybe add a comment to indicate it is not transactional and the work Added a comment with some explanation. http://gerrit.cloudera.org:8080/#/c/17857/2/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/17857/2/tests/custom_cluster/test_events_custom_configs.py@270 PS2, Line 270: n > flake8: E225 missing whitespace around operator Done -- To view, visit http://gerrit.cloudera.org:8080/17857 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ibb8216b37d350469b573dad7fcefdd0ee0599ed5 Gerrit-Change-Number: 17857 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Tue, 28 Sep 2021 07:46:31 +0000 Gerrit-HasComments: Yes
