Sai Hemanth Gantasala has posted comments on this change. ( http://gerrit.cloudera.org:8080/19155 )
Change subject: IMPALA-11626: Handle COMMIT_COMPACTION_EVENT from HMS ...................................................................... Patch Set 9: (4 comments) http://gerrit.cloudera.org:8080/#/c/19155/7/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/19155/7/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@577 PS7, Line 577: { > Sorry that I'm not clear how the MetastoreNotificationException will be thr Ack http://gerrit.cloudera.org:8080/#/c/19155/9/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/19155/9/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4456 PS9, Line 4456: if (table == null && !(table instanceof IncompleteTable) && : (table instanceof HdfsTable)) { > hmm, if 'table' is null, it can't be an instance of HdfsTable. This conditi Ack http://gerrit.cloudera.org:8080/#/c/19155/9/fe/src/test/resources/hive-site.xml.py File fe/src/test/resources/hive-site.xml.py: http://gerrit.cloudera.org:8080/#/c/19155/9/fe/src/test/resources/hive-site.xml.py@167 PS9, Line 167: 'hive.compactor.delta.num.threshold': 1, > Could you add a comment about why we need this for all the tests? Yeah, this would probably introduce a data-loading performance issue. Instead, I'm doing at least 10 inserts so that compaction would trigger for sure. The default compaction threshold is 10. http://gerrit.cloudera.org:8080/#/c/19155/9/tests/custom_cluster/test_events_custom_configs.py File tests/custom_cluster/test_events_custom_configs.py: http://gerrit.cloudera.org:8080/#/c/19155/9/tests/custom_cluster/test_events_custom_configs.py@327 PS9, Line 327: assert EventProcessorUtils.get_event_processor_status() == "ACTIVE" > We also need tests on unpartitioned tables and table level compaction on pa Added test for unpartitioned tables, For partitioned tables, it seems that we cannot trigger compaction at the table label. Compaction expects partition spec if it is a partitioned table. -- To view, visit http://gerrit.cloudera.org:8080/19155 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I464faedb4e3bbcd417bab2e3cb0d57e339d42605 Gerrit-Change-Number: 19155 Gerrit-PatchSet: 9 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Tue, 21 Feb 2023 19:16:50 +0000 Gerrit-HasComments: Yes
