Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/21143 )
Change subject: [WIP] IMPALA-12856: Event processor should ignore processing partition with empty partition values ...................................................................... Patch Set 1: (6 comments) LGTM, just have some minor comments to make this a FE-only change. http://gerrit.cloudera.org:8080/#/c/21143/1/be/src/catalog/catalog-server.cc File be/src/catalog/catalog-server.cc: http://gerrit.cloudera.org:8080/#/c/21143/1/be/src/catalog/catalog-server.cc@192 PS1, Line 192: DEFINE_bool_hidden(is_return_empty_partition_values, false, "This configuration is used " We don't need this for FE tests. We can use the debug_actions flag and add an action appropriately. http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2846 PS1, Line 2846: public int reloadPartitionsFromNames(IMetaStoreClient client, nit: it seems this is only used by the event-processor. We can pass in the eventId and eventType to improve the logging if that's not too hard. http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2858 PS1, Line 2858: warn nit: error() seems more suitable. This can also be simplified as LOG.error("Received partition with empty values: {}. \nIgnoring" + " reloading the partition.", partition)); http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2891 PS1, Line 2891: public int reloadPartitionsFromEvent(IMetaStoreClient client, nit: we can pass in the eventId and eventType to improve the logging http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java File fe/src/main/java/org/apache/impala/service/BackendConfig.java: http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/service/BackendConfig.java@461 PS1, Line 461: public String debugActions() { return backendCfg_.debug_actions; } We can add setDebugAction() for the new FE test. http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java File fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java: http://gerrit.cloudera.org:8080/#/c/21143/1/fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java@212 PS1, Line 212: if (BackendConfig.INSTANCE.getIsReturnEmptyPartitionValues()) { and use the debug action here like DebugUtils.hasDebugAction(BackendConfig.INSTANCE.debugActions(), DebugUtils.MOCK_EMPTY_PARTITION_VALUES) -- To view, visit http://gerrit.cloudera.org:8080/21143 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id2469930ccd74948325f1723bd8b2bd6aad02d09 Gerrit-Change-Number: 21143 Gerrit-PatchSet: 1 Gerrit-Owner: Sai Hemanth Gantasala <[email protected]> Gerrit-Reviewer: Anonymous Coward <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Sai Hemanth Gantasala <[email protected]> Gerrit-Comment-Date: Wed, 13 Mar 2024 12:44:09 +0000 Gerrit-HasComments: Yes
