Vihang Karajgaonkar has posted comments on this change. ( http://gerrit.cloudera.org:8080/17313 )
Change subject: IMPALA-10656: Fire insert events before commit ...................................................................... Patch Set 11: (2 comments) http://gerrit.cloudera.org:8080/#/c/17313/11/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/17313/11/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4867 PS11, Line 4867: of non-transactional tables if there are no existing partitions affected by this : * insert. Can you add a comment about the behavior of the transactional tables here. May be adding something like below will be useful. "This method is replicating what Hive does in case a table or partition is inserted into. There are 2 cases: 1. If the table is transactional, we should first generate ADD_PARTITION events for new partitions which are generated. This is taken care of in the updateCatalog method. Additionally, for each partition including existing partitions which were inserted into, this method creates a ACID_WRITE event using the HMS API addWriteNotificationLog. 2. If the table is not transactional, this method generates INSERT_EVENT for only the pre-existing partitions which were inserted into. This is in-line with what hive does (see https://github.com/apache/hive/blob/25892ea409b3351acbd409d1f66f399a43614798/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L3251) http://gerrit.cloudera.org:8080/#/c/17313/11/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@4885 PS11, Line 4885: insert event type. Can you please add the following to the comment -- "This would show up in HMS as a ACID_WRITE event" -- To view, visit http://gerrit.cloudera.org:8080/17313 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I2ed812dbcb5f55efff3a910a3daeeb76cd3295b9 Gerrit-Change-Number: 17313 Gerrit-PatchSet: 11 Gerrit-Owner: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Thu, 22 Apr 2021 22:10:41 +0000 Gerrit-HasComments: Yes
