Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/15818 )
Change subject: IMPALA-9512: Full ACID Milestone 2: Validate rows against the valid write id list ...................................................................... Patch Set 13: Code-Review+2 (7 comments) I played around with this patch. It works well for LocalCatalog mode. Also verified test_acid_row_validation.py in LocalCatalog mode. Just left some nit comments. Thanks for the work! http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@611 PS13, Line 611: new BitSet() nit: move this to the else-clase? http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java File fe/src/main/java/org/apache/impala/catalog/HdfsTable.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/HdfsTable.java@2096 PS13, Line 2096: sw.elapsed(TimeUnit.NANOSECONDS) I think we should return this and add it into storageMetadataLoadTime_ http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@45 PS13, Line 45: import org.apache.hadoop.hive.metastore.api.TableMeta; nit: unused import http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/CatalogdMetaProvider.java@62 PS13, Line 62: import org.apache.impala.compat.MetastoreShim; nit: unused import http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java@405 PS13, Line 405: throw new NotImplementedException( We can call HMS API getValidWriteIds(fullTableName) to get this directly. http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java@331 PS13, Line 331: if (ref_ != null) { Could you leave a comment about when ref_ will be null? It confused me at first. I think the only case is when the table is the target table of a CTAS statement. http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java File fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java: http://gerrit.cloudera.org:8080/#/c/15818/13/fe/src/main/java/org/apache/impala/catalog/local/MetaProvider.java@23 PS13, Line 23: import org.apache.commons.lang.NotImplementedException; nit: unused import -- To view, visit http://gerrit.cloudera.org:8080/15818 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5ed74585a2d73ebbcee763b0545be4412926299d Gerrit-Change-Number: 15818 Gerrit-PatchSet: 13 Gerrit-Owner: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Comment-Date: Tue, 19 May 2020 09:06:27 +0000 Gerrit-HasComments: Yes
