Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16551 )
Change subject: IMPALA-10165: Implement Bucket and Truncate partition transforms for Iceberg tables ...................................................................... Patch Set 4: (10 comments) http://gerrit.cloudera.org:8080/#/c/16551/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16551/2//COMMIT_MSG@9 PS2, Line 9: This patch adds support for Iceberg Bucket and Truncate partition : transforms. Both accept a parameter: number of buckets and width : respectively. : : Usage: : CREATE TABLE tbl_name (i int, p1 int, p2 timestamp) : PARTITION BY SPEC ( : p1 BUCKET 10, : p1 TRUNCATE 5 : ) STORED AS ICEBERG : TBLPROPERTIES ('iceberg.catalog'='hadoop.tables'); > I saw you add test cases, maybe add some messages about test info? such as: Done http://gerrit.cloudera.org:8080/#/c/16551/2/common/thrift/CatalogObjects.thrift File common/thrift/CatalogObjects.thrift: http://gerrit.cloudera.org:8080/#/c/16551/2/common/thrift/CatalogObjects.thrift@527 PS2, Line 527: transform_param > Maybe add some explains for this variable, since this variable is only vali Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java File fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionField.java@42 PS2, Line 42: Partition transform > IcebergPartitionTransform class is not only partition type, but also contai Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionTransform.java File fe/src/main/java/org/apache/impala/analysis/IcebergPartitionTransform.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionTransform.java@24 PS2, Line 24: /** > Please add some comments. Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/analysis/IcebergPartitionTransform.java@54 PS2, Line 54: > nit: Only Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java File fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@247 PS2, Line 247: Iceber > nit: Iceberg Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@247 PS2, Line 247: curren > nit: current? Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/util/IcebergUtil.java File fe/src/main/java/org/apache/impala/util/IcebergUtil.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/main/java/org/apache/impala/util/IcebergUtil.java@345 PS2, Line 345: in > nit: in Done http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeStmtsTest.java@4846 PS2, Line 4846: AnalysisError("CREATE TABLE tbl1 (i int, p1 int, p2 timestamp) " + : "PARTITION BY SPEC (p1 BUCKET, p2 DAY) STORED AS ICEBERG" + tblProperties, : "BUCKET and TRUNCATE partition transforms should have a parameter."); > You could add this test for TRUNCATE as well. As Truncate had to be handled differently in the .cup file (It was a reserved keyword) not giving a parameter will result a parse error instead of an AnalysisException. I added a test to cover this in the ParserTests Output when not giving TRUNCATE a parameter: ERROR: ParseException: Syntax error in line 1: ...ITION BY SPEC (p1 TRUNCATE) STORED AS ICEBERG TBLPROPE... ^ Encountered: ) Expected: INTEGER LITERAL CAUSED BY: Exception: Syntax error http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/test/java/org/apache/impala/analysis/ParserTest.java File fe/src/test/java/org/apache/impala/analysis/ParserTest.java: http://gerrit.cloudera.org:8080/#/c/16551/2/fe/src/test/java/org/apache/impala/analysis/ParserTest.java@2989 PS2, Line 2989: ParserError("CREATE TABLE tbl1 (i int, p1 int) PARTITION BY SPEC (p1 TRUNCATE) " + : "STORED AS ICEBERG TBLPROPERTIES ('iceberg.catalog'='hadoop.tables')"); > Also add a test for 'BUCKET' BUCKET without parameter causes an AnalysisException hence it is included in the AnalyzerTests. Thi is because in the .cup I had to handle TRUNCATe differently because it is an already reserved keyword. -- To view, visit http://gerrit.cloudera.org:8080/16551 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idc75cd23045b274885607c45886319f4f6da19de Gerrit-Change-Number: 16551 Gerrit-PatchSet: 4 Gerrit-Owner: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: wangsheng <sky...@163.com> Gerrit-Comment-Date: Fri, 09 Oct 2020 11:56:18 +0000 Gerrit-HasComments: Yes
