Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16446 )
Change subject: IMPALA-10164: Supporting HadoopCatalog for Iceberg table ...................................................................... Patch Set 15: Code-Review+1 (6 comments) Thanks for applying the changes! I think the change looks good, I only had some minor comments. http://gerrit.cloudera.org:8080/#/c/16446/15//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/16446/15//COMMIT_MSG@29 PS15, Line 29: table_name Probably 'table_identifier' would be more appropriate as it would follow Iceberg's terminology. http://gerrit.cloudera.org:8080/#/c/16446/15/fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java File fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java: http://gerrit.cloudera.org:8080/#/c/16446/15/fe/src/main/java/org/apache/impala/analysis/CreateTableStmt.java@592 PS15, Line 592: // Only external table can set 'iceberg.table_name' Are we sure that we want to disallow this? I'm thinking of a use case when users use Iceberg's HadoopCatalog with more fine grained table identifiers, e.g. 'organization.database.table_name'. I think Iceberg allows arbitraly deep table identifiers. So users might also want to create some tables managed by Impala, but with the above schema, e.g.: CREATE TABLE my_hms_db.my_ice_table (i int) STORED AS ICEBERG TBLPROPERTIES( 'iceberg.catalog' = 'hadoop.catalog', 'iceberg.catalog_location' = '/warehouse/ice-catalog', 'iceberg.table_identifier' = 'my_organization.my_ice_db.my_table'); So if 'table_identifier' is missing, we should use 'db_name.tbl_name' as Iceberg table identifier, otherwise we should use the explicit param. Though I'm also happy with the current behavior if you don't want to deal with it in this patch. http://gerrit.cloudera.org:8080/#/c/16446/15/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/16446/15/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2582 PS15, Line 2582: newTable.getSd().setLocation(String.format("%s/%s/%s", location, : newTable.getDbName(), newTable.getTableName())); Instead of figuring out the location, we should get it from Iceberg: https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/Table.java#L94 So if Iceberg change its location-computing algorithm in the future, HMS will still point to the right place. http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/datasets/functional/functional_schema_template.sql File testdata/datasets/functional/functional_schema_template.sql: http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/datasets/functional/functional_schema_template.sql@2969 PS15, Line 2969: 'iceberg.catalog_location' nit: please start new line http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test File testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test: http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test@154 PS15, Line 154: 'iceberg.catalog_location'='/$DATABASE/hadoop_catalog_test'); nit: please try to keep lines less than 90 chars in the QUERY section. http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/workloads/functional-query/queries/QueryTest/show-create-table.test File testdata/workloads/functional-query/queries/QueryTest/show-create-table.test: http://gerrit.cloudera.org:8080/#/c/16446/15/testdata/workloads/functional-query/queries/QueryTest/show-create-table.test@585 PS15, Line 585: 'iceberg.catalog_location'='$$location_uri$$') nit: please try to keep lines less than 90 chars. -- To view, visit http://gerrit.cloudera.org:8080/16446 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic1893c50a633ca22d4bca6726c9937b026f5d5ef Gerrit-Change-Number: 16446 Gerrit-PatchSet: 15 Gerrit-Owner: wangsheng <sky...@163.com> Gerrit-Reviewer: Gabor Kaszab <gaborkas...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com> Gerrit-Reviewer: wangsheng <sky...@163.com> Gerrit-Comment-Date: Mon, 28 Sep 2020 12:30:30 +0000 Gerrit-HasComments: Yes
