[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog Impala tries to come up with the table location of external Iceberg tables stored in HadoopCatalog. The current method is not correct for tables that are nested under multiple namespaces. With this patch Imapala loads the Iceberg table and retrieves the location from it. Testing: * added e2e test in iceberg-create.test Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Reviewed-on: http://gerrit.cloudera.org:8080/16795 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test 3 files changed, 37 insertions(+), 6 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 5 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 02 Dec 2020 22:42:11 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 4: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6722/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 02 Dec 2020 17:09:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 4: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 4 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 02 Dec 2020 17:09:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 02 Dec 2020 17:02:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Wed, 02 Dec 2020 11:52:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7758/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 11:03:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Hello Gabor Kaszab, wangsheng, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16795 to look at the new patch set (#3). Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog Impala tries to come up with the table location of external Iceberg tables stored in HadoopCatalog. The current method is not correct for tables that are nested under multiple namespaces. With this patch Imapala loads the Iceberg table and retrieves the location from it. Testing: * added e2e test in iceberg-create.test Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test 3 files changed, 37 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/16795/3 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2738 PS2, Line 2738: // When creating external Iceberg table with 'hadoop.catalog' we load > line too long (91 > 90) Done http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2739 PS2, Line 2739: // the Iceberg table using catalog location and table identifier to get > line too long (96 > 90) Done http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2740 PS2, Line 2740: // the actual location of the table. This way we can also get the > line too long (93 > 90) Done -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 3 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 10:40:00 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7757/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 09:59:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2738 PS2, Line 2738: // When creating external Iceberg table with 'hadoop.catalog' we load the line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2739 PS2, Line 2739: // Iceberg table using catalog location and table identifier to get the actual line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/16795/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2740 PS2, Line 2740: // location of the table. This way we can also get the correct location for line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 09:38:52 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Hello Gabor Kaszab, wangsheng, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/16795 to look at the new patch set (#2). Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog Impala tries to come up with the table location of external Iceberg tables stored in HadoopCatalog. The current method is not correct for tables that are nested under multiple namespaces. With this patch Imapala loads the Iceberg table and retrieves the location from it. Testing: * added e2e test in iceberg-create.test Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test 3 files changed, 37 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/16795/2 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: wangsheng
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Zoltan Borok-Nagy has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/16795/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/16795/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2742 PS1, Line 2742: TableIdentifier identifier = : IcebergUtil.getIcebergTableIdentifier(newTable); : newTable.getSd().setLocation(IcebergUtil.loadTable( > I think it's better to add comment here to describe the multiple namespace Extended the above comment. -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Zoltan Borok-Nagy Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 09:38:12 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 1: Code-Review+1 (1 comment) Thanks for this improvement, LGTM http://gerrit.cloudera.org:8080/#/c/16795/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java: http://gerrit.cloudera.org:8080/#/c/16795/1/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@2742 PS1, Line 2742: newTable.getSd().setLocation(IcebergUtil.loadTable( : TIcebergCatalog.HADOOP_CATALOG, identifier, : IcebergUtil.getIcebergCatalogLocation(newTable)).location()); I think it's better to add comment here to describe the multiple namespaces situation. -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Tue, 01 Dec 2020 07:50:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16795 ) Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/7750/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Gabor Kaszab Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: wangsheng Gerrit-Comment-Date: Mon, 30 Nov 2020 15:58:56 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16795 Change subject: IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog .. IMPALA-10364: Set the real location for external Iceberg tables stored in HadoopCatalog Impala tries to come up with the table location of external Iceberg tables stored in HadoopCatalog. The current method is not correct for tables that are nested under multiple namespaces. With this patch Imapala loads the Iceberg table and retrieves the location from it. Testing: * added e2e test in iceberg-create.test Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 --- M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/IcebergUtil.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-create.test 3 files changed, 33 insertions(+), 4 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/16795/1 -- To view, visit http://gerrit.cloudera.org:8080/16795 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I04b75d219e095ce00b4c48f40b8dee872ba57b78 Gerrit-Change-Number: 16795 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy