Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16143 )
Change subject: IMPALA-9741: Support querying Iceberg table by impala ...................................................................... Patch Set 20: (1 comment) http://gerrit.cloudera.org:8080/#/c/16143/20/testdata/data/iceberg_test/iceberg_non_partitioned/metadata/v1.metadata.json File testdata/data/iceberg_test/iceberg_non_partitioned/metadata/v1.metadata.json: PS20: > Hi Tim, do you mean that I need to use '$DEFAULT_FS' to replace 'hdfs://loc DEFAULT_FS is an environment variables that's set in impala-config.sh E.g. on my dev environment it is: DEFAULT_FS=hdfs://172.19.0.1:20500 And I get errors like this when loading data: MESSAGE: TableLoadingException: Error loading metadata for Iceberg table hdfs://172.19.0.1:20500/test-warehouse/iceberg_test/iceberg_partitioned CAUSED BY: RuntimeIOException: Failed to open input stream for file: hdfs://localhost:20500/test-warehouse/iceberg_test/iceberg_partitioned/metadata/a02a7aba-35c1-437c-905d-845d0ca9f19f-m0.avro CAUSED BY: ConnectException: Call From tarmstrong-box2/127.0.0.1 to localhost:20500 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused CAUSED BY: ConnectException: Connection refused With the JSON files we could do a textual search and replace as part of data loading, e.g. we could do something like this that I modified from testdata/bin/load-metastore-snapshot.sh sed "s|hdfs://localhost:20500|${DEFAULT_FS}|g" testdata/data /iceberg_test/iceberg_non_partitioned/metadata/v1.metadata.json.template > testdata/data/iceberg_test/iceberg_non_partitioned/metadata/v1.metadata.json I tried doing that manually but I still had problems because the avro files seem to have hdfs://localhost:20500 hardcoded in them too. I can try to think of a solution but maybe you also have ideas. -- To view, visit http://gerrit.cloudera.org:8080/16143 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I856cfee4f3397d1a89cf17650e8d4fbfe1f2b006 Gerrit-Change-Number: 16143 Gerrit-PatchSet: 20 Gerrit-Owner: wangsheng <[email protected]> Gerrit-Reviewer: Anonymous Coward (606) Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]> Gerrit-Reviewer: wangsheng <[email protected]> Gerrit-Comment-Date: Sat, 29 Aug 2020 00:40:53 +0000 Gerrit-HasComments: Yes
