David Rorke created IMPALA-11618:
------------------------------------
Summary: Impala fails to read Iceberg snapshot tables created by
Spark using call spark_catalog.system.snapshot()
Key: IMPALA-11618
URL: https://issues.apache.org/jira/browse/IMPALA-11618
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 4.1.0
Reporter: David Rorke
I get an exception from catalog when Impala tries to read an Iceberg "snapshot"
table created by a spark job using this API:
[https://iceberg.apache.org/docs/latest/spark-procedures/#snapshot]
{noformat}
Query submitted at: 2022-09-27 09:43:15 (Coordinator:
http://coordinator-0.coordinator-int.impala-1662757723-t7hk.svc.cluster.local:25000)
ERROR: AnalysisException: Failed to load metadata for table:
'customer_iceberg_snapshot'
CAUSED BY: TableLoadingException: Could not load table
tpcds_3000_string_parquet_external.customer_iceberg_snapshot from catalog
CAUSED BY: TException:
TGetPartialCatalogObjectResponse(status:TStatus(status_code:GENERAL,
error_msgs:[IcebergTableLoadingException: Error loading metadata for Iceberg
table
s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer_iceberg_snapshot
CAUSED BY: RuntimeException: FileSystem returned an unexpected path
s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer/0b443b4d91612be0-af2edb1b00000000_1194661597_data.0.parq
for a file within
s3a://drorke-dwxperf2/warehouse/tablespace/external/hive/tpcds_3000_string_parquet_external.db/customer_iceberg_snapshot]),
lookup_status:OK)
Looks like Impala gets confused in FileSystemUtils.relativizePath() because the
final directory component in the new snapshot table name
(customer_iceberg_snapshot) is different from the corresponding path component
in the file.
{noformat}
It looks like Impala gets confused in FileSystemUtils.relativizePath() because
the final directory component in the new snapshot table name
(customer_iceberg_snapshot) is different from the corresponding path component
in the file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
