[
https://issues.apache.org/jira/browse/IMPALA-14659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18049988#comment-18049988
]
guangxiji commented on IMPALA-14659:
------------------------------------
*Subject:* Re: [IMPALA-XXXXX] Iceberg table loading failure on Tencent Cloud COS
*Hi,*
Thank you for the detailed questions. Here is the feedback based on my tests:
*1. How was the table created?* I have tried two ways to create the table, and
both resulted in the same error:
* *Method A (Explicit Location):* {{CREATE TABLE ... STORED AS ICEBERG
LOCATION
'cosn://cdnlogtest-1252412955/impala_test_db/iceberg_cos_employee_test';}}
* {{such as CREATE TABLE db_on_cos_test.iceberg_cos_employee_test (}}
{{ id INT, name STRING, dept STRING, salary DOUBLE, ts TIMESTAMP}}
{{) STORED AS ICEBERG }}
{{LOCATION
'cosn://cdnlogtest-1252412955/impala_test_db/iceberg_cos_employee_test';}}
* *Method B (Default Location):* {{CREATE TABLE ... STORED AS ICEBERG;}} (My
cluster is configured with COS as the default file system).
* CREATE TABLE db_on_cos_test.iceberg_cos_employee_test (
id INT, name STRING, dept STRING, salary DOUBLE, ts TIMESTAMP
) STORED AS ICEBERG ;
In both cases, the table is successfully registered in HMS, but fails during
the metadata loading.
*2. How was data added/written?* I used a standard {{INSERT}} statement:
{{INSERT INTO iceberg_cos_employee_test VALUES (1, 'Zhang San', 'R&D', 15000,
now());}} The query execution reaches 100%, but it fails at the very end when
Impala tries to reload the table metadata to commit the transaction.
*3. Engine & Environment:*
* *Engines:* I am using {*}Impala exclusively{*}. No other engines
(Hive/Spark) are involved.
* *File Consistency:* No files were manually removed. I can see the Parquet
files and the {{metadata/}} folder (containing {{.metadata.json}} and {{.avro}}
files) present on the COS bucket.
*4. Logs regarding paths:* Interestingly, the logs *do not* specify which file
path failed. It only shows: {{{}failed to load 1 paths{}}}. This makes me
suspect that the {{ParallelFileMetadataLoader}} is encountering an exception
(like a NullPointerException) internally before it can even log the problematic
path, possibly due to how the {{cosn}} driver reports block locations.
!image-2026-01-06-10-11-25-474.png!
> Impala 4.5 Iceberg table on Tencent Cloud COS fails with"failedto load 1
> paths" during metadata refresh.
> --------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-14659
> URL: https://issues.apache.org/jira/browse/IMPALA-14659
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 4.5.0
> Reporter: guangxiji
> Priority: Major
> Labels: catalog-server, newbie
> Attachments: image-2026-01-06-10-11-25-474.png, impala_log.txt
>
>
> I applied {{--max_nonhdfs_partitions_parallel_load=1}} and
> {{--max_hdfs_partitions_parallel_load=1}} to my {{catalogd_flags}} and
> restarted the service. Unfortunately, the issue persists even with
> single-threaded loading.
> It seems that in Impala 4.5, {{ParallelFileMetadataLoader}} encounters
> difficulties when processing metadata returned by the {{cosn://}} driver,
> regardless of the thread count. This suggests there might be a specific
> compatibility issue with how file/block metadata is interpreted for this
> storage scheme.
> Below are the raw logs from my session showing that {*}SHOW METADATA{*},
> {*}INSERT{*}, and *SELECT* all trigger the same metadata loading error:
> [9.192.163.17:21050] db_on_cos_test> SHOW METADATA TABLES IN
> iceberg_cos_employee_test;
> Query: SHOW METADATA TABLES IN iceberg_cos_employee_test
> 2026-01-05 22:28:58 [Exception] ERROR: Query
> 9349f0ef1c788ec2:8548051e00000000 failed:
> AnalysisException: Could not load table
> db_on_cos_test.iceberg_cos_employee_test from catalog
> ... (此处省略部分重复输出以保持简洁，或按您要求保留全部) ...
> CAUSED BY: TableLoadingException: Loading file and block metadata for 1 paths
> for table db_on_cos_test.iceberg_cos_employee_test: failed to load 1 paths.
> Check the catalog server log for more details.
> [9.192.163.17:21050] db_on_cos_test> INSERT INTO iceberg_cos_employee_test
> VALUES (1, '张三', '研发部', 15001.5, now());
> Query: INSERT INTO iceberg_cos_employee_test VALUES (1, '张三', '研发部', 15001.5,
> now())
> Query submitted at: 2026-01-05 22:29:08 (Coordinator:
> http://9.192.163.17:8080)
> 2026-01-05 22:29:08 [Exception] ERROR: Query
> ad4f18bf73bf1157:772c7c7600000000 failed:
> AnalysisException: org.apache.impala.catalog.TableLoadingException: Could not
> load table db_on_cos_test.iceberg_cos_employee_test from catalog
> ...
> CAUSED BY: TableLoadingException: Loading file and block metadata for 1 paths
> for table db_on_cos_test.iceberg_cos_employee_test: failed to load 1 paths.
> [9.192.163.17:21050] db_on_cos_test> select * from iceberg_cos_employee_test;
> Query: select * from iceberg_cos_employee_test
> Query submitted at: 2026-01-05 22:30:42 (Coordinator:
> http://9.192.163.17:8080)
> 2026-01-05 22:30:42 [Exception] ERROR: Query
> 274fea8afd3f9dde:8d9847ab00000000 failed:
> AnalysisException: Failed to load metadata for table:
> 'iceberg_cos_employee_test'
> ...
> CAUSED BY: TableLoadingException: Loading file and block metadata for 1 paths
> for table db_on_cos_test.iceberg_cos_employee_test: failed to load 1 paths.
> I have attached the full *catalogd logs* to this email for further
> investigation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
