Riza Suminto created IMPALA-13836:
-------------------------------------
Summary: TestIcebergV2Table.test_missing_data_files failed in
erasure-coding build
Key: IMPALA-13836
URL: https://issues.apache.org/jira/browse/IMPALA-13836
Project: IMPALA
Issue Type: Bug
Components: Test
Affects Versions: Impala 4.6.0
Reporter: Riza Suminto
Assignee: Zoltán Borók-Nagy
TestIcebergV2Table.test_missing_data_files failed in erasure-coding build with
following error:
{noformat}
Error Message
query_test/test_iceberg.py:1659: in test_missing_data_files
'$PART_FIRST_SNAPSHOT': first_snapshot(missing_files_part)})
common/impala_test_suite.py:904: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:737: in __verify_results_and_errors
replace_filenames_with_placeholder) common/test_result_verifier.py:523: in
verify_raw_results VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:273: in verify_query_result_is_subset assert
expected_literal_strings <= actual_literal_strings E assert Items in expected
results not found in actual results: E 'Erasure Coding Policy:','NONE
','NULL' E Items in actual results: E
'','previous_metadata_location','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part/metadata/00001-0443806d-28fa-4d57-bd60-c8c9f19ea1a0.metadata.json'
E '# col_name ','data_type ','comment ' E
'Retention: ','2147483647 ','NULL' E 'Table Type:
','EXTERNAL_TABLE ','NULL' E '# col_name
','transform_type ','NULL' E '','snapshot-count ','2
' E '','storage_handler
','org.apache.iceberg.mr.hive.HiveIcebergStorageHandler' E 'SerDe Library:
','org.apache.iceberg.mr.hive.HiveIcebergSerDe','NULL' E
'','external.table.purge','TRUE ' E '','engine.hive.enabled
','true ' E '','uuid
','f2fc9b81-6ecb-4c4a-9436-f65d2b115466' E '','numFiles ','2
' E 'Sort Columns: ','[] ','NULL' E
'','transient_lastDdlTime','1741215497 ' E '','EXTERNAL
','TRUE ' E
'','impala.events.catalogServiceId','a4e5d267f4dc462a:bb41a0bb9d8ef3fc' E
'Erasure Coding Policy:','RS-3-2-1024k ','NULL' E
'','current-snapshot-timestamp-ms','1741215501536 ' E 'OwnerType:
','USER ','NULL' E 'OutputFormat:
','org.apache.iceberg.mr.hive.HiveIcebergOutputFormat','NULL' E
'p','IDENTITY','NULL' E '','write.delete.mode ','merge-on-read ' E
'','metadata_location
','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part/metadata/00002-5c0608a4-72e0-4e74-b9f5-b9bed61b766c.metadata.json'
E '','write.update.mode ','merge-on-read ' E 'LastAccessTime:
','Tue Jan 20 23:09:02 PST 1970','NULL' E '','numRows ','2
' E 'Compressed: ','No ','NULL' E
'InputFormat:
','org.apache.iceberg.mr.hive.HiveIcebergInputFormat','NULL' E
'p','int','NULL' E '','write.merge.mode ','merge-on-read ' E
'','totalSize ','1178 ' E '','table_type
','ICEBERG ' E '','NULL','NULL' E '','OBJCAPABILITIES
','EXTREAD,EXTWRITE ' E '','current-snapshot-id ','8010190747966669125 '
E 'Table Parameters:','NULL','NULL' E
'','impala.events.catalogVersion','13092 ' E '# Storage
Information','NULL','NULL' E 'CreateTime: ','Wed Mar 05 14:58:17
PST 2025','NULL' E
'','default-partition-spec','{\\"spec-id\\":0,\\"fields\\":[{\\"name\\":\\"p\\",\\"transform\\":\\"identity\\",\\"source-id\\":2,\\"field-id\\":1000}]}'
E 'i','int','NULL' E '','current-schema
','{\\"type\\":\\"struct\\",\\"schema-id\\":0,\\"fields\\":[{\\"id\\":1,\\"name\\":\\"i\\",\\"required\\":false,\\"type\\":\\"int\\"},{\\"id\\":2,\\"name\\":\\"p\\",\\"required\\":false,\\"type\\":\\"int\\"}]}'
E 'Owner: ','jenkins ','NULL' E
'','write.format.default','parquet ' E '#
Constraints','NULL','NULL' E
'','current-snapshot-summary','{\\"added-data-files\\":\\"1\\",\\"added-records\\":\\"1\\",\\"added-files-size\\":\\"589\\",\\"changed-partition-count\\":\\"1\\",\\"total-records\\":\\"2\\",\\"total-files-size\\":\\"1178\\",\\"total-data-files\\":\\"2\\",\\"total-delete-files\\":\\"0\\",\\"total-position-deletes\\":\\"0\\",\\"total-equality-deletes\\":\\"0\\"}'
E 'Location:
','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part','NULL'
E '# Detailed Table Information','NULL','NULL' E 'Database:
','test_missing_data_files_a4d96b78','NULL' E '# Partition Transform
Information','NULL','NULL'
Stacktrace
query_test/test_iceberg.py:1659: in test_missing_data_files
'$PART_FIRST_SNAPSHOT': first_snapshot(missing_files_part)})
common/impala_test_suite.py:904: in run_test_case
self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:737: in __verify_results_and_errors
replace_filenames_with_placeholder)
common/test_result_verifier.py:523: in verify_raw_results
VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:273: in verify_query_result_is_subset
assert expected_literal_strings <= actual_literal_strings
E assert Items in expected results not found in actual results:
E 'Erasure Coding Policy:','NONE ','NULL'
E Items in actual results:
E
'','previous_metadata_location','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part/metadata/00001-0443806d-28fa-4d57-bd60-c8c9f19ea1a0.metadata.json'
E '# col_name ','data_type ','comment '
E 'Retention: ','2147483647 ','NULL'
E 'Table Type: ','EXTERNAL_TABLE ','NULL'
E '# col_name ','transform_type ','NULL'
E '','snapshot-count ','2 '
E '','storage_handler
','org.apache.iceberg.mr.hive.HiveIcebergStorageHandler'
E 'SerDe Library:
','org.apache.iceberg.mr.hive.HiveIcebergSerDe','NULL'
E '','external.table.purge','TRUE '
E '','engine.hive.enabled ','true '
E '','uuid ','f2fc9b81-6ecb-4c4a-9436-f65d2b115466'
E '','numFiles ','2 '
E 'Sort Columns: ','[] ','NULL'
E '','transient_lastDdlTime','1741215497 '
E '','EXTERNAL ','TRUE '
E '','impala.events.catalogServiceId','a4e5d267f4dc462a:bb41a0bb9d8ef3fc'
E 'Erasure Coding Policy:','RS-3-2-1024k ','NULL'
E '','current-snapshot-timestamp-ms','1741215501536 '
E 'OwnerType: ','USER ','NULL'
E 'OutputFormat:
','org.apache.iceberg.mr.hive.HiveIcebergOutputFormat','NULL'
E 'p','IDENTITY','NULL'
E '','write.delete.mode ','merge-on-read '
E '','metadata_location
','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part/metadata/00002-5c0608a4-72e0-4e74-b9f5-b9bed61b766c.metadata.json'
E '','write.update.mode ','merge-on-read '
E 'LastAccessTime: ','Tue Jan 20 23:09:02 PST 1970','NULL'
E '','numRows ','2 '
E 'Compressed: ','No ','NULL'
E 'InputFormat:
','org.apache.iceberg.mr.hive.HiveIcebergInputFormat','NULL'
E 'p','int','NULL'
E '','write.merge.mode ','merge-on-read '
E '','totalSize ','1178 '
E '','table_type ','ICEBERG '
E '','NULL','NULL'
E '','OBJCAPABILITIES ','EXTREAD,EXTWRITE '
E '','current-snapshot-id ','8010190747966669125 '
E 'Table Parameters:','NULL','NULL'
E '','impala.events.catalogVersion','13092 '
E '# Storage Information','NULL','NULL'
E 'CreateTime: ','Wed Mar 05 14:58:17 PST 2025','NULL'
E
'','default-partition-spec','{\\"spec-id\\":0,\\"fields\\":[{\\"name\\":\\"p\\",\\"transform\\":\\"identity\\",\\"source-id\\":2,\\"field-id\\":1000}]}'
E 'i','int','NULL'
E '','current-schema
','{\\"type\\":\\"struct\\",\\"schema-id\\":0,\\"fields\\":[{\\"id\\":1,\\"name\\":\\"i\\",\\"required\\":false,\\"type\\":\\"int\\"},{\\"id\\":2,\\"name\\":\\"p\\",\\"required\\":false,\\"type\\":\\"int\\"}]}'
E 'Owner: ','jenkins ','NULL'
E '','write.format.default','parquet '
E '# Constraints','NULL','NULL'
E
'','current-snapshot-summary','{\\"added-data-files\\":\\"1\\",\\"added-records\\":\\"1\\",\\"added-files-size\\":\\"589\\",\\"changed-partition-count\\":\\"1\\",\\"total-records\\":\\"2\\",\\"total-files-size\\":\\"1178\\",\\"total-data-files\\":\\"2\\",\\"total-delete-files\\":\\"0\\",\\"total-position-deletes\\":\\"0\\",\\"total-equality-deletes\\":\\"0\\"}'
E 'Location:
','hdfs://localhost:20500/test-warehouse/test_missing_data_files_a4d96b78.db/missing_files_part','NULL'
E '# Detailed Table Information','NULL','NULL'
E 'Database: ','test_missing_data_files_a4d96b78','NULL'
E '# Partition Transform Information','NULL','NULL'{noformat}
The test recently added by IMPALA-13654.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
