Pain Sun created IMPALA-13244:
---------------------------------
Summary: Timestamp partition error in catalogd when insert data
into iceberg table
Key: IMPALA-13244
URL: https://issues.apache.org/jira/browse/IMPALA-13244
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 4.4.0
Environment: centos7.9
Reporter: Pain Sun
create table sql like this:
CREATE TABLE test111.table1 (
xxx STRING,
code STRING,
updatetime TIMESTAMP
) PARTITIONED BY spec(
month(updatetime),
bucket(10, code),
bucket(10, xxx)
) STORED AS ICEBERG TBLPROPERTIES(
'iceberg.catalog' = 'hadoop.catalog',
'iceberg.catalog_location' = '/impalatable',
'iceberg.table_identifier' = 'middle.table1',
'write.metadata.previous-versions-max' = '3',
'write.metadata.delete-after-commit.enabled' = 'true',
'commit.manifest.min-count-to-merge' = '3',
'commit.manifest-merge.enabled' = 'true',
'format-version' = '1'
);
then insert data into this table like this:
insert into
test111.table1 (
mainsellernick,
code,
updatetime
)
select
'm1' as xxx,
'c1' as code,
'2024-07-17 13:44:01' as updatetime;
Catalogd error like this :
E0719 09:50:57.458815 126128 JniUtil.java:183]
964d388b63170b6b:7c6e06c200000000] Error in Update catalog for test111.table1.
Time spent: 6ms
I0719 09:50:57.459015 126128 jni-util.cc:302]
964d388b63170b6b:7c6e06c200000000] java.lang.IllegalStateException
at
com.google.common.base.Preconditions.checkState(Preconditions.java:496)
at
org.apache.impala.util.IcebergUtil.parseMonthToTransformMonth(IcebergUtil.java:882)
at
org.apache.impala.util.IcebergUtil.getPartitionValue(IcebergUtil.java:826)
at
org.apache.impala.util.IcebergUtil.partitionDataFromDataFile(IcebergUtil.java:800)
at
org.apache.impala.service.IcebergCatalogOpExecutor.createDataFile(IcebergCatalogOpExecutor.java:445)
at
org.apache.impala.service.IcebergCatalogOpExecutor.appendFiles(IcebergCatalogOpExecutor.java:487)
at
org.apache.impala.service.IcebergCatalogOpExecutor.execute(IcebergCatalogOpExecutor.java:366)
at
org.apache.impala.service.CatalogOpExecutor.updateCatalogImpl(CatalogOpExecutor.java:7443)
at
org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:7180)
at
org.apache.impala.service.JniCatalog.lambda$updateCatalog$15(JniCatalog.java:504)
at
org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
at
org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:245)
at
org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:259)
at
org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:503)
I0719 09:50:57.459033 126128 status.cc:129] 964d388b63170b6b:7c6e06c200000000]
IllegalStateException: null
@ 0x10546b4
@ 0x1b94d34
@ 0x10040ab
@ 0xfa1c27
@ 0xf61f84
@ 0xf4acc3
@ 0xf5278b
@ 0x14486aa
@ 0x143b0fa
@ 0x1c78d39
@ 0x1c79fd1
@ 0x256da47
@ 0x7fabd2eb8ea5
@ 0x7fabcfe939fd
E0719 09:50:57.459059 126128 catalog-server.cc:324]
964d388b63170b6b:7c6e06c200000000] IllegalStateException: null
bnt spark insert success.
versions :
impala: 4.4.0
jar in impala: iceberg-api-1.3.1.7.2.18.0-369.jar
spark: 3.3.4
iceberg: apache 1.3.1
iceberg-spark jar: iceberg-spark-runtime-3.3_2.12-1.3.1.jar
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
