[
https://issues.apache.org/jira/browse/IMPALA-13244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17867630#comment-17867630
]
Pain Sun commented on IMPALA-13244:
-----------------------------------
Thanks for your reply [~gaborkaszab] .
I know what's you means,and it succeeds with the order of the partition cols.
Maybe spark can succeed without the order of the partition cols that make me
confused,then I think impala also can work.
Anyway, I will follow Impala's guidelines.
> Timestamp partition error in catalogd when insert data into iceberg table
> --------------------------------------------------------------------------
>
> Key: IMPALA-13244
> URL: https://issues.apache.org/jira/browse/IMPALA-13244
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 4.4.0
> Environment: centos7.9
> Reporter: Pain Sun
> Priority: Major
>
> create table sql like this:
>
> CREATE TABLE test111.table1 (
> xxx STRING,
> code STRING,
> updatetime TIMESTAMP
> ) PARTITIONED BY spec(
> month(updatetime),
> bucket(10, code),
> bucket(10, xxx)
> ) STORED AS ICEBERG TBLPROPERTIES(
> 'iceberg.catalog' = 'hadoop.catalog',
> 'iceberg.catalog_location' = '/impalatable',
> 'iceberg.table_identifier' = 'middle.table1',
> 'write.metadata.previous-versions-max' = '3',
> 'write.metadata.delete-after-commit.enabled' = 'true',
> 'commit.manifest.min-count-to-merge' = '3',
> 'commit.manifest-merge.enabled' = 'true',
> 'format-version' = '1'
> );
>
>
>
> then insert data into this table like this:
> insert into
> test111.table1 (
> xxx,
> code,
> updatetime
> )
> select
> 'm1' as xxx,
> 'c1' as code,
> '2024-07-17 13:44:01' as updatetime;
> Catalogd error like this :
> E0719 09:50:57.458815 126128 JniUtil.java:183]
> 964d388b63170b6b:7c6e06c200000000] Error in Update catalog for
> test111.table1. Time spent: 6ms
> I0719 09:50:57.459015 126128 jni-util.cc:302]
> 964d388b63170b6b:7c6e06c200000000] java.lang.IllegalStateException
> at
> com.google.common.base.Preconditions.checkState(Preconditions.java:496)
> at
> org.apache.impala.util.IcebergUtil.parseMonthToTransformMonth(IcebergUtil.java:882)
> at
> org.apache.impala.util.IcebergUtil.getPartitionValue(IcebergUtil.java:826)
> at
> org.apache.impala.util.IcebergUtil.partitionDataFromDataFile(IcebergUtil.java:800)
> at
> org.apache.impala.service.IcebergCatalogOpExecutor.createDataFile(IcebergCatalogOpExecutor.java:445)
> at
> org.apache.impala.service.IcebergCatalogOpExecutor.appendFiles(IcebergCatalogOpExecutor.java:487)
> at
> org.apache.impala.service.IcebergCatalogOpExecutor.execute(IcebergCatalogOpExecutor.java:366)
> at
> org.apache.impala.service.CatalogOpExecutor.updateCatalogImpl(CatalogOpExecutor.java:7443)
> at
> org.apache.impala.service.CatalogOpExecutor.updateCatalog(CatalogOpExecutor.java:7180)
> at
> org.apache.impala.service.JniCatalog.lambda$updateCatalog$15(JniCatalog.java:504)
> at
> org.apache.impala.service.JniCatalogOp.lambda$execAndSerialize$1(JniCatalogOp.java:90)
> at org.apache.impala.service.JniCatalogOp.execOp(JniCatalogOp.java:58)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:89)
> at
> org.apache.impala.service.JniCatalogOp.execAndSerialize(JniCatalogOp.java:100)
> at
> org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:245)
> at
> org.apache.impala.service.JniCatalog.execAndSerialize(JniCatalog.java:259)
> at
> org.apache.impala.service.JniCatalog.updateCatalog(JniCatalog.java:503)
> I0719 09:50:57.459033 126128 status.cc:129]
> 964d388b63170b6b:7c6e06c200000000] IllegalStateException: null
> @ 0x10546b4
> @ 0x1b94d34
> @ 0x10040ab
> @ 0xfa1c27
> @ 0xf61f84
> @ 0xf4acc3
> @ 0xf5278b
> @ 0x14486aa
> @ 0x143b0fa
> @ 0x1c78d39
> @ 0x1c79fd1
> @ 0x256da47
> @ 0x7fabd2eb8ea5
> @ 0x7fabcfe939fd
> E0719 09:50:57.459059 126128 catalog-server.cc:324]
> 964d388b63170b6b:7c6e06c200000000] IllegalStateException: null
>
> but spark insert success.
>
> versions :
> impala: 4.4.0
> jar in impala: iceberg-api-1.3.1.7.2.18.0-369.jar
> spark: 3.3.4
> iceberg: apache 1.3.1
> iceberg-spark jar: iceberg-spark-runtime-3.3_2.12-1.3.1.jar
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
