[
https://issues.apache.org/jira/browse/IMPALA-12758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sai Hemanth Gantasala resolved IMPALA-12758.
--------------------------------------------
Fix Version/s: Impala 4.5.0
Resolution: Fixed
> Event Processor is ignoring the prev_id while reloading the existing
> partitions
> -------------------------------------------------------------------------------
>
> Key: IMPALA-12758
> URL: https://issues.apache.org/jira/browse/IMPALA-12758
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Sai Hemanth Gantasala
> Assignee: Sai Hemanth Gantasala
> Priority: Major
> Labels: catalog-2024
> Fix For: Impala 4.5.0
>
>
> Partitioned table Insert events consumed by the event processor reloads the
> partitions to update file metadata. Currently while reloading the
> filemetadata, 'prev_id' of the old partitions is being ignored in the
> partition builder
> {code:java}
> HdfsPartition oldPartition = entry.getValue();
> HdfsPartition.Builder partBuilder = createPartitionBuilder(
> hmsPartition.getSd(), hmsPartition, permissionCache); {code}
> As a result 'prev_id' of the partBuilder will always be -1, so when the
> catalogDelta is sent from the state store to impala demons, because prev_id
> is not valid, impalads will not know whether to invalidate the current
> partition and then request the new partition information.
> This might lead to data correctness issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
