[
https://issues.apache.org/jira/browse/HIVE-28757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18098165#comment-18098165
]
Feng Liu commented on HIVE-28757:
---------------------------------
We investigated this issue on a production-derived Hive 3.1.2 fork and
confirmed the root cause.
Root cause
The metastore initializes server-owned fields, including createTime, when a new
partition is added. However, the Hive client path in Hive.loadPartition ignores
the Partition returned by add_partition and keeps using the original
client-side object whose createTime is 0. If a following StatsTask alters that
partition, the stale client object is sent back to the metastore and overwrites
the persisted createTime with 0.
The same pattern is still present on the current master branch:
addPartitionToMetastore calls getMSC().add_partition(...) without retaining the
returned partition. The current master also has a batch dynamic-partition path,
so both single and batch creation flows need to be covered.
Proposed fix
1. Preserve the Partition returned by the metastore after a successful add
operation and continue with that server-initialized object.
2. Add a defensive metastore-side check in alterPartition and alterPartitions
so an unset or non-positive incoming createTime cannot overwrite an existing
valid value.
3. Cover both single-partition and batch dynamic-partition creation paths on
master.
Validation on Hive 3.1.2
We implemented the fix on Hive 3.1.2 and added client-side and metastore
regression tests. A clean ql compilation also passed.
We are preparing two clean patches: one adapted to the latest Apache Hive
master, and one based on the official Hive 3.1.2 source for reference/backport.
The upstream patch will be created from apache/master and will not include any
downstream-specific changes.
> The inserted dynamic partitions loss createTime
> -----------------------------------------------
>
> Key: HIVE-28757
> URL: https://issues.apache.org/jira/browse/HIVE-28757
> Project: Hive
> Issue Type: Bug
> Components: Metastore
> Affects Versions: 3.1.3
> Reporter: FangBO
> Priority: Minor
> Attachments: image.png
>
>
> The inserted dynamic partitions loss createTime。
>
> {code:java}
> // code placeholder
> create table p (id bigint) partitioned by (ds string);
> create table src (id bigint, ds string);
> insert into src values(1, '00');
> insert overwrite table p partition(ds) select id,ds from src;{code}
> After the operation, the inserted partition does not have createTime.
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
