[
https://issues.apache.org/jira/browse/IMPALA-11093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yu-Wen Lai updated IMPALA-11093:
--------------------------------
Description:
If we insert data into an acid partitioned table from Hive, the generated
events will be like open_txn -> alter_partition -> commit_txn.
In alter_partition event, we will refresh the partition (without refreshing
file metadata) because the write id with the partition object in the event >
write id of the cached partition. Then, in commit_txn event, we still do the
same write id check. However, the write id is still the same, so we end up
don't fresh file metadata.
was:
If we insert data into an acid partitioned table from Hive, the generated
events will be like open_txn -> alter_partition -> commit_txn.
Let's assume the allocated write id in this transaction is 3.
In alter_partition event, we will refresh the partition (without refreshing
file metadata) because the write id with the partition object in the event >
write id of the cache partition. Then, in commit_txn event, we still do the
same write id check. However, the write id is still the same, so we end up
don't fresh file metadata.
> Fine grained table refreshing doesn't refresh table file metadata
> -----------------------------------------------------------------
>
> Key: IMPALA-11093
> URL: https://issues.apache.org/jira/browse/IMPALA-11093
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Reporter: Yu-Wen Lai
> Assignee: Yu-Wen Lai
> Priority: Major
>
> If we insert data into an acid partitioned table from Hive, the generated
> events will be like open_txn -> alter_partition -> commit_txn.
> In alter_partition event, we will refresh the partition (without refreshing
> file metadata) because the write id with the partition object in the event >
> write id of the cached partition. Then, in commit_txn event, we still do the
> same write id check. However, the write id is still the same, so we end up
> don't fresh file metadata.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
