[
https://issues.apache.org/jira/browse/IMPALA-9778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang updated IMPALA-9778:
-----------------------------------
Description:
Making HdfsPartition immutable has several benefits:
* HdfsPartition meta can be shared across table versions, which means in the
legacy catalog update mode, catalog update can ignore unchanged partitions
(IMPALA-3234) and send the update in partition granularity.
* Aborted DDL/DMLs won’t leave partition metadata in illegal states
(IMPALA-8406), which requires invalidation to recover.
* LocalCatalog coordinators can cache partition meta using the partition id as
the key. When table version updates, only metadata of changed partitions needs
to be reloaded (IMPALA-7533).
* In the work of IMPALA-3127, we don't need to assign a catalog version to
partitions since the partition ids already identify the partitions.
was:
Making HdfsPartition immutable has several benefits:
* HdfsPartition meta can be shared across table versions, which means in the
legacy catalog update mode, catalog update can ignore unchanged partitions
(IMPALA-3234) and send the update in partition granularity.
* Aborted DDL/DMLs won’t leave partition metadata in illegal states, which
requires invalidation to recover.
* LocalCatalog coordinators can cache partition meta using the partition id as
the key. When table version updates, only metadata of changed partitions needs
to be reloaded (IMPALA-7533).
* In the work of IMPALA-3127, we don't need to assign a catalog version to
partitions since the partition ids already identify the partitions.
> Refactor HdfsPartition to be immutable
> --------------------------------------
>
> Key: IMPALA-9778
> URL: https://issues.apache.org/jira/browse/IMPALA-9778
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.0
>
>
> Making HdfsPartition immutable has several benefits:
> * HdfsPartition meta can be shared across table versions, which means in the
> legacy catalog update mode, catalog update can ignore unchanged partitions
> (IMPALA-3234) and send the update in partition granularity.
> * Aborted DDL/DMLs won’t leave partition metadata in illegal states
> (IMPALA-8406), which requires invalidation to recover.
> * LocalCatalog coordinators can cache partition meta using the partition id
> as the key. When table version updates, only metadata of changed partitions
> needs to be reloaded (IMPALA-7533).
> * In the work of IMPALA-3127, we don't need to assign a catalog version to
> partitions since the partition ids already identify the partitions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
