[
https://issues.apache.org/jira/browse/IMPALA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Quanlong Huang resolved IMPALA-3127.
------------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> Decouple partitions from tables
> -------------------------------
>
> Key: IMPALA-3127
> URL: https://issues.apache.org/jira/browse/IMPALA-3127
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 2.2.4
> Reporter: Dimitris Tsirogiannis
> Assignee: Quanlong Huang
> Priority: Major
> Labels: catalog-server, performance
> Fix For: Impala 4.0
>
>
> Currently, partitions are tightly integrated into the HdfsTable objects,
> making incremental metadata updates difficult to perform. Furthermore, the
> catalog transmits entire table metadata even when only few partitions change,
> introducing significant latencies, wasting network bandwidth and CPU cycles
> while updating table metadata at the receiving impalads. As a first step, we
> should decouple partitions from tables and add them as a separate level in
> the hierarchy of catalog entities (server-db-table-partition). Subsequently,
> the catalog should transmit only entities that have changed after DDL/DML
> statements.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
