[
https://issues.apache.org/jira/browse/IMPALA-11538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltán Borók-Nagy updated IMPALA-11538:
---------------------------------------
Description:
In HMS we don’t store information about Iceberg partitions, we let Iceberg
handle them.
In Impala CatalogD/FE/BE we usually treat Iceberg tables as non-partitioned
tables, and only care about Iceberg partitions when really needed, e.g. INSERTs
need to be partition-aware, DPP for identity-partitioned tables, also SHOW
PARTITIONS must show information about the actual Iceberg partitions, etc.
But in some cases the users see inconsistent information because the
non-partitioned nature is revealed in some cases. We need to work on this to
not confuse users.
Some places that shows partitioned Iceberg tables as non-partiotioned:
* EXPLAIN statements
* SHOW TABLE STATS
* COMPUTE [INCREMENTAL]STATS
was:
In HMS we don’t store information about Iceberg partitions, we let Iceberg
handle them.
In Impala CatalogD/FE/BE we usually treat Iceberg tables as non-partitioned
tables, and only care about Iceberg partitions when really needed, e.g. INSERTs
need to be partition-aware, DPP for identity-partitioned tables, also SHOW
PARTITIONS must show information about the actual Iceberg partitions, etc.
But in some cases the users see inconsistent information because the
non-partitioned nature is revealed in some cases. We need to work on this to
not confuse users.
Some places that shows partitioned Iceberg tables as non-partiotioned:
* EXPLAIN statements
* SHOW TABLE STATS
> Provide consistent information about partitioned Iceberg tables
> ---------------------------------------------------------------
>
> Key: IMPALA-11538
> URL: https://issues.apache.org/jira/browse/IMPALA-11538
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Zoltán Borók-Nagy
> Priority: Major
> Labels: impala-iceberg, ramp-up
>
> In HMS we don’t store information about Iceberg partitions, we let Iceberg
> handle them.
> In Impala CatalogD/FE/BE we usually treat Iceberg tables as non-partitioned
> tables, and only care about Iceberg partitions when really needed, e.g.
> INSERTs need to be partition-aware, DPP for identity-partitioned tables, also
> SHOW PARTITIONS must show information about the actual Iceberg partitions,
> etc.
> But in some cases the users see inconsistent information because the
> non-partitioned nature is revealed in some cases. We need to work on this to
> not confuse users.
> Some places that shows partitioned Iceberg tables as non-partiotioned:
> * EXPLAIN statements
> * SHOW TABLE STATS
> * COMPUTE [INCREMENTAL]STATS
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
