[
https://issues.apache.org/jira/browse/IMPALA-13655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Noemi Pap-Takacs updated IMPALA-13655:
--------------------------------------
Description:
When we have an Iceberg table that has lots of partitions, and we want to
update lots of values in the table, it will use much more memory than needed.
Repro steps:
{noformat}
create table tmp_ice_tpch
partitioned by spec(truncate(500, l_orderkey))
stored by iceberg as
select * from tpch.lineitem;
UPDATE TABLE tmp_ice_tpch SET l_partkey=l_partkey+1;
# We likely get a Memory Limit Exceeded error here{noformat}
was:
When we have an Iceberg table that have lots of partitions, and we want to
update lots of values in the table, it will use much more memory than needed.
Repro steps:
{noformat}
create table tmp_ice_tpch
partitioned by spec(truncate(500, l_orderkey))
stored by iceberg as
select * from tpch.lineitem;
UPDATE TABLE tmp_ice_tpch SET l_partkey=l_partkey+1;
# We likely get a Memory Limit Exceeded error here{noformat}
> UPDATE redundantly accumulates memory in HDFS WRITER
> ----------------------------------------------------
>
> Key: IMPALA-13655
> URL: https://issues.apache.org/jira/browse/IMPALA-13655
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Noemi Pap-Takacs
> Assignee: Noemi Pap-Takacs
> Priority: Major
> Labels: impala-iceberg
> Fix For: Impala 4.5.0
>
>
> When we have an Iceberg table that has lots of partitions, and we want to
> update lots of values in the table, it will use much more memory than needed.
> Repro steps:
> {noformat}
> create table tmp_ice_tpch
> partitioned by spec(truncate(500, l_orderkey))
> stored by iceberg as
> select * from tpch.lineitem;
> UPDATE TABLE tmp_ice_tpch SET l_partkey=l_partkey+1;
> # We likely get a Memory Limit Exceeded error here{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
