Andrey Popkov created IMPALA-13251:
--------------------------------------
Summary: Planner uses session MEM_LIMIT with Max MEM_LIMIT set on
Resource Pool
Key: IMPALA-13251
URL: https://issues.apache.org/jira/browse/IMPALA-13251
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 4.3.0
Reporter: Andrey Popkov
impalad version 4.3.0
Query is INSERT SELECT FROM WHERE
with no joins, functions, orders etc
fails with OOM
Query MEM_LIMIT is set to 35Gb
Resource Pool max query MEM_LIMIT range is 20Gb
Cluster has 14 hosts and Admission Control is enabled
so we see in query profile:
Query Options (set by configuration): MEM_LIMIT=37580963840, MT_DOP=8, ..
Query Options (set by configuration and planner): MEM_LIMIT=37580963840,
MT_DOP=8, ..
Cluster Memory Admitted: 280.00 GB
Max Per-Host Resource Reservation: Memory=128.00MB Threads=8
Per-Host Resource Estimates: Memory=123.54GB
Query(x:y): memory limit exceeded. Limit=20.00 GB Reservation=192.25 MB
ReservationLimit=16.00 GB OtherMemory=19.81 GB Total=20.00 GB Peak=20.00 GB
The same query with MEM_LIMIT=0 and MT_DOP=0 is successful with:
Cluster Memory Admitted 280.00 GB
Max Per-Host Resource Reservation: Memory=16.00MB Threads=2
Per-Host Resource Estimates: Memory=35.02GB
The question is why planner uses 35G but not 20G
And perhaps, it has impact on memory utilization
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
