Tim Armstrong created IMPALA-5834:
-------------------------------------
Summary: Handle spilling large number of NULLs on the build side
of a null-aware anti join
Key: IMPALA-5834
URL: https://issues.apache.org/jira/browse/IMPALA-5834
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Tim Armstrong
Priority: Minor
The join can't spill to disk in certain cases. For example, the below query
from spilling-naaj-no-debug-action.test can't spill the null build rows to disk.
{code}
# Execute NAAJ query 1 without enough memory to fit the null build rows in
memory.
select l_orderkey, l_partkey, l_suppkey, l_linenumber
from lineitem
where l_suppkey = 4162 and l_shipmode = 'AIR' and l_returnflag = 'A' and
l_shipdate > '1993-01-01' and
if(l_orderkey > 5500000, NULL, l_orderkey) not in (
select if(o_orderkey % 2 = 0, NULL, o_orderkey + 1)
from orders
where l_orderkey = o_orderkey)
order by 1,2,3,4
---- CATCH
Unable to perform Null-Aware Anti-Join. Could not get enough reservation to fit
all rows with NULLs from the build side in memory
{code}
It's unclear if this case is particularly common or important.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
