YUBI LEE created IMPALA-15737:
---------------------------------
Summary: Cancelled hash join fragments keep running until the
probe batch is fully processed
Key: IMPALA-15737
URL: https://issues.apache.org/jira/browse/IMPALA-15737
Project: IMPALA
Issue Type: Bug
Components: Backend
Affects Versions: Impala 4.5.3, Impala 4.5.0
Reporter: YUBI LEE
Assignee: YUBI LEE
The ProcessProbeRow* loops in PartitionedHashJoinNode
(be/src/exec/partitioned-hash-join-node-ir.cc) never check
RuntimeState::is_cancelled(). They only return to GetNext() when the
output batch is full.
In a many-to-many join whose other join conjuncts are mostly false (the
IMPALA-14863 shape), one probe row can match millions of build rows
without producing output. If the query is cancelled, the fragment
instance keeps running in the probe loop. We saw such instances run for
hours and hold hundreds of GB per executor until impalad was restarted.
IMPALA-14863 bounds the memory growth but does not stop the loop.
Repro: 1,024 x 1,000,000 rows with the same join key, joined with
always-false conjuncts. Cancel the query while it is probing.
impala-server.num-fragments-in-flight stays above zero for minutes.
Fix: check is_cancelled() every 1024 build rows in the probe loops and
return Status::CANCELLED, as IMPALA-5586 did for EvaluateNullProbe().
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
