[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/22600/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 12 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Joe McDonnell Gerrit-Comment-Date: Tue, 26 May 2026 18:23:14 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. If the batch processing requires evaluating expressions that allocate memory for their results (such as UPPER() or LOWER()), that memory is allocated without checking any memory limits. If a large number of these expression evaluations are performed during a single row batch, memory will be consumed until the process memory limit is reached. At that point, the executor begins failing other queries with cannot allocate memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking during batch processing. This patch determines a limit for the amount of memory that conjunct expression evaluation results can consume as a percentage of the lowest soft memory limit for the query execution. Before evaluating conjuncts for each join probe row, the expression results memory pool size is cleared if its allocated memory has exceeded this threshold. Clearing the memory pool does not result in memory being freed but instead marks all memory in the pool as available for re-use. Testing accomplished by running the repro query documented in the Jira with MEM_LIMIT_EXECUTORS=185mb and watching the executor /memz debug UI page to ensure the query fragment instances stop consuming memory once they hit their limit. Regression functionality testing performed by running existing test suite. Regression performance testing performed by running AB test job. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h 3 files changed, 40 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/12 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 12 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Jason Fehr Gerrit-Reviewer: Joe McDonnell
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/24159 )
Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch
Processing
..
Patch Set 7:
(1 comment)
http://gerrit.cloudera.org:8080/#/c/24159/7/be/src/udf/udf.cc
File be/src/udf/udf.cc:
http://gerrit.cloudera.org:8080/#/c/24159/7/be/src/udf/udf.cc@523
PS7, Line 523: if (results_pool_max_mem_ > -1 && UNLIKELY(
: results_pool_->total_allocated_bytes() + byte_size >
results_pool_max_mem_)) {
: results_pool_->Clear();
: }
How do we know that there are no references to the memory we are clearing?
Could I hit this partway through evaluating a conjunct on a single row? For
example with conjunct upper(foo) = upper(bar), could I reach this condition
when evaluating upper(bar) and clear the result from upper(foo)?
--
To view, visit http://gerrit.cloudera.org:8080/24159
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af
Gerrit-Change-Number: 24159
Gerrit-PatchSet: 7
Gerrit-Owner: Jason Fehr
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Comment-Date: Thu, 07 May 2026 21:49:41 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 7: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/22277/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 7 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 29 Apr 2026 22:14:43 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. If the batch processing requires evaluating expressions that allocate memory for their results (such as UPPER() or LOWER()), that memory is allocated without checking any memory limits. If a large number of these expression evaluations are performed during a single row batch, memory will be consumed until the process memory limit is reached. At that point, the executor begins failing other queries with cannot allocate memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking during batch processing. This patch determines a limit for the amount of memory that ScalarExprEvaluator instances can consume for their results based on the lowest soft memory limit in the result MemPool hierarchy reduced by a factor. When expression evaluation results exceeds that threshold, the memory pool is cleared during which row batch processing is paused. Clearing the memory pool does not result in memory being freed but instead marks all memory in the pool as available for re-use. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M be/src/exec/exec-node.cc M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/udf/udf-internal.h M be/src/udf/udf.cc 5 files changed, 44 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/7 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 7 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 6: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/22276/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 6 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 29 Apr 2026 21:38:08 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. If the batch processing requires evaluating expressions that allocate memory for their results (such as UPPER() or LOWER()), that memory is allocated without checking any memory limits. If a large number of these expression evaluations are performed during a single row batch, memory will be consumed until the process memory limit is reached. At that point, the executor begins failing other queries with cannot allocate memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking during batch processing. This patch determines a limit for the amount of memory that ScalarExprEvaluator instances can consume for their results based on the lowest soft memory limit in the result MemPool hierarchy reduced by a factor. When expression evaluation results exceeds that threshold, the memory pool is cleared during which row batch processing is paused. Clearing the memory pool does not result in memory being freed but instead marks all memory in the pool as available for re-use. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M be/src/exec/exec-node.cc M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/udf/udf-internal.h M be/src/udf/udf.cc 5 files changed, 47 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/6 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 6 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. If the batch processing requires evaluating expressions that allocate memory for their results (such as UPPER() or LOWER()), that memory is allocated without checking any memory limits. If a large number of these expression evaluations are performed, memory will be consume until the process memory limit is reached. At that point, the executor begins failing other queries with cannot allocate memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking during batch processing. This patch determines the proper soft memory limit based on the lesser of the query and process soft limits. When expression evaluation results memory pool exceeds that threshold, the memory pool is freed during which row batch processing is paused. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M .gitignore M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/runtime/mem-tracker-test.cc M be/src/runtime/mem-tracker.cc 9 files changed, 144 insertions(+), 50 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/3 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 3 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/22065/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 3 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 Apr 2026 18:19:58 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/22066/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 4 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 Apr 2026 19:27:12 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. If the batch processing requires evaluating expressions that allocate memory for their results (such as UPPER() or LOWER()), that memory is allocated without checking any memory limits. If a large number of these expression evaluations are performed, memory will be consume until the process memory limit is reached. At that point, the executor begins failing other queries with cannot allocate memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking during batch processing. This patch determines the proper soft memory limit based on the lesser of the query and process soft limits. When expression evaluation results memory pool exceeds that threshold, the memory pool is freed during which row batch processing is paused. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M .gitignore M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h M be/src/runtime/mem-tracker-test.cc M be/src/runtime/mem-tracker.cc 9 files changed, 144 insertions(+), 50 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/4 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 4 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/24159 ) Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/22064/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 1 Gerrit-Owner: Jason Fehr Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Wed, 01 Apr 2026 17:45:42 + Gerrit-HasComments: No
[Impala-ASF-CR] [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing
Jason Fehr has uploaded this change for review. ( http://gerrit.cloudera.org:8080/24159 Change subject: [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing .. [WIP] IMPALA-14863: Fixes Potential OOM During Row Batch Processing Impala checks at the end of every row batch to determine if the query or process memory limit has been exceeded. In certain cases, row batch processing can consume memory until the process memory limit is reached. At that point, the executor begins failing other queries with cannotallocated memory errors as they request memory. However, the row batch processing continues to run since it does no memory limit checking. This patch determines the proper soft memory limit based on the lesser of the query and process soft limit. When expression evaluation results memory pool exceeds that threshold, the memory pool is freed during which row batch processing is paused. Generated-by: Github Copilot (GPT-5.3-Codex) Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af --- M .gitignore M be/src/codegen/gen_ir_descriptions.py M be/src/exec/partitioned-hash-join-node-ir.cc M be/src/exec/partitioned-hash-join-node.cc M be/src/exec/partitioned-hash-join-node.h M be/src/exprs/scalar-expr-evaluator.cc M be/src/exprs/scalar-expr-evaluator.h 7 files changed, 82 insertions(+), 46 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/24159/1 -- To view, visit http://gerrit.cloudera.org:8080/24159 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic54b5c39e1388681275681f22e61b27728dba5af Gerrit-Change-Number: 24159 Gerrit-PatchSet: 1 Gerrit-Owner: Jason Fehr
