[
https://issues.apache.org/jira/browse/FLINK-41310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bonnie Varghese updated FLINK-41310:
------------------------------------
Summary: BatchExecAdaptiveJoin should take external-buffer-memory from the
compiled plan (was: BatchExecAdaptiveJoin sizes the external buffer from the
runtime config instead of the compiled plan)
> BatchExecAdaptiveJoin should take external-buffer-memory from the compiled
> plan
> -------------------------------------------------------------------------------
>
> Key: FLINK-41310
> URL: https://issues.apache.org/jira/browse/FLINK-41310
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Planner
> Affects Versions: 2.5.0
> Reporter: Bonnie Varghese
> Assignee: Bonnie Varghese
> Priority: Major
> Fix For: 2.5.0
>
>
> BatchExecAdaptiveJoin computes the join's managed memory from the compiled
> plan's config. It builds the actual join operator at runtime, though, and
> that operator reads
> `table.exec.resource.external-buffer-memory` from the config of the session
> that executes the plan.
> If the two sessions use different values, the buffer no longer fits the
> budget. There are two problems associated with this:
> 1. A large enough value fails the job with "Memory size is too small".
> 2. A smaller difference just takes memory from the sorters without any error.
> BatchExecHashJoin and BatchExecSortMergeJoin aren't affected, because they
> read both values from the plan. This should be fixed after FLINK-41263 is
> merged.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
