[
https://issues.apache.org/jira/browse/FLINK-37160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928645#comment-17928645
]
Lei Yang edited comment on FLINK-37160 at 2/20/25 5:21 AM:
-----------------------------------------------------------
Verification results:
The results match the description.
Verification Steps:
# Construct a join case that does not meet the criteria for conversion to a
broadcast join during the table compilation phase, but satisfies the conditions
for being converted to a broadcast join at runtime. Set
{{*table.optimizer.adaptive-broadcast-join.strategy*}} to {{{}*auto*{}}}.
Submit the job and observe from the UI that the join operator is converted to
an AdaptiveJoin operator and optimized into a broadcast join.
# Construct a join case that meets the criteria for conversion to a broadcast
join during the table compilation phase. Set
*{{table.optimizer.adaptive-broadcast-join.strategy}}* to
*{{{}runtime_only{}}}.* Submit the job and observe from the UI that the join
operator is converted to an AdaptiveJoin operator and optimized into a
broadcast join.
# Reuse the data from Step 1, set
{{*table.optimizer.adaptive-broadcast-join.strategy*}} to {*}{{none}}{*}.
Submit the job and observe from the UI that the join operator is not optimized
into a broadcast join under any circumstances.
was (Author: noorall):
Verification results:
The results match the description.
Verification Steps:
# Construct a join case that does not meet the criteria for conversion to a
broadcast join during the table compilation phase, but satisfies the conditions
for being converted to a broadcast join at runtime. Set
{{*table.optimizer.adaptive-broadcast-join.strategy*}} to {{{}*auto*{}}}.
Submit the job and observe from the UI that the join operator is converted to
an AdaptiveJoin operator and optimized into a broadcast join.
# Construct a join case that meets the criteria for conversion to a broadcast
join during the table compilation phase. Set
*{{table.optimizer.adaptive-broadcast-join.strategy}}* to
*{{{}runtime_only{}}}.* Submit the job and observe from the UI that the join
operator is converted to an AdaptiveJoin operator and optimized into a
broadcast join.
# Reuse the data from Step 1, set
{{*table.optimizer.adaptive-broadcast-join.strategy*}} to {*}{{none}}{*}.
Submit the job and observe from the UI that the join operator is not optimized
into a broadcast join under any circumstances.
> Cross-team verification for "Adaptive broadcast join for batch jobs"
> --------------------------------------------------------------------
>
> Key: FLINK-37160
> URL: https://issues.apache.org/jira/browse/FLINK-37160
> Project: Flink
> Issue Type: Sub-task
> Reporter: Junrui Lee
> Assignee: Lei Yang
> Priority: Blocker
> Fix For: 2.0.0
>
>
> In Flink 2.0, we support the capability of adaptive broadcast join for batch
> jobs, which can determine at runtime whether the small table can be broadcast
> based on the actual input data size of the join operator. It then transforms
> the operators that meet the condition to achieve better join performance.
> We may need the following tests:
> 1. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
> set to 'auto', we may need to construct a case (e.g. a case where the source
> includes a downstream filter operator) such that the join operator is not
> optimized as a broadcast hash join during the table compilation phase but can
> be dynamically optimized as a broadcast hash join at runtime.
> 2. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
> set to 'runtime_only', to verify that the join operator is not optimized into
> a broadcast hash join during the table compilation phase, even if the small
> table side meets the broadcast threshold, while also being able to
> dynamically convert into a broadcast hash join at runtime.
> 3. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` as
> none, and verify that the join operator will not be optimized into a
> broadcast join under any circumstances.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
