[
https://issues.apache.org/jira/browse/FLINK-37999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gustavo de Morais updated FLINK-37999:
--------------------------------------
Description:
This issue aims to verify
[FLIP-516.|https://issues.apache.org/jira/browse/FLINK-35293]
We now support a StreamingMultiJoinOperator that performs a join across N
tables without intermediate state. We've done some testing with considerable
performance and state improvement results.
We'd like to have some cross team testing to validate it. We want to cover the
following cases
* Test 1: Enable multi joins, confirm 2/3 way inner joins are supported and
produce correct results. Materialized results should be the same as binary
joins.
* Test 2: Enable multi joins, confirm 2/3 way left joins are supported and
produce correct results. Materialized results should be the same as binary
joins.
* Test 3: Enable 5, 10, 20 way joins with/without primary keys and confirm
output plans look good with `explain`.
* Test 4: Enable 5, 10, 20 way joins with/without primary keys and confirm
they run as expected - checking results' correctness here is usually
complicated and not strictly required.
Multi joins can be enabled with ??SET 'table.optimizer.multi-join.enabled' =
'true';.?? Currently, we have streaming mode support for inner and left joins.
was:
This issue aims to verify FLIP-445.
The Hive Source now supports dynamic parallelism inference, and the default
parallelism inference mode has also changed from static to dynamic inference.
For detailed information, please refer to the
[documentation|https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/hive/hive_read_write/#source-parallelism-inference].
We may need to cover the following types of test cases:
Test 1: Confirm that the dynamic parallelism inference feature is effective.
Test 2: Confirm whether it is possible to switch the parallelism inference mode
or disable parallelism inference through the config option
`table.exec.hive.infer-source-parallelism.mode`.
Test 3: Construct scenarios where DynamicPartitionPruning is effective, and
confirm whether the dynamically inferred source parallelism has been optimized.
Follow up the test for https://issues.apache.org/jira/browse/FLINK-35293
> Release Testing Instructions: Verify FLIP-516: Streaming Multi-Way Join
> Optimization
> ------------------------------------------------------------------------------------
>
> Key: FLINK-37999
> URL: https://issues.apache.org/jira/browse/FLINK-37999
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Planner
> Affects Versions: 2.1.0
> Reporter: dalongliu
> Assignee: Gustavo de Morais
> Priority: Blocker
> Labels: release-testing
> Fix For: 2.1.0
>
>
> This issue aims to verify
> [FLIP-516.|https://issues.apache.org/jira/browse/FLINK-35293]
> We now support a StreamingMultiJoinOperator that performs a join across N
> tables without intermediate state. We've done some testing with considerable
> performance and state improvement results.
>
> We'd like to have some cross team testing to validate it. We want to cover
> the following cases
> * Test 1: Enable multi joins, confirm 2/3 way inner joins are supported and
> produce correct results. Materialized results should be the same as binary
> joins.
> * Test 2: Enable multi joins, confirm 2/3 way left joins are supported and
> produce correct results. Materialized results should be the same as binary
> joins.
> * Test 3: Enable 5, 10, 20 way joins with/without primary keys and confirm
> output plans look good with `explain`.
> * Test 4: Enable 5, 10, 20 way joins with/without primary keys and confirm
> they run as expected - checking results' correctness here is usually
> complicated and not strictly required.
>
> Multi joins can be enabled with ??SET 'table.optimizer.multi-join.enabled' =
> 'true';.?? Currently, we have streaming mode support for inner and left joins.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
