Jeyhun Karimov created FLINK-34442:
--------------------------------------
Summary: Support optimizations for pre-partitioned [external] data
sources
Key: FLINK-34442
URL: https://issues.apache.org/jira/browse/FLINK-34442
Project: Flink
Issue Type: Improvement
Components: Table SQL / API, Table SQL / Planner
Affects Versions: 1.18.1
Reporter: Jeyhun Karimov
There are some use-cases in which data sources are pre-partitioned:
- Kafka broker is already partitioned w.r.t. some key
- There are multiple Flink jobs that materialize their outputs and read them
as input subsequently
One of the main benefits is that we might avoid unnecessary shuffling.
There is already an experimental feature in DataStream to support a subset of
these [1].
We should support this for Flink Table/SQL as well.
[1]
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/experimental/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
