[
https://issues.apache.org/jira/browse/FLINK-19322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17200718#comment-17200718
]
Jingsong Lee commented on FLINK-19322:
--------------------------------------
Hi [~liuyufei] and [~libenchao], I have started a discussion about improving
the new TableSource and TableSink:
[http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-146-Improve-new-TableSource-and-TableSink-interfaces-td45161.html]
It includes parallelism setting, welcome to join the discussion and look
forward to your comments.
> Specified parallelism in create table operation
> -----------------------------------------------
>
> Key: FLINK-19322
> URL: https://issues.apache.org/jira/browse/FLINK-19322
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Common, Table SQL / API
> Affects Versions: 1.12.0
> Reporter: YufeiLiu
> Priority: Major
>
> Flink SQL job only has global parallelism, and all operators share the same
> parallelism. But in many cases like using kafka connector, I want to set
> table parallelism separately to control resource usage.
> * Source & Sink topic has different partition size.
> * Consume multi topic with different throughput.
> * Increase sink parallelism to insure write speed.
> I think it would be better if we can specified parallelism property in create
> table operation, and the actual implementation depends on different
> connectors.
> The create table operation may like this:
> {code:sql}
> CREATE TABLE source (
> ...
> ) WITH (
> 'connector' = 'kafka',
> 'parallelism' = '2'
> )
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
