[
https://issues.apache.org/jira/browse/FLINK-18161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17135006#comment-17135006
]
godfrey he edited comment on FLINK-18161 at 6/14/20, 2:48 AM:
--------------------------------------------------------------
For 1.11, I think we want to just fix the problem that the flink configurations
overwrite sql-client properties.
For 1.12, I want to remove the sql-client's specific properties (e.g.
{{execution.parallelism}}, {{execution.time-characteristic}},
{{execution.restart-strategy.type}}), instead we should use the properties
defined in flink runtime (e.g. {{parallelism.default}},
{{pipeline.time-characteristic}}, {{restart-strategy}}). {{Configuration}}
section in sql-client file can be set options defined in table and runtime. And
then, we can also set checkpointing/watermark in SQL CLI.
I investigate that if we just deprecate the sql-client's specific properties
and also introduce flink runtime's properties. (that means they can both work
in sql-client) the code is very complex to handle the conflicts and users will
very confuse about the behavior. The possible scenarios and solutions are
listed below:
1. conflicts in one sql-client config file, which one should be used? --->
throw exception if exists, add log to tell users which keys are deprecated, and
give suggested keys to replace them.
2. conflicts in default config file and session config file, which one should
be used? ---> throw exception if exists. add log to tell users which keys
are deprecated, and give suggested keys to replace them.
3. conflicts in {{set}} operation, which one should be used? ---> if the given
key in {{set}} operation is deprecated, check wether there is a corresponding
new key exists in {{Environment}}. if exists, remove it first (to make sure the
final result is correct), and then set the value to {{Environment}} . vice
versa. If users execute {{set}} command to list all properties, they may find
some properties do not exist.
WDYT? [~dwysakowicz] [~jark]
was (Author: godfreyhe):
For 1.11, I think we want to just fix the problem that the flink configurations
overwrite sql-client properties.
For 1.12, I want to remove the sql-client's specific properties (e.g.
{{execution.parallelism}}, {{execution.time-characteristic}},
{{execution.restart-strategy.type}}), instead we should use the properties
defined in flink runtime (e.g. {{parallelism.default}},
{{pipeline.time-characteristic}}, {{restart-strategy}}). {{Configuration}}
section in sql-client file can be set options defined in table and runtime. And
then, we can also set checkpointing/watermark in SQL CLI.
I investigate that if we just deprecate the sql-client's specific properties
and also introduce flink runtime's properties. (that means they can both work
in sql-client) the code is very complex to handle the conflicts and users will
very confuse about the behavior. The possible scenarios and solutions are
listed below:
1. conflicts in one sql-client config file, which one should be used? --->
throw exception if exists, add log to tell users which keys are deprecated, and
give suggested keys to replace them.
2. conflicts in default config file and session config file, which one should
be used? ---> throw exception if exists. add log to tell users which keys
are deprecated, and give suggested keys to replace them.
3. conflicts in {{set}} operation, which one should be used? ---> if the given
key in {{set}} operation is deprecated, check wether there is a corresponding
new key exists in {{Environment}}. if exists, remove it first (to make sure the
final result is correct), and then set the value to {{Environment}} . vice
versa.
WDYT? [~dwysakowicz] [~jark]
> Changing parallelism is not possible in sql-client.sh
> -----------------------------------------------------
>
> Key: FLINK-18161
> URL: https://issues.apache.org/jira/browse/FLINK-18161
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
> Affects Versions: 1.11.0
> Reporter: Robert Metzger
> Assignee: godfrey he
> Priority: Critical
> Fix For: 1.11.0
>
>
> I tried using
> {code}
> SET execution.parallelism=12
> {code}
> and changing the parallelism in the configuration file.
> My SQL queries were always running with p=1 for all operators.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
