[
https://issues.apache.org/jira/browse/FLINK-17392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17092914#comment-17092914
]
Jark Wu commented on FLINK-17392:
---------------------------------
Hi [~phoenixjiangnan], sorry, I'm not fully understand.
> in minicluster, if it is single parallelism job, then chain all operators
> together
Why do users want to avoid shuffling in minicluster? Performance purpose? Why
do users care performance in minicluster? I think minicluster is not used for
production for now.
However, removing shuffling is in our roadmap but not in a high priority.
Removing shuffling shouldn't be bound by minicluster. But this is not a small
effort, we need to be careful with the keyed state.
> enable configuring minicluster in Flink SQL in IDE.
I don't think exposing minicluster to TableEnvironment is a good idea. Even
{{StreamExecutionEnvironment}} doesn't provide this. If users want to modify
resources. He can just set parallelism using the {{parallelism.default}}
configuration.
> enable configuring minicluster resources in Flink SQL in IDE
> ------------------------------------------------------------
>
> Key: FLINK-17392
> URL: https://issues.apache.org/jira/browse/FLINK-17392
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / API
> Affects Versions: 1.11.0
> Reporter: Bowen Li
> Assignee: Kurt Young
> Priority: Major
> Fix For: 1.11.0
>
>
> It's very common case that users who want to learn and test Flink SQL will
> try to run a SQL job in IDE like Intellij, with Flink minicluster. Currently
> it's fine to do so with a simple job requiring only one task slot, which is
> the default resource config of minicluster.
> However, users cannot run even a little bit more complicated job since they
> cannot configure task slots of minicluster thru Flink SQL, e.g. single
> parallelism job requires shuffle. This incapability has been very frustrating
> to new users.
> There are two solutions to this problem:
> - in minicluster, if it is single parallelism job, then chain all operators
> together
> - enable configuring minicluster in Flink SQL in IDE.
> The latter feels more proper.
> Expected: users can configure minicluster resources via either SQL ("set
> ...=...") or TableEnvironment ("tEnv.setMiniclusterResources(..., ...)").
> [~jark] [~lzljs3620320]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
