[
https://issues.apache.org/jira/browse/FLINK-8712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17631364#comment-17631364
]
Xintong Song commented on FLINK-8712:
-------------------------------------
When this issue is originally reported, one major problem is user not being
able to configure the number of slots/TM in MiniCluster. As these are
configurable now, it is already possible to execute ideally any job in
MiniCluster with a potential manual tuning. What you are trying to do is to
make such tuning automatic, which is more a usability improvement rather than a
bug fixing to me. Moreover, the improvement only benefits some non-production
corner cases (MiniCluster is not designed for production usage and it's not
very common that users tune slot sharing groups for non-production jobs IMO).
Therefore, I don't think this improvement worths introducing an imperfect
approach that only works fine for streaming jobs. I'd be positive for this
improvement if there's a proposal works fine for both streaming and batch
workloads, negative otherwise.
> Cannot execute job with multiple slot sharing groups on LocalExecutor
> ---------------------------------------------------------------------
>
> Key: FLINK-8712
> URL: https://issues.apache.org/jira/browse/FLINK-8712
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Task
> Affects Versions: 1.5.0
> Reporter: Till Rohrmann
> Priority: Not a Priority
> Labels: auto-deprioritized-critical, auto-deprioritized-major,
> auto-deprioritized-minor
>
> Currently, it is not possible to run a job with multiple slot sharing groups
> on the LocalExecutor. The problem is that we determine the number of required
> slots simply by looking for the max parallelism of the job but do not
> consider slot sharing groups.
>
> {code:java}
> // set up the streaming execution environment
> final StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> env.setParallelism(1);
> final DataStreamSource<Integer> input = env.addSource(new InfinitySource());
> final SingleOutputStreamOperator<Integer> different = input.map(new
> MapFunction<Integer, Integer>() {
> @Override
> public Integer map(Integer integer) throws Exception {
> return integer;
> }
> }).slotSharingGroup("Different");
> different.print();
> // execute program
> env.execute("Flink Streaming Java API Skeleton");{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
