[
https://issues.apache.org/jira/browse/FLINK-23372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392186#comment-17392186
]
Timo Walther edited comment on FLINK-23372 at 8/3/21, 9:10 AM:
---------------------------------------------------------------
[~sewen] That's what I actually tried in the JIRA description by copying our
offline chat. I don't have enough background/field knowledge to decide this on
my own and was gathering feedback from [~dwysakowicz] [~zhuzh] and
[~lzljs3620320].
was (Author: twalthr):
[~sewen] That's what I actually tried in the JIRA description by copying our
offline chat. I don't have enough background knowledge to decide this on my own
and was gathering feedback from [~dwysakowicz] [~zhuzh] and [~lzljs3620320].
> Disable AllVerticesInSameSlotSharingGroupByDefault in DataStream batch mode
> ---------------------------------------------------------------------------
>
> Key: FLINK-23372
> URL: https://issues.apache.org/jira/browse/FLINK-23372
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
> Reporter: Timo Walther
> Assignee: Timo Walther
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> In order to unify the behavior of DataStream API and Table API batch mode, we
> should disable AllVerticesInSameSlotSharingGroupByDefault also in DataStream
> API.
> FLINK-20001 reverted setting this flag but without concrete arguments and the
> following comment: {{reconsider actually setting this flag in the future}}
> After a offline chat with [~zhuzh], we should introduce this again for
> consistency:
> {code}
> The goal to assign different regions to different slot sharing groups by
> default is to reduce waste of resources. In batch jobs, there can be one
> region which has data dependency on another region. And the resource
> computation for slots and managed memory will be affected:
> 1. If these regions are in the same slot sharing group, the group will
> require a large slot which can host tasks from both the regions.
> 2. In managed memory fraction computing, tasks from both regions will be
> considered to compete for managed memory, so each task will be assigned with
> a smaller managed memory fraction (FLIP-53).
> However, those regions will not run at the same time and results in a waste
> of resources.
> For streaming jobs, all tasks will run at the same time. So assigning them to
> the same slot sharing group will not result resource waste.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
