[
https://issues.apache.org/jira/browse/FLINK-23372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17392174#comment-17392174
]
Stephan Ewen commented on FLINK-23372:
--------------------------------------
This is a pretty foundational change and decision here. It may make sense, but
can we outline the pros/cons here to make that decision transparent?
> Disable AllVerticesInSameSlotSharingGroupByDefault in DataStream batch mode
> ---------------------------------------------------------------------------
>
> Key: FLINK-23372
> URL: https://issues.apache.org/jira/browse/FLINK-23372
> Project: Flink
> Issue Type: Sub-task
> Components: API / DataStream
> Reporter: Timo Walther
> Assignee: Timo Walther
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.14.0
>
>
> In order to unify the behavior of DataStream API and Table API batch mode, we
> should disable AllVerticesInSameSlotSharingGroupByDefault also in DataStream
> API.
> FLINK-20001 reverted setting this flag but without concrete arguments and the
> following comment: {{reconsider actually setting this flag in the future}}
> After a offline chat with [~zhuzh], we should introduce this again for
> consistency:
> {code}
> The goal to assign different regions to different slot sharing groups by
> default is to reduce waste of resources. In batch jobs, there can be one
> region which has data dependency on another region. And the resource
> computation for slots and managed memory will be affected:
> 1. If these regions are in the same slot sharing group, the group will
> require a large slot which can host tasks from both the regions.
> 2. In managed memory fraction computing, tasks from both regions will be
> considered to compete for managed memory, so each task will be assigned with
> a smaller managed memory fraction (FLIP-53).
> However, those regions will not run at the same time and results in a waste
> of resources.
> For streaming jobs, all tasks will run at the same time. So assigning them to
> the same slot sharing group will not result resource waste.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
