[
https://issues.apache.org/jira/browse/FLINK-19714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu updated FLINK-19714:
----------------------------
Description:
Currently, job vertices of a DataSet job will be put into the same
SlotSharingGroup. It does not make sense because producer and consumer
pipelined regions will not be able to share slots. This can lead to a waste of
managed memory.
We should assign different logical pipelined regions of a DataSet job to
different slot sharing group, like how we does for Blink planner batch jobs.
Then we can refine the managed memory fraction calculation for DataSet
operators, i.e. TaskConfig.MEMORY_DRIVER and TaskConfig.MEMORY_INPUT_PREFIX.*
was:
Currently, job vertices of a DataSet job will be put into the same
SlotSharingGroup. It does not make sense because producer and consumer
pipelined regions will not be able to share slots.
We should
> Assign different logical pipelined regions of a DataSet job to different slot
> sharing group
> -------------------------------------------------------------------------------------------
>
> Key: FLINK-19714
> URL: https://issues.apache.org/jira/browse/FLINK-19714
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.12.0
> Reporter: Zhu Zhu
> Priority: Major
> Fix For: 1.12.0
>
>
> Currently, job vertices of a DataSet job will be put into the same
> SlotSharingGroup. It does not make sense because producer and consumer
> pipelined regions will not be able to share slots. This can lead to a waste
> of managed memory.
> We should assign different logical pipelined regions of a DataSet job to
> different slot sharing group, like how we does for Blink planner batch jobs.
> Then we can refine the managed memory fraction calculation for DataSet
> operators, i.e. TaskConfig.MEMORY_DRIVER and TaskConfig.MEMORY_INPUT_PREFIX.*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
