[
https://issues.apache.org/jira/browse/FLINK-19715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler reassigned FLINK-19715:
----------------------------------------
Assignee: Chesnay Schepler
> Optimize re-assignment of excess resources
> ------------------------------------------
>
> Key: FLINK-19715
> URL: https://issues.apache.org/jira/browse/FLINK-19715
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Critical
> Fix For: 1.13.0
>
>
> The {{JobScopedResourceTracker}} tracks acquired resources that exceed the
> jobs requirements as {{excess}} resources.
> Whenever the requirements increase, or a (non-excess) resource was lost, we
> try to assign any excess resources we have to fill either fulfill the
> requirements of fill in the lost resource.
> This re-assignment is currently implemented by doing a full copy of map
> containing the excess resources, and going through the usual code path for
> acquired resources.
> This is fine in terms of correctness (although it can cause misleading log
> messages), but in the worst case, where we cannot re-assign any excess
> resources, we not only rebuild the original map in the process, but also
> potentially iterate over every outstanding requirement for every excess slot.
> We should optimize this step by iterating over the excess resource map once,
> removing items on-demand and aborting early for a given excess resource
> profile if no matching requirement could be found.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
