azagrebin commented on pull request #13628: URL: https://github.com/apache/flink/pull/13628#issuecomment-709376678
I generally agree with the proposed steps. We should try to take into account locations of all available producers for a given `SlotSharingGroup`, hence futures are not needed in case of pipelined region scheduling. I have a note about `SlotSharingGroup` assignment for batch scheduling: > each logical pipelined region has a different `SlotSharingGroup` Although, this assignment simplifies things, given @tillrohrmann 's example for the parallelism of one (as I understand `v1 --> v2` and `v3 --> v4` are basically disjoint pipelines): > ``` > v1 --> v2 > v3 --> v4 > ``` > > here `v1, v2, v3, v4` are `JobVertices` and `-->` is a blocking data exchange v11 and v31 executions cannot run at the same time if there is only one slot. Not sure, how important it is for performance, probably depends on the use case. On the other hand, deciding about such fine-grained slot sharing for some bigger example may be a non-trivial task and up to the user to decide using API. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
