dmvk commented on pull request #18238: URL: https://github.com/apache/flink/pull/18238#issuecomment-1002923253
@tillrohrmann Thanks for the first pass on the overall direction! I was thinking about the algorithmic complexity and I think we can do some optimizations by leveraging few facts: - For assigning slots to "execution groups": We treat each vertex in a slot equally. Considering the simple case where all vertices have the same parallelism, we can simply count the "weight" / "# of key groups" of a single vertex. 🤔 - For splitting slots to slot sharing groups: We can leverage the fact that they had a disjoint set of slots in the previous run. I'll probably continue with finishing the test suite first and then we can focus on the performance. But something along these lines might do the trick. I'm also thinking whether it would make a difference if we also leverage TM locality 🤔 (this could be also a follow up) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
