zhijiangW edited a comment on issue #7186: [FLINK-10941] Keep slots which contain unconsumed result partitions URL: https://github.com/apache/flink/pull/7186#issuecomment-453919737 Hi @QiLuo-BD , very sorry for the late response! During these days, the proposal of shuffle manager is under discussion and already reaches an basic agreement. So the formal solution for this issue is introducing a ShuffleMaster component in JobMaster for global partition management, then the life cycle of task/partition/TM could be decoupled completely. And the fist version wishes to be covered in FLINK 1.8. I think your current solution might also work from functional aspect in current implementation. But it seems a little hacky/tricky, the reasons are as follows: - Task and its partitions are still coupled. The task is finished, but `task#canBeReleased` is based on `partition#isReleased`. - TM and internal partitions are still coupled. For external shuffle service in future, the TM might be released if the partition is written onto external store. So my suggestion is if the current modifications are small enough to work around temporarily, then it might make sense for merging. But this PR seems a bit large, involved in slot manager, task slot and network stack modifications. It is better to wait for the formal FLINK 1.8 or adjust the parameters to work around currently. :)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
