Ngone51 commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-755867630
Thanks for the review @tgravescs > The only thing is now we could hang infinitely if we never fill the slots, correct? I guess we could hang before if we didn't ever get enough slots as it only info logs at that point. Theoretically, the barrier tsm should get scheduled at the end since we have ensured enough slots (via calculateAvailableSlots) before scheduling it. But I'm not sure now is that, maybe in legacy mode, the barrier tsm would hang because of level resetting? (like you mentioned above). I need to think more about this. > Perhaps we should at least add a debug log (if not info log) when we reset things to let the user know we have enough slots but something else must be using them. Good idea! ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
