Github user squito commented on the issue:
https://github.com/apache/spark/pull/19041
Thanks @brad-kaiser -- want to re-iterate my comment from Feb 2nd, I think
that is really the most important part to address before getting into the
details of the current implementation:
> Thought some more about the race between RemoveBlock getting sent back
from the executor vs when the CacheRecoveryManager tries to replicate the next
block -- actually why is there the back-and-forth with the driver for every
block? Why isn't there just one message from the CacheRecoveryManager to the
executor, saying "Drain all RDD blocks" and then one message from the executor
back to the driver when its done?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
