Github user squito commented on the issue:
https://github.com/apache/spark/pull/21346
btw I may have made the pull-based approach sound more complex than I meant
to, I'm happy to take that approach if you think its better. The fact the
replication is synchronous doesn't really matter, I just meant its not a
fire-and-forget msg, we have to setup the callbacks to confirm the block has
been fetched (or a failure). It just seemed like extra indirection to me, and
I thought it would be better to stay closer to the UploadBlock path.
Are there particular reasons you think that approach would be better? I
guess the receiver can throttle the requests, but on the other hand the task on
the sender will block waiting for the replication to finish (whether its
success or failure), so we really don't want it to wait too long.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
