Github user jinxing64 commented on the issue:
https://github.com/apache/spark/pull/16503
@zsxing, @vanzin
Maybe using `ask` in method `canCommit` is not suitable(i think). Because
`ask` returns a Future, but it should be a blocking process to get result of
`AskPermissionToCommitOutput` in `canCommit`. RPC layer doesn't drop message
but message can be timeout. If timeout happens, we can retry or fail the task
and let spark reschedule it, both of them are ok. But for some heavy tasks,
retry is light weight (i think). So my suggestion is to keep the `askWithRetry`
here and change the receiver to be idempotent. What do you think ?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
