Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2337#issuecomment-56281339
@rxin @pwendell Since we have job groups and the ability to cancel all jobs
running in a job group (`sc.cancelJobGroup()`), then why do we need
FutureAction? It looks like the only benefit that it offers over regular Scala
`Future` is cancellation.
I imagine that many developers would like to be able to fire off an entire
workflow, potentially comprising multiple actions, monitor its overall
progress, and cancel the whole thing. It seems like job groups offer a
strictly more powerful set of features that allows users to perform
progress-monitoring and cancellation on entire workflows, not just individual
actions.
If the motivation for FutureAction is that job groups are inconvenient for
simple things, then I think we can address that by adding convenience wrappers
that act like Python context managers and make it easy to run a block of code
inside of a particular job group. Or, we could add an API that executes an
arbitrary user-defined code block using a specified job group and returns a
cancelable future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
