Josh Rosen created SPARK-3626:
---------------------------------
Summary: Replace AsyncRDDActions with a more general async. API
Key: SPARK-3626
URL: https://issues.apache.org/jira/browse/SPARK-3626
Project: Spark
Issue Type: Improvement
Components: Spark Core
Reporter: Josh Rosen
Assignee: Josh Rosen
The experimental AsyncRDDActions APIs seem to only exist in order to enable job
cancellation.
We've been considering extending these APIs to support progress monitoring, but
this would require stabilizing them so they're no longer {{@Experimental}}.
Instead, I propose to replace all of the AsyncRDDActions with a mechanism based
on job groups which allows arbitrary computations to be run in job groups and
supports cancellation / monitoring of Spark jobs launched from those
computations.
(full design pending; see my GitHub PR for more details).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
