Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/8180#issuecomment-137459384
Before deciding whether it's a big change, do also take a look at the
change. As I said, it's only about 100-200 lines of actual changes, the rest is
comments for the existing code and tests. And the actual mechanism by which
this works already existed in the DAGScheduler design (the scheduler already
tracked map stages, including sharing them between jobs, etc, but there was no
way to submit a map stage by itself). If you want to make it seem less scary, I
can send a PR without the comments in the original scheduler :).
By the way, we have a working prototype of SPARK-9850 end-to-end (including
prototypes of the other JIRAs in it) so it's not like this is untested either.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
