Github user reggert commented on the issue:
https://github.com/apache/spark/pull/15899
Using RDD's in `for` comprehensions dates back _at least_ to the examples
given in the old 2010 AMPLab paper "Spark: Cluster Computing with Working
Sets". `for` comprehensions are in no way limited to local collections. They
provide syntactic sugar that, for many use cases, makes the code easier to
comprehend than a series of chained method calls.
Other examples of using them for non-collections include the Scala `Future`
class as well as query construction using the SLICK database access library.
I agree that a similar change should be made for other abstractions built
on top of RDD's, such as `DataSet`s and `DStream`s, though. I'll add those when
I get a chance.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
