On Mon, Jul 28, 2014 at 12:53 PM, Nathan Kronenfeld < nkronenf...@oculusinfo.com> wrote:
> But when done processing, one would still have to pull out the wrapped > object, knowing what it was, and I don't see how to do that. It's pretty tricky to get the level of type safety you're looking for. I know of two ways: 1. Leave RDD and DStream as they are, but define a typeclass <http://danielwestheide.com/blog/2013/02/06/the-neophytes-guide-to-scala-part-12-type-classes.html> that allows converting them to a common DistributedCollection type. Example <https://gist.github.com/ankurdave/f5d4df4b521ac83b9c7d#file-distributed-collection-via-typeclass-scala> . 2. Make RDD and DStream inherit from a common DistributedCollection trait, as in your example, but use F-bounded polymorphism <https://twitter.github.io/scala_school/advanced-types.html#fbounded> to express the concrete types. Example <https://gist.github.com/ankurdave/f5d4df4b521ac83b9c7d#file-distributed-collection-via-fbounded-polymorphism-scala> . Ankur <http://www.ankurdave.com/>
