I have a class which is a subclass of Tuple2, and I want to use it with
PairRDDFunctions. However, I seem to be limited by the invariance of T in
RDD[T] (see SPARK-1296 https://issues.apache.org/jira/browse/SPARK-1296).
My Scala-fu is weak: the only way I could think to make this work would be
to
I think you should also be able to get away with casting it back and forth
in this case using .asInstanceOf.
On Wed, Nov 19, 2014 at 4:39 PM, Daniel Siegmann daniel.siegm...@velos.io
wrote:
I have a class which is a subclass of Tuple2, and I want to use it with
PairRDDFunctions. However, I
Casting to Tuple2 is easy, but the output of reduceByKey is presumably a
new Tuple2 instance so I'll need to map those to new instances of my class.
Not sure how much overhead will be added by the creation of those new
instances.
If I do that everywhere in my code though, it will make the code
