On Wed, Jun 18, 2014 at 9:39 PM, Dmitriy Lyubimov <dlie...@gmail.com> wrote:
> > > > BTW Spark rdd type is RDD[K:ClassTag] as well. nobody yet complained. not > a single time. For all their list activity. > > >> Actually it is even "scarier" in Spark. Consider this type system: To enable groupBy, for example, RDD needs to match RDD[(K:ClassTag,V:ClassTag)]. To enable sort, RDD needs to match RDD[(K<%Comparable:ClassTag, V:ClassTag)]. And to enable persisting something to a sequence file, it has to match RDD[(K<%WritableComparable:ClassTag,V<%Writable :ClassTag)]. And probably even something else i don't immediately remember. Compared to these, we are just simplicity itself. Still, nobody yet thought of complaining about those.
