Moments after sending this I tracked down the issue to a subsequent
transformation of .top(10) which ran without error in Spark 1.6 (but who
knows how it was sorting since the POJO doesn't implement Comparable)
whereas in Spark 2.0 it now fails if the POJO is not Comparable.
The new behavior is be
Hi,
I have some working Java code with Spark 1.6 that I am upgrading to Spark
2.0
I have this valid RDD:
JavaRDD popSummary
I want to sort using a function I provide for performing comparisons:
popSummary
.sortBy((Function) p -> p.getMale()
* 1.0f / p.getFemale(), true, 1)
