Github user mateiz commented on a diff in the pull request:
https://github.com/apache/spark/pull/1500#discussion_r15153273
--- Diff: core/src/test/scala/org/apache/spark/PartitioningSuite.scala ---
@@ -91,6 +91,17 @@ class PartitioningSuite extends FunSuite with
SharedSparkContext with PrivateMet
}
}
+ test("RangePartitioner for keys that are not Comparable (but with
Ordering)") {
+ // Row does not extend Comparable, but has an implicit Ordering
defined.
+ implicit object RowOrdering extends Ordering[Row] {
+ override def compare(x: Row, y: Row) = x.value - y.value
+ }
+
+ val rdd = sc.parallelize(1 to 4500).map(x => (Row(x), Row(x)))
+ val partitioner = new RangePartitioner(1500, rdd)
+ partitioner.getPartition(Row(100))
--- End diff --
Shouldn't we also add a test where we do sortByKey and then collect and
make sure stuff is in order?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
