Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19387#discussion_r141776977
--- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala ---
@@ -108,11 +108,21 @@ class HashPartitioner(partitions: Int) extends
Partitioner {
class RangePartitioner[K : Ordering : ClassTag, V](
partitions: Int,
rdd: RDD[_ <: Product2[K, V]],
- private var ascending: Boolean = true)
+ private var ascending: Boolean = true,
+ val samplePointsPerPartitionHint: Int = 20)
extends Partitioner {
+ // A constructor declared in order to maintain backward compatibility
for Java, when we add the
+ // 4th constructor parameter samplePointsPerPartitionHint. See
SPARK-22160.
+ // This is added to make sure from a bytecode point of view, there is
still a 3-arg ctor.
+ def this(partitions: Int, rdd: RDD[_ <: Product2[K, V]], ascending:
Boolean) = {
+ this(partitions, rdd, ascending, samplePointsPerPartitionHint = 20)
--- End diff --
The default value is 100 now in SQLConf, shall we also use 100 here for
consistency?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
