Re: What's the meaning when the partitions is zero?

When numPartitions is 0, there is no data in the rdd: so getPartition is never invoked. - Mridul On Friday, September 16, 2016, WangJianfei wrote: > if so, we will get exception when the numPartitions is 0. > def getPartition(key: Any): Int = key match { >

Re: What's the meaning when the partitions is zero?

if so, we will get exception when the numPartitions is 0. def getPartition(key: Any): Int = key match { case null => 0 //case None => 0 case _ => Utils.nonNegativeMod(key.hashCode, numPartitions) } -- View this message in context:

Re: What's the meaning when the partitions is zero?

There are almost no cases in which you'd want a zero-partition RDD. The only one I can think of is an empty RDD, where the number of partitions is irrelevant. Still, I would not be surprised if other parts of the code assume at least 1 partition. Maybe this check could be tightened. It would be