Looks like NumericRange in Scala is just a joke. scala> val x = 0.0 to 1.0 by 0.1 x: scala.collection.immutable.NumericRange[Double] = NumericRange(0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6, 0.7, 0.7999999999999999, 0.8999999999999999, 0.9999999999999999)
scala> x.take(3) res1: scala.collection.immutable.NumericRange[Double] = NumericRange(0.0, 0.1, 0.2) scala> x.drop(3) res2: scala.collection.immutable.NumericRange[Double] = NumericRange(0.30000000000000004, 0.4, 0.5, 0.6, 0.7, 0.7999999999999999, 0.8999999999999999, 0.9999999999999999) So far so good. scala> x.drop(3).take(3) res3: scala.collection.immutable.NumericRange[Double] = NumericRange(0.30000000000000004, 0.4) Why only two values? Where's 0.5? scala> x.drop(6) res4: scala.collection.immutable.NumericRange[Double] = NumericRange(0.6000000000000001, 0.7000000000000001, 0.8, 0.9) And where did the last value disappear now? You have to approach Scala with a healthy amount of distrust. You're on the right track with toArray. On Fri, Apr 18, 2014 at 8:01 PM, Mark Hamstra <m...@clearstorydata.com>wrote: > Please file an issue: Spark Project > JIRA<https://issues.apache.org/jira/browse/SPARK> > > > > On Fri, Apr 18, 2014 at 10:25 AM, Aureliano Buendia > <buendia...@gmail.com>wrote: > >> Hi, >> >> I just notices that sc.makeRDD() does not make all values given with >> input type of NumericRange, try this in spark shell: >> >> >> $ MASTER=local[4] bin/spark-shell >> >> scala> sc.makeRDD(0.0 to 1 by 0.1).collect().length >> >> *8* >> >> >> The expected length is 11. This works correctly when lanching spark with >> only one core: >> >> >> $ MASTER=local[1] bin/spark-shell >> >> scala> sc.makeRDD(0.0 to 1 by 0.1).collect().length >> >> *11* >> >> >> This also works correctly when using toArray(): >> >> $ MASTER=local[4] bin/spark-shell >> >> scala> sc.makeRDD((0.0 to 1 by 0.1).*toArray*).collect().length >> >> *8* >> > >
