Github user mengxr commented on a diff in the pull request:
https://github.com/apache/spark/pull/1787#discussion_r16327236
--- Diff: core/src/main/scala/org/apache/spark/rdd/DoubleRDDFunctions.scala
---
@@ -96,6 +96,10 @@ class DoubleRDDFunctions(self: RDD[Double]) extends
Logging with Serializable {
*/
def histogram(bucketCount: Int): Pair[Array[Double], Array[Long]] = {
// Compute the minimum and the maxium
+ def customRange(min: Double, max: Double, steps: Int):
IndexedSeq[Double] = {
+ val span = max - min
--- End diff --
Both approaches should give reasonable results because the last one is
skipped. The edge case is that both min and max are very very small values
(customRange1 computes stepSize first):
~~~
scala> customRange(1e-322, 2e-322, 10)
res15: IndexedSeq[Double] = Vector(1.0E-322, 1.1E-322, 1.2E-322, 1.3E-322,
1.4E-322, 1.5E-322, 1.58E-322, 1.7E-322, 1.8E-322, 1.9E-322, 2.0E-322)
scala> customRange1(1e-322, 2e-322, 10)
res14: IndexedSeq[Double] = Vector(1.0E-322, 1.1E-322, 1.2E-322, 1.3E-322,
1.4E-322, 1.5E-322, 1.58E-322, 1.7E-322, 1.8E-322, 1.9E-322, 2.0E-322)
scala> customRange(1e-323, 2e-323, 10)
res12: IndexedSeq[Double] = Vector(1.0E-323, 1.0E-323, 1.0E-323, 1.5E-323,
1.5E-323, 1.5E-323, 1.5E-323, 1.5E-323, 2.0E-323, 2.0E-323, 2.0E-323)
scala> customRange1(1e-323, 2e-323, 10)
res13: IndexedSeq[Double] = Vector(1.0E-323, 1.0E-323, 1.0E-323, 1.0E-323,
1.0E-323, 1.0E-323, 1.0E-323, 1.0E-323, 1.0E-323, 1.0E-323, 2.0E-323)
~~~
I think we can ignore this edge case.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
