Github user attilapiros commented on a diff in the pull request:
https://github.com/apache/spark/pull/20686#discussion_r171657137
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/QuantileDiscretizerSuite.scala
---
@@ -324,19 +352,46 @@ class QuantileDiscretizerSuite
.setStages(Array(discretizerForCol1, discretizerForCol2,
discretizerForCol3))
.fit(df)
- val resultForMultiCols = plForMultiCols.transform(df)
- .select("result1", "result2", "result3")
- .collect()
-
- val resultForSingleCol = plForSingleCol.transform(df)
- .select("result1", "result2", "result3")
- .collect()
+ val expected = Seq(
+ (0.0, 0.0, 0.0),
+ (0.0, 0.0, 1.0),
+ (0.0, 0.0, 1.0),
+ (0.0, 1.0, 2.0),
+ (0.0, 1.0, 2.0),
+ (0.0, 1.0, 2.0),
+ (0.0, 1.0, 3.0),
+ (0.0, 2.0, 4.0),
+ (0.0, 2.0, 4.0),
+ (1.0, 2.0, 5.0),
+ (1.0, 2.0, 5.0),
+ (1.0, 2.0, 5.0),
+ (1.0, 3.0, 6.0),
+ (1.0, 3.0, 6.0),
+ (1.0, 3.0, 7.0),
+ (1.0, 4.0, 8.0),
+ (1.0, 4.0, 8.0),
+ (1.0, 4.0, 9.0),
+ (1.0, 4.0, 9.0),
+ (1.0, 4.0, 9.0)
+ ).toDF("result1", "result2", "result3")
+ .collect().toSeq
--- End diff --
I think having a bit bigger array in the test is better then checking
df.transform result against df.transform result (as the function testTransform
uses df.transform for DF tests).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
