Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/9756#discussion_r46610719
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/regression/LinearRegressionSuite.scala
---
@@ -592,21 +594,47 @@ class LinearRegressionSuite
}
/*
- Use the following R code to generate model training results.
-
- predictions <- predict(fit, newx=features)
- residuals <- label - predictions
- > mean(residuals^2) # MSE
- [1] 0.009720325
- > mean(abs(residuals)) # MAD
- [1] 0.07863206
- > cor(predictions, label)^2# r^2
- [,1]
- s0 0.9998749
+ # Use the following R code to generate model training results.
+
+ # path/part-00000 is the file generated by running
LinearDataGenerator.generateLinearInput
+ # as described before the beforeAll() method.
+ d1 <- read.csv("path/part-00000", header=FALSE,
stringsAsFactors=FALSE)
+ fit <- glm(V1 ~ V2 + V3, data = d1, family = "gaussian")
+ names(f1)[1] = c("V2")
+ names(f1)[2] = c("V3")
+ f1 <- data.frame(as.numeric(d1$V2), as.numeric(d1$V3))
+ predictions <- predict(fit, newdata=f1)
+ l1 <- as.numeric(d1$V1)
+
+ residuals <- l1 - predictions
+ > mean(residuals^2) # MSE
+ [1] 0.00985449
+ > mean(abs(residuals)) # MAD
+ [1] 0.07961668
+ > cor(predictions, l1)^2 # r^2
+ [1] 0.9998737
+
+ > summary(fit)
+
+ Call:
+ glm(formula = V1 ~ V2 + V3, family = "gaussian", data = d1)
+
+ Deviance Residuals:
+ Min 1Q Median 3Q Max
+ -0.47082 -0.06797 0.00002 0.06725 0.34635
+
+ Coefficients:
+ Estimate Std. Error t value Pr(>|t|)
+ (Intercept) 6.3022157 0.0018600 3388 <2e-16 ***
+ V2 4.6982442 0.0011805 3980 <2e-16 ***
+ V3 7.1994344 0.0009044 7961 <2e-16 ***
+ ---
+
+ ....
*/
- assert(model.summary.meanSquaredError ~== 0.00972035 relTol 1E-5)
- assert(model.summary.meanAbsoluteError ~== 0.07863206 relTol 1E-5)
- assert(model.summary.r2 ~== 0.9998749 relTol 1E-5)
+ assert(model.summary.meanSquaredError ~== 0.00985449 relTol 1E-5)
+ assert(model.summary.meanAbsoluteError ~== 0.07961668 relTol 1E-5)
+ assert(model.summary.r2 ~== 0.9998737 relTol 1E-5)
--- End diff --
Yes, I get all that. I'm not suggesting trying a bunch of seeds though any
data so generated should produce the same answer within some tolerance. Same
goes for your new generation process. The fact that the test then fails means
your data generation process is wrong or the test is. So, something has to be
done right?
You did, but your change suggests that the 'expected value' of the data
changed. It is not clear we should believe that. Hence fix the threshold and
yes 10x isn't any more principled but has the advantage of being not incorrect
in that it is too loose if anything.
Really the current change is only very slightly suboptimal and just pushes
the tiny problem to a future change. Maybe it is worth punting on, even though
making the test righter here seems easy.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
