[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/16652


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[imatiach-msft]

Github user imatiach-msft commented on a diff in the pull request:

https://github.com/apache/spark/pull/16652#discussion_r98516538
  
--- Diff: 
mllib/src/test/scala/org/apache/spark/ml/regression/AFTSurvivalRegressionSuite.scala
 ---
@@ -400,6 +400,17 @@ class AFTSurvivalRegressionSuite
 val trainer = new AFTSurvivalRegression()
 trainer.fit(dataset)
   }
+
+  test("SPARK-19234: Fail fast on zero-valued labels") {
+val dataset = spark.createDataFrame(Seq(
+ (1.218, 1.0, Vectors.dense(1.560, -0.605)),
+ (0.000, 0.0, Vectors.dense(0.346, 2.158)), // ← generates 
error; zero labels invalid
+ (4.199, 0.0, Vectors.dense(0.795, -0.226.toDF("label", 
"censor", "features")
+val aft = new AFTSurvivalRegression()
+intercept[SparkException] {
--- End diff --

it's recommended to verify the error message using withClue, eg:
withClue("label of AFTPoint must be positive") {
 intercept[SparkException] { 
aft.fit(dataset)
 }
}


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/16652#discussion_r98435106
  
--- Diff: 
mllib/src/test/scala/org/apache/spark/ml/regression/AFTSurvivalRegressionSuite.scala
 ---
@@ -18,20 +18,26 @@
 package org.apache.spark.ml.regression
 
 import scala.util.Random
-
--- End diff --

This is still causing a style check failure @admackin 


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/16652#discussion_r97056877
  
--- Diff: 
mllib/src/test/scala/org/apache/spark/ml/regression/AFTSurvivalRegressionSuite.scala
 ---
@@ -18,20 +18,26 @@
 package org.apache.spark.ml.regression
 
 import scala.util.Random
-
--- End diff --

(Leave the blank)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/16652#discussion_r97057073
  
--- Diff: 
mllib/src/test/scala/org/apache/spark/ml/regression/AFTSurvivalRegressionSuite.scala
 ---
@@ -415,4 +421,40 @@ object AFTSurvivalRegressionSuite {
 "maxIter" -> 2,
 "tol" -> 0.01
   )
+
+  private[AFTSurvivalRegressionSuite] def checkNumericTypes[M <: Model[M], 
T <: Estimator[M]](
--- End diff --

This is being copied in from MLUtils? why is it necessary?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #16652: [SPARK-19234][MLLib] AFTSurvivalRegression should...

[admackin]

GitHub user admackin opened a pull request:

https://github.com/apache/spark/pull/16652

[SPARK-19234][MLLib] AFTSurvivalRegression should fail fast when any labels 
are zero

## What changes were proposed in this pull request?

If any labels of 0.0 (which are invalid) are supplied, 
AFTSurvivalRegression gives an error straight away rather than 
hard-to-interpret warnings and zero-valued coefficients in the output.

## How was this patch tested?

Verified against current test suite. (One test needed to be updated as it 
was providing values of zero for labels so was failing after this patch)

Please review http://spark.apache.org/contributing.html before opening a 
pull request.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/admackin/spark master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/16652.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #16652


commit ab6d4148c4aa721898733b14eec5068652ca1085
Author: Andy MacKinlay 
Date:   2017-01-20T01:56:45Z

Addresses SPARK-19234 - make sure label is positive

commit b07c281c378d68d86b81498ca247c7346719973e
Author: Andy MacKinlay 
Date:   2017-01-20T04:02:54Z

Addresses SPARK-19234 - fix test suite to ensure no zero-labels get passed 
in test cases as they now throw errors




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org