Joseph K. Bradley created SPARK-7432:
----------------------------------------
Summary: Flaky test in PySpark CrossValidator doc test
Key: SPARK-7432
URL: https://issues.apache.org/jira/browse/SPARK-7432
Project: Spark
Issue Type: Bug
Components: ML, PySpark
Affects Versions: 1.4.0
Reporter: Joseph K. Bradley
Priority: Critical
There was a test failure in the doc test in Python CrossValidator:
[https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/32058/consoleFull]
Here's the full doc test:
{code}
>>> from pyspark.ml.classification import LogisticRegression
>>> from pyspark.ml.evaluation import BinaryClassificationEvaluator
>>> from pyspark.mllib.linalg import Vectors
>>> dataset = sqlContext.createDataFrame(
... [(Vectors.dense([0.0, 1.0]), 0.0),
... (Vectors.dense([1.0, 2.0]), 1.0),
... (Vectors.dense([0.55, 3.0]), 0.0),
... (Vectors.dense([0.45, 4.0]), 1.0),
... (Vectors.dense([0.51, 5.0]), 1.0)] * 10,
... ["features", "label"])
>>> lr = LogisticRegression()
>>> grid = ParamGridBuilder().addGrid(lr.maxIter, [0, 1, 5]).build()
>>> evaluator = BinaryClassificationEvaluator()
>>> cv = CrossValidator(estimator=lr, estimatorParamMaps=grid,
evaluator=evaluator)
>>> cvModel = cv.fit(dataset)
>>> expected = lr.fit(dataset, {lr.maxIter: 5}).transform(dataset)
>>> cvModel.transform(dataset).collect() == expected.collect()
True
{code}
Here's the failure message:
{code}
Running test: pyspark/ml/tuning.py ...
**********************************************************************
File "pyspark/ml/tuning.py", line 108, in __main__.CrossValidator
Failed example:
cvModel.transform(dataset).collect() == expected.collect()
Expected:
True
Got:
False
**********************************************************************
1 of 11 in __main__.CrossValidator
***Test Failed*** 1 failures.
Had test failures; see logs.
[error] Got a return code of 255 on line 240 of the run-tests script.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
