[
https://issues.apache.org/jira/browse/SPARK-11284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16791671#comment-16791671
]
Alex Combessie edited comment on SPARK-11284 at 3/13/19 1:00 PM:
-
Hello everyone, [~ddahlem], [~mengxr]
I am still getting a Type error error when evaluating an ALS model in a
pipeline. I have tested it on Spark 2.2.0.2.6.4.0-91. It is strange as it seems
the issue is closed.
Here is the error message:
_Caused by: java.lang.ClassCastException: java.lang.Float cannot be cast to
java.lang.Double_
Happy to provide more details. I am running the classic MovieLens example on a
100k dataset. Any views on this?
Thanks,
Alex
was (Author: alex_combessie):
Hello everyone,
I am still getting a Type error error when evaluating an ALS model in a
pipeline. I have tested it on Spark 2.2.0.2.6.4.0-91. It is strange as it seems
the issue is closed.
Here is the error message:
_Caused by: java.lang.ClassCastException: java.lang.Float cannot be cast to
java.lang.Double_
Happy to provide more details. I am running the classic MovieLens example on a
100k dataset. Any views on this?
Thanks,
Alex
> ALS produces predictions as floats and should be double
> ---
>
> Key: SPARK-11284
> URL: https://issues.apache.org/jira/browse/SPARK-11284
> Project: Spark
> Issue Type: Bug
> Components: ML
>Affects Versions: 1.5.1
> Environment: All
>Reporter: Dominik Dahlem
>Priority: Major
> Labels: ml, recommender
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Using pyspark.ml and DataFrames, The ALS recommender cannot be evaluated
> using the RegressionEvaluator, because of a type mis-match between the model
> transformation and the evaluation APIs. One can work around this by casting
> the prediction column into double before passing it into the evaluator.
> However, this does not work with pipelines and cross validation.
> Code and traceback below:
> {code}
> als = ALS(rank=10, maxIter=30, regParam=0.1, userCol='userID',
> itemCol='movieID', ratingCol='rating')
> model = als.fit(training)
> predictions = model.transform(validation)
> evaluator = RegressionEvaluator(predictionCol='prediction',
> labelCol='rating')
> validationRmse = evaluator.evaluate(predictions,
> {evaluator.metricName: 'rmse'})
> {code}
> Traceback:
> validationRmse = evaluator.evaluate(predictions, {evaluator.metricName:
> 'rmse'})
> File
> "/Users/dominikdahlem/software/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/ml/evaluation.py",
> line 63, in evaluate
> File
> "/Users/dominikdahlem/software/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/pyspark.zip/pyspark/ml/evaluation.py",
> line 94, in _evaluate
> File
> "/Users/dominikdahlem/software/spark-1.6.0-SNAPSHOT-bin-custom-spark/python/lib/py4j-0.9-src.zip/py4j/java_gateway.py",
> line 813, in __call__
> File
> "/Users/dominikdahlem/projects/repositories/spark/python/pyspark/sql/utils.py",
> line 42, in deco
> raise IllegalArgumentException(s.split(': ', 1)[1])
> pyspark.sql.utils.IllegalArgumentException: requirement failed: Column
> prediction must be of type DoubleType but was actually FloatType.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
