[
https://issues.apache.org/jira/browse/SPARK-16348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15359896#comment-15359896
]
Apache Spark commented on SPARK-16348:
--------------------------------------
User 'jkbradley' has created a pull request for this issue:
https://github.com/apache/spark/pull/14023
> pyspark.ml MLSerDe should be called using full classpath
> --------------------------------------------------------
>
> Key: SPARK-16348
> URL: https://issues.apache.org/jira/browse/SPARK-16348
> Project: Spark
> Issue Type: Bug
> Components: ML, PySpark
> Affects Versions: 2.0.0
> Reporter: Joseph K. Bradley
> Assignee: Joseph K. Bradley
> Priority: Critical
>
> Depending on how Spark is set up, pyspark.ml may or may not be able to find
> the MLSerDe instance when referenced as {{sc._jvm.MLSerDe}}. This can cause
> failures {{'JavaPackage' object is not callable}} when trying to access
> Vector or Matrix values from pyspark, such as retrieving the coefficients of
> a LinearRegressionModel.
> Proposal: Whenever we reference a class in the _jvm from pyspark, we should
> use the full classpath: {{sc._jvm.org.apache.spark.ml.python.MLSerDe}}. This
> fixes the bug in my case.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
