[
https://issues.apache.org/jira/browse/SYSTEMML-1280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Deron Eriksson updated SYSTEMML-1280:
-------------------------------------
Description:
SYSTEMML-1194 replaced SQLContext with SparkSession in SystemML for Spark 2.1.0
since SQLContext is deprecated.
Restore the old Java SQLContext method signatures in case any users are using
SystemML methods and are unable to use SparkSessions (SparkSessions are
generally easy to create, as described in
https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html)
Classes where this applies:
old MLContext class (whole class is deprecated)
old MLMatrix class (whole class is deprecated)
old MLOutput class (whole class is deprecated)
FrameRDDConverterUtils (this is a non-API class)
RDDConverterUtils (this is a non-API class)
RDDConverterUtilsExt (this is a non-API class)
In non-API classes, these SQLContext methods should be marked as deprecated and
removed in a future version of SystemML (1.0) since SparkSessions should
generally be used with Spark 2. As mentioned in SQLContext documentation, "As
of Spark 2.0, this is replaced by SparkSession. However, we are keeping the
class here for backward compatibility."
was:
SYSTEMML-1194 replaced SQLContext with SparkSession in SystemML for Spark 2.1.0
since SQLContext is deprecated.
Restore the old Java SQLContext method signatures in case any users are using
SystemML methods and are unable to use SparkSessions (SparkSessions are
generally easy to create, as described in
https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html)
Classes where this applies:
old MLContext class (whole class is deprecated)
old MLMatrix class (whole class is deprecated)
old MLOutput class (whole class is deprecated)
FrameRDDConverterUtils (this is a non-API class)
RDDConverterUtils (this is a non-API class)
RDDConverterUtilsExt (this is a non-API class)
In non-API classes, these SQLContext methods should be marked as deprecated and
removed in a future version of SystemML (1.0) since SparkSessions should be
used with Spark 2. See
https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html
> Restore and deprecate SQLContext methods
> ----------------------------------------
>
> Key: SYSTEMML-1280
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1280
> Project: SystemML
> Issue Type: Task
> Components: APIs, Runtime
> Reporter: Deron Eriksson
> Assignee: Deron Eriksson
>
> SYSTEMML-1194 replaced SQLContext with SparkSession in SystemML for Spark
> 2.1.0 since SQLContext is deprecated.
> Restore the old Java SQLContext method signatures in case any users are using
> SystemML methods and are unable to use SparkSessions (SparkSessions are
> generally easy to create, as described in
> https://databricks.com/blog/2016/08/15/how-to-use-sparksession-in-apache-spark-2-0.html)
> Classes where this applies:
> old MLContext class (whole class is deprecated)
> old MLMatrix class (whole class is deprecated)
> old MLOutput class (whole class is deprecated)
> FrameRDDConverterUtils (this is a non-API class)
> RDDConverterUtils (this is a non-API class)
> RDDConverterUtilsExt (this is a non-API class)
> In non-API classes, these SQLContext methods should be marked as deprecated
> and removed in a future version of SystemML (1.0) since SparkSessions should
> generally be used with Spark 2. As mentioned in SQLContext documentation, "As
> of Spark 2.0, this is replaced by SparkSession. However, we are keeping the
> class here for backward compatibility."
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
