[
https://issues.apache.org/jira/browse/BAHIR-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16299315#comment-16299315
]
ASF subversion and git services commented on BAHIR-138:
-------------------------------------------------------
Commit eae02f29eb011f50bc313714e6cde62ce65804c4 in bahir's branch
refs/heads/master from [~emlaver]
[ https://git-wip-us.apache.org/repos/asf?p=bahir.git;h=eae02f2 ]
[BAHIR-138] fix deprecated warnings in sql-cloudant
Fix warnings in DefaultSource class, and in CloudantStreaming
and CloudantStreamingSelector examples.
How
Imported spark.implicits._ to convert Spark RDD to Dataset
Replaced deprecated json(RDD[String]) with json(Dataset[String])
Improved streaming examples:
Replaced registerTempTable with preferred createOrReplaceTempView
Replaced !isEmpty with nonEmpty
Use an accessible sales database so users can run the example without any setup
Fixed error message when stopping tests by adding logic to streaming
receiver to not store documents in Spark memory when stream has stopped
Closes #59
> Fix sql-cloudant deprecation messages
> -------------------------------------
>
> Key: BAHIR-138
> URL: https://issues.apache.org/jira/browse/BAHIR-138
> Project: Bahir
> Issue Type: Task
> Components: Spark Structured Streaming Connectors
> Affects Versions: Spark-2.2.0
> Reporter: Esteban Laver
> Assignee: Esteban Laver
> Priority: Minor
> Labels: warnings
>
> Deprecation warnings in {{DefaultSource}}:
> {code}
> [INFO] --- scala-maven-plugin:3.2.2:compile (scala-compile-first) @
> spark-sql-cloudant_2.11 ---
> [INFO] Compiling 11 Scala sources to sql-cloudant/target/scala-2.11/classes...
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:59:
> method json in class DataFrameReader is deprecated: Use json(Dataset[String])
> instead.
> [WARNING] val df = sqlContext.read.json(cloudantRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:115:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] dataFrame = sqlContext.read.json(cloudantRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:121:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] sqlContext.read.json(aRDD)
> [WARNING] ^
> [WARNING]
> sql-cloudant/src/main/scala/org/apache/bahir/cloudant/DefaultSource.scala:152:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] dataFrame =
> sqlContext.sparkSession.read.json(globalRDD)
> [WARNING] ^
> [WARNING] four warnings found
> {code}
> Deprecation warnings in {{CloudantStreaming}} and
> {{CloudantStreamingSelector}} examples:
> {code}
> [INFO] --- scala-maven-plugin:3.2.2:testCompile (scala-test-compile-first) @
> spark-sql-cloudant_2.11 ---
> [INFO] Compiling 11 Scala sources to
> sql-cloudant/target/scala-2.11/test-classes...
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreaming.scala:46:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] val changesDataFrame = spark.read.json(rdd)
> [WARNING] ^
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreaming.scala:67:
> method registerTempTable in class Dataset is deprecated: Use
> createOrReplaceTempView(viewName) instead.
> [WARNING] changesDataFrame.registerTempTable("airportcodemapping")
> [WARNING] ^
> [WARNING]
> sql-cloudant/examples/src/main/scala/org/apache/spark/examples/sql/cloudant/CloudantStreamingSelector.scala:50:
> method json in class DataFrameReader is deprecated: Use
> json(Dataset[String]) instead.
> [WARNING] val changesDataFrame = spark.read.json(rdd)
> [WARNING] ^
> [WARNING] three warnings found
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
