Github user fuqiliang commented on the issue:
https://github.com/apache/spark/pull/20666
Hi, Thanks for help.
And do we have follow-up stories on the data loss in spark2.2?
I have tried to use
`sql.read.format("my.spark.sql.execution.datasources.json").load(file_path)` by
shading the spark-sql_2.10:1.6.jar to my.spark.sql-shade.jar, But found the
json.DefaultSource from spark1.6 can not be used in spark2.2 with error:
> Exception in thread "main" org.apache.spark.sql.AnalysisException:
my.spark.sql.execution.datasources.json is not a valid Spark SQL Data Source.;
> at
org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:376)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:156)
> at com.test.shade.SparkTest$.main(SparkTest.scala:32)
> at com.test.shade.SparkTest.main(SparkTest.scala)
So can I have some other suggestions ? @HyukjinKwon
Thanks a lot.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
