Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r223988838
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Relation.scala
---
@@ -169,15 +174,16 @@ object DataSourceV2Relation {
options: Map[String, String],
tableIdent: Option[TableIdentifier] = None,
userSpecifiedSchema: Option[StructType] = None):
DataSourceV2Relation = {
- val reader = source.createReader(options, userSpecifiedSchema)
+ val readSupport = source.createReadSupport(options,
userSpecifiedSchema)
--- End diff --
This looks a regression comparing 2.3 - Data Source V2 is under heavy
development so I understand but this is quite crucial. From a cursory look,
this is introduced in
https://github.com/apache/spark/commit/5fef6e3513d6023a837c427d183006d153c7102b
I would suggest to partially revert this commit.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
