cloud-fan commented on a change in pull request #25291: [SPARK-28554][SQL]
implement basic catalog functionalities for JDBC v2 with a DS v1 fallback API
URL: https://github.com/apache/spark/pull/25291#discussion_r308275971
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala
##########
@@ -253,6 +260,16 @@ class FindDataSourceTable(sparkSession: SparkSession)
extends Rule[LogicalPlan]
case UnresolvedCatalogRelation(tableMeta) =>
DDLUtils.readHiveTable(tableMeta)
+
+ case AppendData(DataSourceV2Relation(t: DataSourceV1Table, output, _),
query, false) =>
+ InsertIntoDataSourceCommand(fallbackToV1Relation(t, output), query,
overwrite = false)
+
+ case OverwriteByExpression(
+ DataSourceV2Relation(t: DataSourceV1Table, output, _), Literal(true,
_), query, false) =>
Review comment:
The Data Source V1 API don't support static partition overwrite (e.g.
`INSERT OVERWRITE t PARTITION(a=1) SELECT ...`). We must fully migrate JDBC to
Data Source V2 before supporting it.
Here we only need to match the OverwriteByExpression(...,
deleteExpr=Literal(true)), as this is the only thing DS v1 supports.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
