xianyinxin commented on a change in pull request #25626: [SPARK-28892][SQL] Add
UPDATE support for DataSource V2
URL: https://github.com/apache/spark/pull/25626#discussion_r320588816
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala
##########
@@ -245,6 +246,56 @@ object DataSourceV2Strategy extends Strategy with
PredicateHelper {
}.toArray
DeleteFromTableExec(r.table.asDeletable, filters) :: Nil
+ case UpdateTable(maybeRelation, attrs, values, condition) =>
+ val nested =
attrs.asInstanceOf[Seq[Any]].filterNot(_.isInstanceOf[AttributeReference])
+ if (nested.nonEmpty) {
+ throw new AnalysisException(s"Update only support non-nested fields.
Nested: $nested")
+ }
+
+ val (relation, attrsNames, newValues, newCond) = maybeRelation match {
+ case d: DataSourceV2Relation =>
+ (d, attrs.map(_.name), values, condition)
+ case Project(aliasList: Seq[Alias], r: DataSourceV2Relation) =>
Review comment:
I've checked
> @xianyinxin, can you explain the required semantics for your proposed API?
>
> ```
> void updateWhere(Map<String, Expression> sets, Filter[] filters);
> ```
>
> It isn't clear what the requirement for a source would be.
>
> In addition, `Expression` is internal to catalyst and should be removed
from the API.
Thank you @rdblue . Here `Expression` is the public datasource expression
`org.apache.spark.sql.catalog.v2.expressions.Expression`. Datasource needs to
know the value that the field be updated to, so here the key of `sets`
specifies the field to be updated, and the value of `sets` is the updated
value. The value can be an expression, not just a literal, IMHO, like the case
`UPDATE tbl SET a=a+1 WHERE ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
