cloud-fan commented on code in PR #47233:
URL: https://github.com/apache/spark/pull/47233#discussion_r1765133675
##########
sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala:
##########
@@ -1654,6 +1655,43 @@ class Dataset[T] private[sql](
new MergeIntoWriterImpl[T](table, this, condition)
}
+ /**
+ * Update rows in a table.
+ *
+ * Scala Example:
+ * {{{
+ * spark.table("source")
+ * .update(Map("salary" -> lit(200)))
+ * .execute()
+ * }}}
+ * @param assignments A Map of column names to Column expressions
representing the updates
+ * to be applied.
+ * @group basic
+ * @since 4.0.0
+ */
+ def update(assignments: Map[String, Column]): Unit = {
+ updateInternal(assignments)
+ }
+
+ /**
+ * Update rows in a table that match a condition.
+ *
+ * Scala Example:
+ * {{{
+ * spark.table("source")
+ * .update(Map("salary" -> lit(200)), $"salary" === 100)
+ * .execute()
+ * }}}
+ * @param assignments A Map of column names to Column expressions
representing the updates
+ * to be applied.
+ * @param condition the update condition
+ * @group basic
+ * @since 4.0.0
+ */
+ def update(assignments: Map[String, Column], condition: Column): Unit = {
+ updateInternal(assignments, Some(condition))
+ }
Review Comment:
`spark.table("simple").write.update(mapping, cond)` This looks really weird
to me. Normally the DataFrame before `.write` is the input data to write out,
but now it becomes the target to write data into. I believe this is
counterintuitive to many Spark users.
What's your concern for `spark.catalog.getTable(...).update(...)`? It
basically uses a dedicated `Table` API instead of DataFrame to host the update
API. And we already have other DDL APIs there, such as
`spark.catalog.createTable`, `spark.catalog.listTables`, etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
