fqaiser94 commented on a change in pull request #27066:
URL: https://github.com/apache/spark/pull/27066#discussion_r426113568
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala
##########
@@ -879,6 +879,17 @@ class Column(val expr: Expression) extends Logging {
*/
def getItem(key: Any): Column = withExpr { UnresolvedExtractValue(expr,
Literal(key)) }
+ /**
+ * An expression that adds/replaces a field by name in a `StructType`.
+ * If schema contains multiple fields with fieldName, they will all be
replaced with fieldValue.
+ *
+ * @group expr_ops
+ * @since 3.1.0
+ */
+ def withField(fieldName: String, fieldValue: Column): Column = withExpr {
Review comment:
@viirya Originally I wanted to mirror the `withColumn` API design and if
users wanted to add multiple fields they could chain multiple `withField` calls
themselves:
```
'a.withField("b", lit(3)).withField("c", lit(4))
```
The limitation of this design is that you can never add multiple fields with
the same name (whereas with the SQL syntax you can).
An alternative approach I'm now considering is to follow the
`org.apache.spark.sql.functions.struct` API design so that you could do
something like this to add multiple fields at a time:
```
'a.withFields(lit(3).as("b"), lit(4).as("c"))
```
This would make things more consistent between the Scala and SQL APIs. What
do you think of this idea?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
