dbtsai edited a comment on issue #27066: [SPARK-31317][SQL] Add withField method to Column class URL: https://github.com/apache/spark/pull/27066#issuecomment-609567218 @fqaiser94 for some cases, the expressions are not resolved when `Column.withField` is called. We should be able to workaround this by ```scala case class WithField(fieldName: String, field: Expression, fieldValue: Expression) extends BinaryExpression { override def doGenCode(ctx: CodegenContext, ev: ExprCode): ExprCode = { throw new Exception("This will never hit!") } override def left: Expression = field override def right: Expression = fieldValue override lazy val dataType: DataType = ... } ``` And then create a spark catalyst rule that transforms `WithField` expression to `CreateNamedStruct` once all the attributes are resolved. The optimizer rule should be similar to the above `withField` implementation.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
