Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21732#discussion_r235062757
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoder.scala
---
@@ -198,7 +189,7 @@ case class ExpressionEncoder[T](
val serializer: Seq[NamedExpression] = {
val clsName = Utils.getSimpleName(clsTag.runtimeClass)
- if (isSerializedAsStruct) {
+ if (isSerializedAsStruct && !isOptionType) {
--- End diff --
There is a `isSerializedAsStruct` check in `TypedAggregateExpression` which
doesn't come with `isOptionType`.
I'd think `isSerializedAsStruct` is true or false only depending how we
serialize object `T` and isn't related to how we use (flatten or not) it in
`ExpressionEncoder`. At least it is more clear to me.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
