alexeykudinkin commented on code in PR #7871:
URL: https://github.com/apache/hudi/pull/7871#discussion_r1105004523
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala:
##########
@@ -554,4 +657,16 @@ object MergeIntoHoodieTableCommand {
}
}
+ def stripCasting(expr: EqualTo): EqualTo = expr match {
+ case EqualTo(MatchCast(leftExpr, leftTargetType, _, _),
MatchCast(rightExpr, rightTargetType, _, _))
+ if leftTargetType.sameType(rightTargetType) => EqualTo(leftExpr,
rightExpr)
+ case _ => expr
+ }
+
+ def toStructType(attrs: Seq[Attribute]): StructType =
+ StructType(attrs.map(a => StructField(a.qualifiedName.replace('.', '_'),
a.dataType, a.nullable, a.metadata)))
Review Comment:
Here we combine the output of joining operation back into schema so that we
can assert that the records we receive in `ExpressionPayload` still adhere to
the expected schema. Here we need to replace dots, b/c
- Names could be qualified (in case of nested structs; note that this
schema is essentially a flattened one)
- Field names can't have dots
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
