dongkelun commented on a change in pull request #3517:
URL: https://github.com/apache/hudi/pull/3517#discussion_r713041105
##########
File path:
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala
##########
@@ -196,9 +196,11 @@ case class MergeIntoHoodieTableCommand(mergeInto:
MergeIntoTable) extends Runnab
}
private def isEqualToTarget(targetColumnName: String, sourceExpression:
Expression): Boolean = {
+ val sourceColNameMap = sourceDFOutput.map(attr => (attr.name.toLowerCase,
attr.name)).toMap
+
sourceExpression match {
- case attr: AttributeReference if
attr.name.equalsIgnoreCase(targetColumnName) => true
- case Cast(attr: AttributeReference, _, _) if
attr.name.equalsIgnoreCase(targetColumnName) => true
+ case attr: AttributeReference if
sourceColNameMap(attr.name.toLowerCase).equals(targetColumnName) => true
Review comment:
Hi,is it like this?
```scala
val resolver = sparkSession.sessionState.conf.resolver
case attr: AttributeReference if resolver(attr.name, targetColumnName) =>
true
```
I'm not sure if I understand,resolver is not case sensitive when comparing
equality.However, the comparison of equality here must be case
sensitive.Therefore, use sourceColNameMap(attr.name.toLowerCase) to obtain the
original column name of source table without case conversion,Then compare with
targetcolumnname for equality.If not, add the corresponding column name with
withcolumn later. It is case sensitive because sourceDF is case sensitive when
writing data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
