jonvex commented on code in PR #9083:
URL: https://github.com/apache/hudi/pull/9083#discussion_r1249593464
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala:
##########
@@ -125,18 +127,68 @@ case class MergeIntoHoodieTableCommand(mergeInto:
MergeIntoTable) extends Hoodie
* expression involving [[source]] column(s), we will have to add "phony"
column matching the
* primary-key one of the target table.
*/
- private lazy val primaryKeyAttributeToConditionExpression: Seq[(Attribute,
Expression)] = {
+ private lazy val recordKeyAttributeToConditionExpression: Seq[(Attribute,
Expression)] = {
+ val primaryKeyFields = hoodieCatalogTable.tableConfig.getRecordKeyFields
val conditions = splitConjunctivePredicates(mergeInto.mergeCondition)
- if (!conditions.forall(p => p.isInstanceOf[EqualTo])) {
- throw new AnalysisException(s"Currently only equality predicates are
supported in MERGE INTO statement " +
- s"(provided ${mergeInto.mergeCondition.sql}")
+ if (primaryKeyFields.isPresent) {
+ //pkless tables can have more complex conditions
+ if (!conditions.forall(p => p.isInstanceOf[EqualTo])) {
Review Comment:
What this does is prevent conditions besides equality for pk tables. We only
want s.id = t.id and possibly s.partition = t.partition. Conditions includes
all the merge conditions. We are only filtering for non equality conditions at
this point. Later on in this method we make sure it's only primary key and
partition path for the conditions
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
