[
https://issues.apache.org/jira/browse/HUDI-2234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17443274#comment-17443274
]
Yann Byron edited comment on HUDI-2234 at 11/14/21, 9:21 AM:
-------------------------------------------------------------
[~xushiyan] [~codope] [~shivnarayan]
HUDI-2706 has cancelled the assertion as above. If source table has the same
fields to the record fields and precombine field of the target table, that can
merge on the non-pk fields.
was (Author: [email protected]):
#HUDI-2706 has cancelled the assertion as above. If source table has the same
fields to the record fields and precombine field of the target table, that can
merge on the non-pk fields.
> MERGE INTO works only ON primary key
> ------------------------------------
>
> Key: HUDI-2234
> URL: https://issues.apache.org/jira/browse/HUDI-2234
> Project: Apache Hudi
> Issue Type: Sub-task
> Components: Spark Integration
> Reporter: Sagar Sumit
> Assignee: Yann Byron
> Priority: Blocker
> Fix For: 0.10.0
>
>
> {code:sql}
> drop table if exists hudi_gh_ext_fixed;
> create table hudi_gh_ext_fixed (id int, name string, price double, ts long)
> using hudi options(primaryKey = 'id', precombineField = 'ts') location
> 'file:///tmp/hudi-h4-fixed';
> insert into hudi_gh_ext_fixed values(3, 'AMZN', 300, 120);
> insert into hudi_gh_ext_fixed values(2, 'UBER', 300, 120);
> insert into hudi_gh_ext_fixed values(4, 'GOOG', 300, 120);
> update hudi_gh_ext_fixed set price = 150.0 where name = 'UBER';
> drop table if exists hudi_fixed;
> create table hudi_fixed (id int, name string, price double, ts long) using
> hudi options(primaryKey = 'id', precombineField = 'ts') partitioned by (ts)
> location 'file:///tmp/hudi-h4-part-fixed';
> insert into hudi_fixed values(2, 'UBER', 200, 120);
> MERGE INTO hudi_fixed
> USING (select id, name, price, ts from hudi_gh_ext_fixed) updates
> ON hudi_fixed.name = updates.name
> WHEN MATCHED THEN
> UPDATE SET *
> WHEN NOT MATCHED
> THEN INSERT *;
> -- java.lang.IllegalArgumentException: Merge Key[name] is not Equal to the
> defined primary key[id] in table hudi_fixed
> -- at
> org.apache.spark.sql.hudi.command.MergeIntoHoodieTableCommand.buildMergeIntoConfig(MergeIntoHoodieTableCommand.scala:425)
> -- at
> org.apache.spark.sql.hudi.command.MergeIntoHoodieTableCommand.run(MergeIntoHoodieTableCommand.scala:146)
> -- at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)
> -- at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)
> -- at
> org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:79)
> -- at
> org.apache.spark.sql.Dataset.$anonfun$logicalPlan$1(Dataset.scala:229)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
