[
https://issues.apache.org/jira/browse/HUDI-8942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo closed HUDI-8942.
-----------------------------
Resolution: Fixed
> Upgrade fails from 0.15 when OverwriteWithLatestPayload and
> DEFAULT_MERGER_STRATEGY_UUID set in table config
> ------------------------------------------------------------------------------------------------------------
>
> Key: HUDI-8942
> URL: https://issues.apache.org/jira/browse/HUDI-8942
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Sagar Sumit
> Assignee: Y Ethan Guo
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.1
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
> # Create a table with 0.15.0 with following configs (precombine needs to be
> set):
> Map(
> hoodie.clustering.inline.max.commits -> 6, hoodie.write.auto.upgrade -> true,
> hoodie.datasource.write.precombine.field -> ts, hoodie.metadata.enable ->
> false, hoodie.datasource.write.operation -> upsert,
> hoodie.parquet.compression.codec -> snappy,
> hoodie.datasource.write.recordkey.field -> uuid, hoodie.table.name ->
> table_comp_test_0_15_0_1_1_0-SNAPSHOT_1738255863,
> hoodie.datasource.write.table.type -> MERGE_ON_READ,
> hoodie.datasource.write.hive_style_partitioning -> true,
> hoodie.clustering.inline -> false, hoodie.compact.inline -> true,
> hoodie.keep.max.commits -> 15, hoodie.cleaner.commits.retained -> 8,
> hoodie.write.lock.provider ->
> org.apache.hudi.client.transaction.lock.InProcessLockProvider,
> hoodie.keep.min.commits -> 10, hoodie.datasource.write.partitionpath.field ->
> partitionpath, hoodie.compact.inline.max.delta.commits -> 1)
> 2. Load some data
> 3. Now load another batch with master build (1.1.0-SNAPSHOT). It fails with
> below exception:
> {code:java}
> java.lang.IllegalArgumentException: Configured payload class
> (org.apache.hudi.common.model.OverwriteWithLatestAvroPayload) and record
> merge strategy ID (eeb8d96f-b1e4-49fd-bbf8-28ac514178e5) conflict with each
> other. Please only set one of them in the write config.
> at
> org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:42)
> at
> org.apache.hudi.common.table.HoodieTableConfig.inferCorrectMergingBehavior(HoodieTableConfig.java:785)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.mergeParamsAndGetHoodieConfig(HoodieSparkSqlWriter.scala:1136)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.writeInternal(HoodieSparkSqlWriter.scala:253)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.$anonfun$write$1(HoodieSparkSqlWriter.scala:193)
> at
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$6(SQLExecution.scala:109)
> at
> org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:169)
> at
> org.apache.spark.sql.execution.SQLExecution$.$anonfun$withNewExecutionId$1(SQLExecution.scala:95)
> at org.apache.spark.sql.SparkSession.withActive(SparkSession.scala:779)
> at
> org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:64)
> at
> org.apache.spark.sql.adapter.BaseSpark3Adapter.sqlExecutionWithNewExecutionId(BaseSpark3Adapter.scala:105)
> at
> org.apache.hudi.HoodieSparkSqlWriterInternal.write(HoodieSparkSqlWriter.scala:215)
> at
> org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:130)
> at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:170)
> at
> org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:47)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
