kazdy commented on code in PR #9277:
URL: https://github.com/apache/hudi/pull/9277#discussion_r1276494722
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/UpdateHoodieTableCommand.scala:
##########
@@ -31,6 +34,22 @@ import org.apache.spark.sql.hudi.ProvidesHoodieConfig
case class UpdateHoodieTableCommand(ut: UpdateTable) extends
HoodieLeafRunnableCommand
with SparkAdapterSupport with ProvidesHoodieConfig {
+ private def buildUpdateConfig( sparkSession: SparkSession,
+ hoodieCatalogTable: HoodieCatalogTable):
Map[String, String] = {
+ val optimizedWrite =
sparkSession.sqlContext.conf.getConfString(SPARK_SQL_OPTIMIZED_WRITES.key()
+ , SPARK_SQL_OPTIMIZED_WRITES.defaultValue()) == "true"
+ val shouldCombine =
!StringUtils.isNullOrEmpty(hoodieCatalogTable.preCombineKey.getOrElse(""))
+
+ if (!shouldCombine &&
hoodieCatalogTable.tableType.name().equals("MERGE_ON_READ")) {
+ throw new HoodieException("Precombine field must be set for
MERGE_ON_READ table type to execute UPDATE statement.")
Review Comment:
@danny0405 Yes, this is intentional as for MoR it is a requirement to have
precombine field (but it's not enforced yet atm), see:
https://github.com/apache/hudi/pull/9031
In theory, once #9091 is merged this exception should never be thrown for
new tables, but for old tables it's better to guard against this.
I decided to throw this exception here since different record impl throw
different exceptions for MoR when there's no precombine field - "Failed to
upsert for commit" or "(Part -) not found, acceptable fields were: ..." it was
hard to test for this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
