yihua commented on code in PR #11943:
URL: https://github.com/apache/hudi/pull/11943#discussion_r1809700398
##########
hudi-spark-datasource/hudi-spark/src/main/scala/org/apache/spark/sql/hudi/command/MergeIntoHoodieTableCommand.scala:
##########
@@ -742,7 +744,10 @@ case class MergeIntoHoodieTableCommand(mergeInto:
MergeIntoTable) extends Hoodie
HoodieSyncConfig.META_SYNC_PARTITION_EXTRACTOR_CLASS.key ->
hiveSyncConfig.getString(HoodieSyncConfig.META_SYNC_PARTITION_EXTRACTOR_CLASS),
SqlKeyGenerator.PARTITION_SCHEMA -> partitionSchema.toDDL,
PAYLOAD_CLASS_NAME.key -> classOf[ExpressionPayload].getCanonicalName,
+ HoodieTableConfig.PAYLOAD_CLASS_NAME.key() ->
classOf[ExpressionPayload].getCanonicalName,
Review Comment:
Should this be avoided?
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/HoodieSparkValidateDuplicateKeyRecordMerger.scala:
##########
@@ -43,5 +45,9 @@ class HoodieSparkValidateDuplicateKeyRecordMerger extends
HoodieSparkRecordMerge
/**
* The kind of merging strategy this recordMerger belongs to. An UUID
represents merging strategy.
*/
- override def getMergingStrategy: String =
HoodieRecordMerger.DEFAULT_MERGER_STRATEGY_UUID
+ override def getMergingStrategy: String = STRATEGY_STR
+}
+
+object HoodieSparkValidateDuplicateKeyRecordMerger {
+ final val STRATEGY_STR = "fb092649-0fdc-4c14-9113-acde3034a6c4"
Review Comment:
Also is it necessary to use UUID format, maybe "0", "1", "2" are shorter IDs.
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/spark/sql/hudi/command/HoodieSparkValidateDuplicateKeyRecordMerger.scala:
##########
@@ -43,5 +45,9 @@ class HoodieSparkValidateDuplicateKeyRecordMerger extends
HoodieSparkRecordMerge
/**
* The kind of merging strategy this recordMerger belongs to. An UUID
represents merging strategy.
*/
- override def getMergingStrategy: String =
HoodieRecordMerger.DEFAULT_MERGER_STRATEGY_UUID
+ override def getMergingStrategy: String = STRATEGY_STR
+}
+
+object HoodieSparkValidateDuplicateKeyRecordMerger {
+ final val STRATEGY_STR = "fb092649-0fdc-4c14-9113-acde3034a6c4"
Review Comment:
Could we make the ID easier to read, e.g.,
"00000000-0000-0000-0000-000000000000": Avro payload based merging;
"00000000-0000-0000-0000-000000000001": overwrite with latest;
"00000000-0000-0000-0000-000000000002": event time ordering / default hoodie
payload. Still reserve the existing ID as an alternative ID;
"00000000-0000-0000-0000-000000000003": validate duplicate key record merger;
etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
