rkkalluri commented on code in PR #5205:
URL: https://github.com/apache/hudi/pull/5205#discussion_r842197121
##########
hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/HoodieWriterUtils.scala:
##########
@@ -173,6 +175,29 @@ object HoodieWriterUtils {
}
}
+ /**
+ * Detects conflicts between datasourceKeyGen and existing table
configuration keyGen
+ */
+ def validateKeyGeneratorConfig(datasourceKeyGen: String, tableConfig:
HoodieConfig): Unit = {
+ val diffConfigs = StringBuilder.newBuilder
+
+ if (null != tableConfig) {
Review Comment:
Case 1: when, in first commit it was default key gen (user did not pass
any key gen), and in 2nd commit, it was non-partitioned key gen. ---> This
should work even before our fix as we will be picking SimpleKeyGen by default
and it will be compared to the non-partitioned key gen. Test case
testDefaultKeyGenToNonpartitoned added for this.
Case 2 : 1st commit: no key gen passed. 2nd commit: explicitly setting
simple key gen. Result: should succeed w/o issues. -->
testNoKeyGenToSimpleKeyGen added to verify this case
Case 3: same as (1), but flipped. i..e 1st commit: simple key gen. 2nd
commit: no key gen set. Result: should succeed w/o issues. -->
testSimpleKeyGenToNoKeyGen added to verify this case
I did not want to make this overly generic as it might break other logic
with spark DDL and other keygen selections based on partitions etc.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
