yihua commented on code in PR #18187:
URL: https://github.com/apache/hudi/pull/18187#discussion_r2800024207
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1419,7 +1419,10 @@ public void
validateAgainstTableProperties(HoodieTableConfig tableConfig, Hoodie
}
if (!keyGenClass.equals("org.apache.hudi.keygen.SimpleKeyGenerator")
&&
!keyGenClass.equals("org.apache.hudi.keygen.NonpartitionedKeyGenerator")
- &&
!keyGenClass.equals("org.apache.hudi.keygen.ComplexKeyGenerator")) {
+ && !keyGenClass.equals("org.apache.hudi.keygen.ComplexKeyGenerator")
Review Comment:
The fix is correct. As suggested in other comment, good to maintain an
allowlist with `KeyGeneratorType` on this.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -1419,7 +1419,10 @@ public void
validateAgainstTableProperties(HoodieTableConfig tableConfig, Hoodie
}
if (!keyGenClass.equals("org.apache.hudi.keygen.SimpleKeyGenerator")
&&
!keyGenClass.equals("org.apache.hudi.keygen.NonpartitionedKeyGenerator")
- &&
!keyGenClass.equals("org.apache.hudi.keygen.ComplexKeyGenerator")) {
+ && !keyGenClass.equals("org.apache.hudi.keygen.ComplexKeyGenerator")
+ &&
!keyGenClass.equals("org.apache.hudi.keygen.SimpleAvroKeyGenerator")
Review Comment:
The fix looks correct to me. When Flink writes a table with
`populateMetaFields=false`, it stores the Avro key generator class name (e.g.,
`ComplexAvroKeyGenerator`) in `hoodie.properties`. Spark then reads that class
name back at validation time, and the current code rejects it because only the
Spark wrapper names were allowlisted.
One thought: this validation could be made more robust by checking against
`KeyGeneratorType` instead of hardcoded strings, e.g., resolving the class name
via `KeyGeneratorType.fromClassName()` and then checking against a `Set` of
allowed types (`SIMPLE`, `SIMPLE_AVRO`, `COMPLEX`, `COMPLEX_AVRO`,
`NON_PARTITION`, `NON_PARTITION_AVRO`). That way future key generator variants
wouldn't require another string addition here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
