danny0405 commented on code in PR #13557:
URL: https://github.com/apache/hudi/pull/13557#discussion_r2209127314
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/configuration/FlinkOptions.java:
##########
@@ -128,6 +128,20 @@ private FlinkOptions() {
.withDescription("Payload class used. Override this, if you like to roll
your own merge logic, when upserting/inserting.\n"
+ "This will render any value set for the option in-effective");
+ public static final ConfigOption<String> INSERT_PARTITIONER_CLASS_NAME =
ConfigOptions
+ .key("write.insert.partitioner.class.name")
+ .stringType()
+ .defaultValue("")
+ .withDescription("Insert partitioner to use aiming to re-balance records
and reducing small file number "
+ + "in the scenario of multi-level partitioning. For example
dt/hour/eventID"
+ + "Currently support
org.apache.hudi.sink.partitioner.DefaultInsertPartitioner");
+
+ public static final ConfigOption<Integer> DEFAULT_PARALLELISM_PER_PARTITION
= ConfigOptions
+ .key("write.insert.partitioner.default_parallelism_per_partition")
+ .intType()
+ .defaultValue(30)
Review Comment:
we have `OptionsInference` to help infer the parallelism.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
