vinothchandar commented on a change in pull request #1500: [HUDI-772] Make
UserDefinedBulkInsertPartitioner configurable for DataSource
URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r407232824
##########
File path: hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java
##########
@@ -152,6 +154,24 @@ public static KeyGenerator
createKeyGenerator(TypedProperties props) throws IOEx
}
}
+ /**
+ * Create a UserDefinedBulkInsertPartitioner class via reflection,
+ * <br>
+ * if the class name of UserDefinedBulkInsertPartitioner is configured
through the HoodieWriteConfig.
+ * @see HoodieWriteConfig#getUserDefinedBulkInsertPartitionerClass()
+ */
+ private static Option<UserDefinedBulkInsertPartitioner>
createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)
+ throws IOException {
+ String bulkInsertPartitionerClass =
config.getUserDefinedBulkInsertPartitionerClass();
+ try {
+ return bulkInsertPartitionerClass == null ||
bulkInsertPartitionerClass.isEmpty()
+ ? Option.empty() :
+ Option.of((UserDefinedBulkInsertPartitioner)
ReflectionUtils.loadClass(bulkInsertPartitionerClass));
+ } catch (Throwable e) {
+ throw new IOException("Could not create UserDefinedBulkInsertPartitioner
class " + bulkInsertPartitionerClass, e);
Review comment:
throw a HoodieException? this is not an IOException per se?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
