vinothchandar commented on a change in pull request #1500: [HUDI-772] Make
UserDefinedBulkInsertPartitioner configurable for DataSource
URL: https://github.com/apache/incubator-hudi/pull/1500#discussion_r409747421
##########
File path: hudi-spark/src/main/java/org/apache/hudi/DataSourceUtils.java
##########
@@ -152,6 +154,24 @@ public static KeyGenerator
createKeyGenerator(TypedProperties props) throws IOEx
}
}
+ /**
+ * Create a UserDefinedBulkInsertPartitioner class via reflection,
+ * <br>
+ * if the class name of UserDefinedBulkInsertPartitioner is configured
through the HoodieWriteConfig.
+ * @see HoodieWriteConfig#getUserDefinedBulkInsertPartitionerClass()
+ */
+ private static Option<UserDefinedBulkInsertPartitioner>
createUserDefinedBulkInsertPartitioner(HoodieWriteConfig config)
+ throws IOException {
+ String bulkInsertPartitionerClass =
config.getUserDefinedBulkInsertPartitionerClass();
+ try {
+ return bulkInsertPartitionerClass == null ||
bulkInsertPartitionerClass.isEmpty()
+ ? Option.empty() :
+ Option.of((UserDefinedBulkInsertPartitioner)
ReflectionUtils.loadClass(bulkInsertPartitionerClass));
+ } catch (Throwable e) {
+ throw new IOException("Could not create UserDefinedBulkInsertPartitioner
class " + bulkInsertPartitionerClass, e);
Review comment:
correct.. a cleanup of these exceptions would be a good follow up.. For now,
let's try fixing in baby steps, by having this throw a generic HoodieException
(both are uncaught for the user anwyay).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
