lokeshj1703 commented on code in PR #11601:
URL: https://github.com/apache/hudi/pull/11601#discussion_r1730694520
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -518,6 +518,58 @@ public static boolean validateChecksum(Properties props) {
return Long.parseLong(props.getProperty(TABLE_CHECKSUM.key())) ==
generateChecksum(props);
}
+ /**
+ * This function returns the partition fields joined by
BaseKeyGenerator.FIELD_SEPARATOR. It will also
+ * include the key generator partition type with the field. The key
generator partition type is used for
+ * Custom Key Generator.
+ */
+ public static Option<String>
getPartitionFieldPropForKeyGenerator(HoodieConfig config) {
+ return Option.ofNullable(config.getString(PARTITION_FIELDS));
+ }
+
+ /**
+ * This function returns the partition fields joined by
BaseKeyGenerator.FIELD_SEPARATOR. It will
+ * strip the partition key generator related info from the fields.
+ */
+ public static Option<String> getPartitionFieldProp(HoodieConfig config) {
+ // With table version eight, the table config
org.apache.hudi.common.table.HoodieTableConfig.PARTITION_FIELDS
+ // stores the corresponding partition type as well. This partition type is
useful for CustomKeyGenerator
+ // and CustomAvroKeyGenerator.
+ return getPartitionFields(config).map(fields ->
String.join(BaseKeyGenerator.FIELD_SEPARATOR, fields));
+ }
+
+ /**
+ * This function returns the partition fields only. This method strips the
key generator related
+ * partition key types from the configured fields.
+ */
+ public static Option<String[]> getPartitionFields(HoodieConfig config) {
+ if (contains(PARTITION_FIELDS, config)) {
+ return
Option.of(Arrays.stream(config.getString(PARTITION_FIELDS).split(","))
+ .filter(p -> !p.isEmpty())
+ .map(p -> getPartitionFieldWithoutKeyGenPartitionType(p, config))
+ .collect(Collectors.toList()).toArray(new String[] {}));
+ }
+ return Option.empty();
+ }
+
+ /**
+ * This function returns the partition fields only. The input partition
field would contain partition
+ * type corresponding to the custom key generator if table version is eight
and if custom key
+ * generator is configured. This function would strip the partition type and
return the partition field.
+ */
+ public static String getPartitionFieldWithoutKeyGenPartitionType(String
partitionField, HoodieConfig config) {
Review Comment:
Yes, it works for older format as well. Created a minor PR to add test as
well.
https://github.com/apache/hudi/pull/11824
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
