hudi-agent commented on code in PR #18885:
URL: https://github.com/apache/hudi/pull/18885#discussion_r3429140995
##########
hudi-common/src/main/java/org/apache/hudi/common/util/ConfigUtils.java:
##########
@@ -183,6 +186,40 @@ public static boolean
shouldKeepConsistentLogicalTimestamp(TypedProperties props
KEYGENERATOR_CONSISTENT_LOGICAL_TIMESTAMP_ENABLED.defaultValue()));
}
+ /**
+ * Removes Hudi metadata fields from schema-valued config properties.
+ */
+ public static void removeMetadataFieldsFromSchemaConfigs(HoodieConfig
config, List<ConfigProperty<String>> schemaConfigProperties) {
+ for (ConfigProperty<String> schemaConfigProperty : schemaConfigProperties)
{
+ if (config.getProps().containsKey(schemaConfigProperty.key())) {
+ config.setValue(schemaConfigProperty,
removeMetadataFieldsFromSchema(config.getString(schemaConfigProperty)));
+ }
+ }
+ }
+
+ /**
+ * Removes Hudi metadata fields from a Hudi schema string.
+ */
+ public static String removeMetadataFieldsFromSchema(String schemaStr) {
+ if (StringUtils.isNullOrEmpty(schemaStr) ||
!containsHudiMetadataField(schemaStr)) {
+ return schemaStr;
+ }
+ HoodieSchema schema = HoodieSchema.parse(schemaStr);
+ if (schema.isSchemaNull()) {
+ return schemaStr;
+ }
+ return HoodieSchemaUtils.removeMetadataFields(schema).toString();
+ }
+
+ private static boolean containsHudiMetadataField(String schemaStr) {
+ for (String metaField : HoodieRecord.HOODIE_META_COLUMNS_WITH_OPERATION) {
Review Comment:
🤖 nit: could you simplify the loop to `return
HoodieRecord.HOODIE_META_COLUMNS_WITH_OPERATION.stream().anyMatch(schemaStr::contains);`?
Reads more naturally and drops the early-return boilerplate.
<sub><i>- AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
