prashantwason commented on code in PR #18183:
URL: https://github.com/apache/hudi/pull/18183#discussion_r2843417937
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -1370,6 +1371,21 @@ public Map<String, String> propsMap() {
.collect(Collectors.toMap(e -> String.valueOf(e.getKey()), e ->
String.valueOf(e.getValue())));
}
+ public static String getFileComment(Properties props) {
+ final long ts = System.currentTimeMillis();
Review Comment:
Done. Added hostname caching with double-checked locking and wrapped in
try-catch with fallback to "unknown".
##########
hudi-client/hudi-java-client/src/main/java/org/apache/hudi/client/common/HoodieJavaEngineContext.java:
##########
@@ -192,6 +193,13 @@ public void cancelAllJobs() {
// no operation for now
}
+ @Override
+ public Map<String, String> getInfo() {
+ final Map<String, String> info = new HashMap<String, String>();
+ System.getProperties().stringPropertyNames().forEach(property ->
info.put(property, System.getProperty(property)));
Review Comment:
Fixed. Now using an allowlist of safe system properties: java.version,
java.vendor, java.vm.name, java.vm.version, os.name, os.version, os.arch.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieClient.java:
##########
@@ -307,4 +310,14 @@ protected boolean
isStreamingWriteToMetadataEnabled(HoodieTable table) {
return config.isMetadataTableEnabled()
&&
config.isMetadataStreamingWritesEnabled(table.getMetaClient().getTableConfig().getTableVersion());
}
+
+ protected Option<Map<String, String>> updateExtraMetadata(Option<Map<String,
String>> extraMetadata) {
+ // Add write config, HUDI version and engine specific extra metadata
+ extraMetadata = extraMetadata.isPresent() ? extraMetadata : Option.of(new
HashMap<String, String>());
+ extraMetadata.get().put("hudi.version", HoodieVersion.get());
Review Comment:
Addressed. Now storing only key config properties (table name, type,
version, write operation, record key, partition path, parallelism settings,
concurrency mode, metadata enable) with a "config." prefix instead of dumping
the full config.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieClient.java:
##########
@@ -307,4 +310,14 @@ protected boolean
isStreamingWriteToMetadataEnabled(HoodieTable table) {
return config.isMetadataTableEnabled()
&&
config.isMetadataStreamingWritesEnabled(table.getMetaClient().getTableConfig().getTableVersion());
}
+
+ protected Option<Map<String, String>> updateExtraMetadata(Option<Map<String,
String>> extraMetadata) {
Review Comment:
Fixed. Now always creating a new HashMap and copying entries from the input
map if present.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieTableServiceClient.java:
##########
@@ -666,6 +666,7 @@ Option<String> scheduleTableServiceInternal(Option<String>
providedInstantTime,
if (!tableServicesEnabled(config)) {
return Option.empty();
}
+ extraMetadata = updateExtraMetadata(extraMetadata);
Review Comment:
Moved the updateExtraMetadata call after the ARCHIVE and CLEAN early-return
checks to avoid unnecessary computation.
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/client/common/HoodieFlinkEngineContext.java:
##########
@@ -233,6 +233,11 @@ public void cancelAllJobs() {
// no operation for now
}
+ @Override
+ public Map<String, String> getInfo() {
Review Comment:
Renamed to getEngineProperties() across all engine context implementations.
##########
hudi-common/src/main/java/org/apache/hudi/common/table/HoodieTableConfig.java:
##########
@@ -1370,6 +1371,21 @@ public Map<String, String> propsMap() {
.collect(Collectors.toMap(e -> String.valueOf(e.getKey()), e ->
String.valueOf(e.getValue())));
}
+ public static String getFileComment(Properties props) {
+ final long ts = System.currentTimeMillis();
+ return String.format("Date=%s, ts=%d, host=%s, #properties=%d,
hudi_version=%s",
+ new java.util.Date(ts), ts, NetworkUtils.getHostname(), props.size(),
HoodieVersion.get());
+ }
+
+ public static String getFileComment(Properties props, Map<String, String>
extraKeyValue) {
Review Comment:
Removed the unused overload method.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
