danny0405 commented on code in PR #13901:
URL: https://github.com/apache/hudi/pull/13901#discussion_r2353963209
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/metadata/FlinkHoodieBackedTableMetadataWriter.java:
##########
@@ -190,6 +190,13 @@ public void deletePartitions(String instantTime,
List<MetadataPartitionType> par
@Override
protected void preWrite(String instantTime) {
metadataMetaClient.getActiveTimeline().transitionRequestedToInflight(HoodieActiveTimeline.DELTA_COMMIT_ACTION,
instantTime);
+ // Every time `HFile` class is loaded in class loader, it'll create a
static `MetricsIO` instance for region server IO and
+ // register a metric 'RegionServer,sub=IO' in
`DefaultMetricsSystem#newSourceName`. There is no configuration to disable it.
+ // When the flink cluster is deployed as session mode and hudi flink
bundle jar is summited with SQL or datastream job,
+ // multiple userCode context classloaders may register same region server
metric which leads to validating exception in
+ // `DefaultMetricsSystem#newSourceName`. So here we set the init mode for
hadoop metrics as `STANDBY` to disable metric
+ // registering for HFile writer.
+ System.setProperty("hadoop.metrics.init.mode", "STANDBY");
Review Comment:
Does the issue exist on master? guess no because Hbase jar has been removed?
Also there is no need to set it up every time for MDT commit? Just set up once
in coordinator `#initMetadataTable` should be enough?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
