cshuo commented on code in PR #13901:
URL: https://github.com/apache/hudi/pull/13901#discussion_r2354188983
##########
hudi-client/hudi-flink-client/src/main/java/org/apache/hudi/metadata/FlinkHoodieBackedTableMetadataWriter.java:
##########
@@ -190,6 +190,13 @@ public void deletePartitions(String instantTime,
List<MetadataPartitionType> par
@Override
protected void preWrite(String instantTime) {
metadataMetaClient.getActiveTimeline().transitionRequestedToInflight(HoodieActiveTimeline.DELTA_COMMIT_ACTION,
instantTime);
+ // Every time `HFile` class is loaded in class loader, it'll create a
static `MetricsIO` instance for region server IO and
+ // register a metric 'RegionServer,sub=IO' in
`DefaultMetricsSystem#newSourceName`. There is no configuration to disable it.
+ // When the flink cluster is deployed as session mode and hudi flink
bundle jar is summited with SQL or datastream job,
+ // multiple userCode context classloaders may register same region server
metric which leads to validating exception in
+ // `DefaultMetricsSystem#newSourceName`. So here we set the init mode for
hadoop metrics as `STANDBY` to disable metric
+ // registering for HFile writer.
+ System.setProperty("hadoop.metrics.init.mode", "STANDBY");
Review Comment:
Yes, the issue only exists before 1.1.
We can put the setting up in `#initMetadataTable`, but my concern is
`hadoop.metrics.init.mode` is not only for hbase metric, it's a hadoop
configuration, and not sure whether it'll affect the initialization of other
component's metric, so I incline to delay setting this configuration until the
HFile writer initialization is required.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
