lokeshj1703 commented on code in PR #8041:
URL: https://github.com/apache/hudi/pull/8041#discussion_r1119786931
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java:
##########
@@ -40,27 +48,51 @@ public class Metrics {
private static Metrics instance = null;
private final MetricRegistry registry;
- private MetricsReporter reporter;
+ private final List<MetricsReporter> reporters;
private final String commonMetricPrefix;
private Metrics(HoodieWriteConfig metricConfig) {
registry = new MetricRegistry();
commonMetricPrefix = metricConfig.getMetricReporterMetricsNamePrefix();
- reporter = MetricsReporterFactory.createReporter(metricConfig, registry);
- if (reporter == null) {
- throw new RuntimeException("Cannot initialize Reporter.");
+ reporters = new ArrayList<>();
+ MetricsReporter defaultReporter =
MetricsReporterFactory.createReporter(metricConfig, registry);
+ if (defaultReporter != null) {
+ reporters.add(defaultReporter);
}
- reporter.start();
+ reporters.addAll(addAdditionalMetricsExporters(metricConfig));
+ if (reporters.size() == 0) {
+ throw new RuntimeException("Cannot initialize Reporters.");
+ }
+ reporters.forEach(MetricsReporter::start);
Runtime.getRuntime().addShutdownHook(new Thread(Metrics::shutdown));
}
+ private List<MetricsReporter>
addAdditionalMetricsExporters(HoodieWriteConfig metricConfig) {
+ List<MetricsReporter> reporterList = new ArrayList<>();
+ if
(!StringUtils.isNullOrEmpty(metricConfig.getMetricReporterFileBasedConfigs())) {
+ List<String> propPathList =
StringUtils.split(metricConfig.getMetricReporterFileBasedConfigs(), ",");
+ try (FileSystem fs = FSUtils.getFs(propPathList.get(0), new
Configuration())) {
+ for (String propPath: propPathList) {
+ HoodieWriteConfig secondarySourceConfig =
HoodieWriteConfig.newBuilder().fromInputStream(
+ fs.open(new
Path(propPath))).withPath(metricConfig.getBasePath()).build();
+
reporterList.add(MetricsReporterFactory.createReporter(secondarySourceConfig,
registry));
+ }
+ } catch (IOException e) {
+ LOG.error("Failed to add MetricsExporters", e);
+ throw new HoodieException("failed to MetricsExporters", e);
Review Comment:
In the latest commit I have removed the exception.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java:
##########
@@ -118,6 +150,11 @@ public static void registerGauges(Map<String, Long>
metricsMap, Option<String> p
metricsMap.forEach((k, v) -> registerGauge(metricPrefix + k, v));
}
+ public static void registerGauges(Map<String, Long> metricsMap,
Option<String> prefix, Map<String, String> labels) {
Review Comment:
Removed it.
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/MetricsReporterFactory.java:
##########
@@ -56,6 +56,9 @@ public static MetricsReporter
createReporter(HoodieWriteConfig config, MetricReg
MetricsReporterType type = config.getMetricsReporterType();
MetricsReporter reporter = null;
+ if (type == null) {
+ return reporter;
Review Comment:
If type is null, there is no default reporter created. Type is used to
determine whether reporter needs to be created or not today. So I am not
throwing an exception here but I am converting return type to Option for this
function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
