lokeshj1703 commented on code in PR #8041:
URL: https://github.com/apache/hudi/pull/8041#discussion_r1119632628
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/metrics/Metrics.java:
##########
@@ -40,27 +48,51 @@ public class Metrics {
private static Metrics instance = null;
private final MetricRegistry registry;
- private MetricsReporter reporter;
+ private final List<MetricsReporter> reporters;
private final String commonMetricPrefix;
private Metrics(HoodieWriteConfig metricConfig) {
registry = new MetricRegistry();
commonMetricPrefix = metricConfig.getMetricReporterMetricsNamePrefix();
- reporter = MetricsReporterFactory.createReporter(metricConfig, registry);
- if (reporter == null) {
- throw new RuntimeException("Cannot initialize Reporter.");
+ reporters = new ArrayList<>();
+ MetricsReporter defaultReporter =
MetricsReporterFactory.createReporter(metricConfig, registry);
+ if (defaultReporter != null) {
+ reporters.add(defaultReporter);
}
- reporter.start();
+ reporters.addAll(addAdditionalMetricsExporters(metricConfig));
+ if (reporters.size() == 0) {
+ throw new RuntimeException("Cannot initialize Reporters.");
+ }
+ reporters.forEach(MetricsReporter::start);
Runtime.getRuntime().addShutdownHook(new Thread(Metrics::shutdown));
}
+ private List<MetricsReporter>
addAdditionalMetricsExporters(HoodieWriteConfig metricConfig) {
+ List<MetricsReporter> reporterList = new ArrayList<>();
+ if
(!StringUtils.isNullOrEmpty(metricConfig.getMetricReporterFileBasedConfigs())) {
+ List<String> propPathList =
StringUtils.split(metricConfig.getMetricReporterFileBasedConfigs(), ",");
+ try (FileSystem fs = FSUtils.getFs(propPathList.get(0), new
Configuration())) {
+ for (String propPath: propPathList) {
+ HoodieWriteConfig secondarySourceConfig =
HoodieWriteConfig.newBuilder().fromInputStream(
+ fs.open(new
Path(propPath))).withPath(metricConfig.getBasePath()).build();
+
reporterList.add(MetricsReporterFactory.createReporter(secondarySourceConfig,
registry));
+ }
+ } catch (IOException e) {
+ LOG.error("Failed to add MetricsExporters", e);
+ throw new HoodieException("failed to MetricsExporters", e);
Review Comment:
I guess we could ignore a connection issue but this error would occur while
reading the file. This would be more of a setup issue and could lead to a
silent failure. It usually would take some time to debug these issues. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
