danielcweeks commented on a change in pull request #4254:
URL: https://github.com/apache/iceberg/pull/4254#discussion_r819178970
##########
File path: core/src/main/java/org/apache/iceberg/CatalogProperties.java
##########
@@ -81,4 +81,8 @@ private CatalogProperties() {
public static final String APP_ID = "app-id";
public static final String USER = "user";
+ public static final String IO_METRICS_IMPL = "io.metrics-impl";
+ public static final String IO_METRICS_IMPL_DEFAULT =
"org.apache.iceberg.hadoop.HadoopMetricsContext";
+ public static final String IO_METRICS_SCHEME = "io.metrics-scheme";
Review comment:
`io.metrics-scheme` is hadoop specific since it's tied to a hadoop
filesystem scheme in terms of reporting. Do we really need to move it here?
##########
File path: aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java
##########
@@ -116,16 +118,19 @@ public void initialize(Map<String, String> properties) {
this.s3 = AwsClientFactories.from(properties)::s3;
}
- // Report Hadoop metrics if Hadoop is available
- try {
- DynConstructors.Ctor<MetricsContext> ctor =
-
DynConstructors.builder(MetricsContext.class).hiddenImpl(DEFAULT_METRICS_IMPL,
String.class).buildChecked();
- this.metrics = ctor.newInstance("s3");
+ String metricsImpl = properties.getOrDefault(
+ CatalogProperties.IO_METRICS_IMPL,
CatalogProperties.IO_METRICS_IMPL_DEFAULT);
- metrics.initialize(properties);
- } catch (NoSuchMethodException | ClassCastException e) {
- LOG.warn("Unable to load metrics class: '{}', falling back to null
metrics", DEFAULT_METRICS_IMPL, e);
+ Map<String, String> metricContextProperties = properties;
+ if (!properties.containsKey(CatalogProperties.IO_METRICS_SCHEME)) {
+ metricContextProperties = ImmutableMap.<String, String>builder()
+ .putAll(properties)
+ .put(CatalogProperties.IO_METRICS_SCHEME, IO_METRICS_SCHEME_S3)
+ .build();
}
+
+ this.metrics = CatalogUtil.loadFileIOMetricsContext(
Review comment:
this overrides the default `MetricsContext.nullMetrics()` that gets set
in the variable initialization win will result in NPE (see other note in
catalog util)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
