LIKE-HUB opened a new issue, #18141:
URL: https://github.com/apache/hudi/issues/18141
### Bug Description
**What happened:**
When I was using Flink to write data to Hudi, I wanted to obtain the metric
information during the writing process.
The only information I obtained in the Metric class is key =
"sink_table.index.type", value = 6.Why can't we obtain the metric information?
flink1.20.3
hudi 1.1.1
**What you expected:**
I will eventually write the metric information to MySQL.
**Steps to reproduce:**
public class MetricNew extends AbstractUserDefinedMetricsReporter {
private static final Logger log = LogManager.getLogger(MetricNew.class);
private final ScheduledExecutorService exec =
Executors.newSingleThreadScheduledExecutor(r -> {
Thread t = new Thread(r, "hudi-user-metrics-reporter");
t.setDaemon(true);
return t;
});
public MetricNew(Properties props, MetricRegistry registry) {
super(props, registry);
}
/**
* ⚠️ Flink 场景下必须自己调度 report
*/
@Override
public void start() {
// 延迟 10s 后开始，每 30s 打印一次
exec.scheduleAtFixedRate(this::report, 10, 30, TimeUnit.SECONDS);
log.info("MetricNew started, schedule metrics report");
}
/**
* 只做一件事：打印 registry 里已有的 metrics
*/
@Override
public void report() {
try {
if (getRegistry() == null ||
getRegistry().getGauges().isEmpty()) {
log.info("No gauges found in MetricRegistry");
return;
}
getRegistry().getGauges().forEach((key, value) -> {
Object v;
try {
v = value.getValue();
} catch (Exception e) {
v = "N/A";
}
log.info("[HUDI-METRIC] key=" + key + ", value=" + v);
});
} catch (Throwable t) {
log.error("MetricNew report failed", t);
}
}
@Override
public void stop() {
exec.shutdownNow();
log.info("MetricNew stopped");
}
}
### Environment
**Hudi version:**
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
