[
https://issues.apache.org/jira/browse/NUTCH-3147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18140160#comment-18140160
]
Lewis John McGibbney commented on NUTCH-3147:
---------------------------------------------
The metrics analysis report lists CPU, memory, and network utilization under
its missing metrics. These are JVM runtime signals rather than crawl counters.
When this issue adopts the OpenTelemetry Java SDK, we can enable the runtime
instrumentation (the Java runtime metrics for CPU, memory, and network) so
those signals are exported through the same pipeline as the Nutch counters. No
new Hadoop counters are needed for them, and no separate Nutch issue is planned.
> Nutch JMX Metrics Evolution with OpenTelemetry
> ----------------------------------------------
>
> Key: NUTCH-3147
> URL: https://issues.apache.org/jira/browse/NUTCH-3147
> Project: Nutch
> Issue Type: Sub-task
> Components: metrics
> Reporter: Lewis John McGibbney
> Priority: Major
> Fix For: 1.24
>
>
> This task will evolve the Nutch metrics system to provide real-time JMX
> observability using OpenTelemetry, supporting both NutchServer mode and
> MapReduce job execution while maintaining backward compatibility with
> existing Hadoop counters. This would dramatically improve the observability
> experience for Nutch administrators.
> Currently, Nutch metrics are only accessible via Hadoop MapReduce counters
> after job completion. The JMX enhancement will add real-time observability
> through:
> * JMX MBeans for live monitoring
> * Prometheus endpoint for scraping (if/when running NutchServer)
> * OTLP export for OpenTelemetry collectors
> I propose we instrument Nutch using the OpenTelemetry Java SDK. This will be
> a significant undertaking and should likely be implemented after everything
> else in NUTCH-3131.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
