Lewis John McGibbney created NUTCH-3140:
-------------------------------------------
Summary: Create example Observability Dashboards for Nutch Metrics
Key: NUTCH-3140
URL: https://issues.apache.org/jira/browse/NUTCH-3140
Project: Nutch
Issue Type: Sub-task
Components: metrics
Reporter: Lewis John McGibbney
Assignee: Lewis John McGibbney
Fix For: 1.22
The last while I have been experimenting bringing to visual aspect to Nutch
metrics :)
This ticket tracks the creation of a dedicated repository
(nutch-grafana-resources) providing ready-to-use Grafana observability
resources for monitoring Nutch web. The resources enable operators to gain
real-time visibility into crawl job performance, health, and issues.
Over time I will experiment using the Prometheus JMX exporter but for now this
is an adequate next step.
Some narrative on my technology choices...
*Why Grafana?*
* Industry-standard open-source visualization platform with extensive plugin
ecosystem
* Supports both self-hosted and managed (Grafana Cloud) deployment options
* Native integration with Prometheus (metrics) and Loki (logs)
* Active community and commercial support available
* Already widely adopted in the Apache ecosystem
* Free tier available for small-scale monitoring
*Why Grafana Alloy?*
* Official Grafana telemetry collector (successor to Grafana Agent)
* OpenTelemetry Collector distribution with Prometheus pipeline support
* Single binary that handles both logs and metrics collection
* Built-in log processing pipeline with regex extraction and metrics generation
* Native support for Loki and Prometheus remote write
* Eliminates need for separate Prometheus exporters or log shippers
* Declarative configuration format with live debugging capabilities
Disclaimer: I am now employed by Grafana Labs. I DO NOT intend to integrate
these dashboards or collector resources into Apache Nutch. I just wanted to
open this ticket and share some of the resources I've created so others could
use them if so inclined. (y)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
