[
https://issues.apache.org/jira/browse/NUTCH-3140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lewis John McGibbney updated NUTCH-3140:
----------------------------------------
Description:
The last while I have been experimenting bringing to visual aspect to Nutch
metrics :)
This ticket tracks the creation of a dedicated repository
(nutch-grafana-resources) providing ready-to-use Grafana observability
resources for monitoring Nutch web. The resources enable operators to gain
real-time visibility into crawl job performance, health, and issues.
Over time I will experiment using the Prometheus JMX exporter but for now this
is an adequate next step.
Some narrative on my technology choices...
*Why Grafana?*
* Industry-standard open-source visualization platform with extensive plugin
ecosystem
* Supports both self-hosted and managed (Grafana Cloud) deployment options
* Native integration with Prometheus (metrics) and Loki (logs)
* Active community and commercial support available
* Already widely adopted in the Apache ecosystem
* Free tier available for small-scale monitoring
*Why Grafana Alloy?*
* Official Grafana telemetry collector (successor to Grafana Agent)
* OpenTelemetry Collector distribution with Prometheus pipeline support
* Single binary that handles both logs and metrics collection
* Built-in log processing pipeline with regex extraction and metrics generation
* Native support for Loki and Prometheus remote write
* Eliminates need for separate Prometheus exporters or log shippers
* Declarative configuration format with live debugging capabilities
Disclaimer: I am now employed by Grafana Labs. I DO NOT intend to integrate
these dashboards or collector resources into Apache Nutch unless I am asked to
do so. I just wanted to open this ticket and share some of the resources I've
created so others could use them if so inclined. (y)
was:
The last while I have been experimenting bringing to visual aspect to Nutch
metrics :)
This ticket tracks the creation of a dedicated repository
(nutch-grafana-resources) providing ready-to-use Grafana observability
resources for monitoring Nutch web. The resources enable operators to gain
real-time visibility into crawl job performance, health, and issues.
Over time I will experiment using the Prometheus JMX exporter but for now this
is an adequate next step.
Some narrative on my technology choices...
*Why Grafana?*
* Industry-standard open-source visualization platform with extensive plugin
ecosystem
* Supports both self-hosted and managed (Grafana Cloud) deployment options
* Native integration with Prometheus (metrics) and Loki (logs)
* Active community and commercial support available
* Already widely adopted in the Apache ecosystem
* Free tier available for small-scale monitoring
*Why Grafana Alloy?*
* Official Grafana telemetry collector (successor to Grafana Agent)
* OpenTelemetry Collector distribution with Prometheus pipeline support
* Single binary that handles both logs and metrics collection
* Built-in log processing pipeline with regex extraction and metrics generation
* Native support for Loki and Prometheus remote write
* Eliminates need for separate Prometheus exporters or log shippers
* Declarative configuration format with live debugging capabilities
Disclaimer: I am now employed by Grafana Labs. I DO NOT intend to integrate
these dashboards or collector resources into Apache Nutch. I just wanted to
open this ticket and share some of the resources I've created so others could
use them if so inclined. (y)
> Create example Observability Dashboards for Nutch Metrics
> ----------------------------------------------------------
>
> Key: NUTCH-3140
> URL: https://issues.apache.org/jira/browse/NUTCH-3140
> Project: Nutch
> Issue Type: Sub-task
> Components: metrics
> Reporter: Lewis John McGibbney
> Assignee: Lewis John McGibbney
> Priority: Major
> Fix For: 1.22
>
>
> The last while I have been experimenting bringing to visual aspect to Nutch
> metrics :)
> This ticket tracks the creation of a dedicated repository
> (nutch-grafana-resources) providing ready-to-use Grafana observability
> resources for monitoring Nutch web. The resources enable operators to gain
> real-time visibility into crawl job performance, health, and issues.
> Over time I will experiment using the Prometheus JMX exporter but for now
> this is an adequate next step.
> Some narrative on my technology choices...
> *Why Grafana?*
> * Industry-standard open-source visualization platform with extensive plugin
> ecosystem
> * Supports both self-hosted and managed (Grafana Cloud) deployment options
> * Native integration with Prometheus (metrics) and Loki (logs)
> * Active community and commercial support available
> * Already widely adopted in the Apache ecosystem
> * Free tier available for small-scale monitoring
> *Why Grafana Alloy?*
> * Official Grafana telemetry collector (successor to Grafana Agent)
> * OpenTelemetry Collector distribution with Prometheus pipeline support
> * Single binary that handles both logs and metrics collection
> * Built-in log processing pipeline with regex extraction and metrics
> generation
> * Native support for Loki and Prometheus remote write
> * Eliminates need for separate Prometheus exporters or log shippers
> * Declarative configuration format with live debugging capabilities
> Disclaimer: I am now employed by Grafana Labs. I DO NOT intend to integrate
> these dashboards or collector resources into Apache Nutch unless I am asked
> to do so. I just wanted to open this ticket and share some of the resources
> I've created so others could use them if so inclined. (y)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
