wu-sheng commented on code in PR #13891:
URL: https://github.com/apache/skywalking/pull/13891#discussion_r3362560315
##########
docs/en/setup/backend/backend-airflow-monitoring.md:
##########
@@ -0,0 +1,239 @@
+# Airflow monitoring
+
+## Airflow metrics via native OpenTelemetry
+
+SkyWalking receives Airflow metrics through Airflow's native OpenTelemetry
exporter and the
+[OpenTelemetry receiver](opentelemetry-receiver.md), then aggregates them with
+[MAL](../../concepts-and-designs/mal.md).
+
+## Data flow
+
+1. Enable OpenTelemetry metrics in Airflow (`pip install
'apache-airflow[otel]'`, `otel_on = True`
+ or standard `OTEL_EXPORTER_OTLP_*` environment variables).
+2. Airflow **pushes** OTLP metrics to OpenTelemetry Collector.
+3. OpenTelemetry Collector forwards metrics to SkyWalking OAP via OTLP gRPC
exporter.
+4. OAP applies MAL rules under `otel-rules/airflow/` and stores Service /
Instance entities on
+ `Layer: AIRFLOW`.
+
+```mermaid
+graph LR;
+ Airflow("Airflow") --> Collector("OTel Collector")
+ Collector --> OAP("SkyWalking OAP")
+ OAP --> UI("Horizon UI")
+```
+
+In the Horizon UI, Airflow appears under the **Workflow Scheduler** menu group.
+
+## Setup
+
+### 1. Enable Airflow OpenTelemetry metrics
+
+Install the OTel extra and enable metrics export. See the
+[Airflow metrics
documentation](https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/logging-monitoring/metrics.html).
+
+Example environment variables for Airflow 3.x:
+
+```bash
+pip install 'apache-airflow[otel]'
+
+export OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4318
+export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
+export OTEL_RESOURCE_ATTRIBUTES=cluster=prod-airflow
+```
+
+`cluster` is required so SkyWalking can name the Airflow Service
(`airflow::prod-airflow`). You
+can also inject it with a Collector `resource` processor.
Review Comment:
The default works, but the **hard requirement** behind it isn't stated, and
it's the most likely silent-failure misconfiguration. Suggest making it
explicit:
```suggestion
`cluster` is required so SkyWalking can name the Airflow Service
(`airflow::prod-airflow`). You
can also inject it with a Collector `resource` processor.
> **Required — `service.name` must be `Airflow`:** OAP maps the OTLP
resource `service.name` to the
> `job_name` tag, and every MAL rule under `otel-rules/airflow/` filters on
`job_name == 'Airflow'`
> (case-sensitive). Airflow's default OTel service name is `Airflow` (the
`[metrics] otel_service`
> default in both 2.x and 3.x), so a default install works out of the box.
If you override it via
> `OTEL_SERVICE_NAME`, set it to exactly `Airflow`, and do **not** put a
custom `job_name` in
> `OTEL_RESOURCE_ATTRIBUTES` — an explicit `job_name` takes precedence over
the `service.name`
> fallback. Otherwise OAP silently drops every Airflow metric and the
AIRFLOW layer stays empty.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
