This is an automated email from the ASF dual-hosted git repository.
onikolas pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 34d6efc2d5 Update Metrics docs (#31913)
34d6efc2d5 is described below
commit 34d6efc2d55fe60642ef7617ad061aae298938f1
Author: D. Ferruzzi <[email protected]>
AuthorDate: Mon Jun 19 11:40:39 2023 -0700
Update Metrics docs (#31913)
---
.../logging-monitoring/metrics.rst | 86 ++++++++++++++++------
1 file changed, 65 insertions(+), 21 deletions(-)
diff --git
a/docs/apache-airflow/administration-and-deployment/logging-monitoring/metrics.rst
b/docs/apache-airflow/administration-and-deployment/logging-monitoring/metrics.rst
index 93a2a782b2..4998fcb71d 100644
---
a/docs/apache-airflow/administration-and-deployment/logging-monitoring/metrics.rst
+++
b/docs/apache-airflow/administration-and-deployment/logging-monitoring/metrics.rst
@@ -17,21 +17,22 @@
-Metrics
-=======
+Metrics Configuration
+=====================
-Airflow can be set up to send metrics to `StatsD
<https://github.com/etsy/statsd>`__.
+Airflow can be set up to send metrics to `StatsD
<https://github.com/etsy/statsd>`__
+or `OpenTelemetry <https://opentelemetry.io/>`__.
-Setup
------
+Setup - StatsD
+--------------
-First you must install StatsD requirement:
+To use StatsD you must first install the required packages:
.. code-block:: bash
pip install 'apache-airflow[statsd]'
-Add the following lines to your configuration file e.g. ``airflow.cfg``
+then add the following lines to your configuration file e.g. ``airflow.cfg``
.. code-block:: ini
@@ -41,39 +42,82 @@ Add the following lines to your configuration file e.g.
``airflow.cfg``
statsd_port = 8125
statsd_prefix = airflow
-If you want to avoid sending all the available metrics to StatsD, you can
configure an allow list of prefixes to send only
-the metrics that start with the elements of the list:
+If you want to use a custom StatsD client instead of the default one provided
by Airflow,
+the following key must be added to the configuration file alongside the module
path of your
+custom StatsD client. This module must be available on your
:envvar:`PYTHONPATH`.
.. code-block:: ini
[metrics]
- statsd_allow_list = scheduler,executor,dagrun
+ statsd_custom_client_path = x.y.customclient
-If you want to redirect metrics to different name, you can configure
``stat_name_handler`` option
-in ``[metrics]`` section. It should point to a function that validates the
StatsD stat name, applies changes
-to the stat name if necessary, and returns the transformed stat name. The
function may looks as follow:
+See :doc:`../modules_management` for details on how Python and Airflow manage
modules.
-.. code-block:: python
- def my_custom_stat_name_handler(stat_name: str) -> str:
- return stat_name.lower()[:32]
+Setup - OpenTelemetry
+---------------------
+
+To use OpenTelemetry you must first install the required packages:
+
+.. code-block:: bash
-If you want to use a custom StatsD client instead of the default one provided
by Airflow, the following key must be added
-to the configuration file alongside the module path of your custom StatsD
client. This module must be available on
-your :envvar:`PYTHONPATH`.
+ pip install 'apache-airflow[otel]'
+
+Add the following lines to your configuration file e.g. ``airflow.cfg``
.. code-block:: ini
[metrics]
- statsd_custom_client_path = x.y.customclient
+ otel_on = False
+ otel_host = localhost
+ otel_port = 8889
+ otel_prefix = airflow
+ otel_interval_milliseconds = 30000 # The interval between exports,
defaults to 60000
+
+
+Allow/Block Lists
+-----------------
+
+If you want to avoid sending all the available metrics, you can configure an
allow list or block list
+of prefixes to send or block only the metrics that start with the elements of
the list:
+
+.. code-block:: ini
+
+ [metrics]
+ metrics_allow_list = scheduler,executor,dagrun
+
+.. code-block:: ini
+
+ [metrics]
+ metrics_block_list = scheduler,executor,dagrun
+
+
+Rename Metrics
+--------------
+
+If you want to redirect metrics to a different name, you can configure the
``stat_name_handler`` option
+in ``[metrics]`` section. It should point to a function that validates the
stat name, applies changes
+to the stat name if necessary, and returns the transformed stat name. The
function may look as follows:
+
+.. code-block:: python
+
+ def my_custom_stat_name_handler(stat_name: str) -> str:
+ return stat_name.lower()[:32]
-See :doc:`../modules_management` for details on how Python and Airflow manage
modules.
+
+Other Configuration Options
+---------------------------
.. note::
For a detailed listing of configuration options regarding metrics,
see the configuration reference documentation - :ref:`config:metrics`.
+
+Metric Descriptions
+===================
+
+
Counters
--------
