xBis7 commented on code in PR #74046: URL: https://github.com/apache/airflow/pull/74046#discussion_r4336727058
########## airflow-core/docs/administration-and-deployment/logging-monitoring/metrics.rst: ########## @@ -184,6 +185,37 @@ How the attributes surface depends on the backend. Those implementing the OpenTe `Prometheus compatibility <https://opentelemetry.io/docs/specs/otel/compatibility/prometheus_and_openmetrics/>`_ spec expose them as the ``job`` and ``instance`` labels. +.. _short-lived-processes: + +Short-lived processes +^^^^^^^^^^^^^^^^^^^^^ + +A task runs in a process that exits when the attempt ends, so a running total exported by that +process never grows if the values are cumulative. Task SDK processes therefore export counters +and histograms as **delta** values and share ``service.instance.id=task-sdk``, so that every +attempt adds to the same series. ``OTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE`` and +``OTEL_RESOURCE_ATTRIBUTES`` override either, if set. + +For backends that support delta values natively, no change is needed. Backends that only +accept cumulative values, such as Prometheus, need the Collector to convert the +values back and to keep idle series long enough for the next run of a Dag to add to them: + +.. code-block:: yaml + + processors: + deltatocumulative: + max_stale: 168h # 1 week Review Comment: I updated the docs with more information about this and some scenarios where we might end up with an OOM error. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
