github-actions[bot] opened a new pull request, #71854: URL: https://github.com/apache/airflow/pull/71854
* Document how to separate metrics from replicated Airflow components Every Airflow process reports the same service.name, so in HA the schedulers publish the same gauge series and the backend keeps whichever export landed last. Setting service.instance.id per replica already fixes this, but nothing in the metrics docs says so. Signed-off-by: 1fanwang <[email protected]> * Cover both service.name and service.instance.id, nest under OpenTelemetry Frame the section around identifying a metric's source rather than around the collision that motivated it: service.name says which component reports, and service.instance.id says which instance of it, so both are needed and each has its own environment variable. Make it a subsection of Setup - OpenTelemetry, since it only applies there, and move the StatsD caveat to the StatsD section where a StatsD user will meet it. Name the job label alongside instance, and drop the claim that one backend implements the Prometheus compatibility spec. Signed-off-by: 1fanwang <[email protected]> * Describe processes rather than replicas, and state the defaults Airflow processes running the same component are independent, with no leader-follower relationship, so calling them replicas describes something the deployment does not have. Talk about components and their processes instead. Say what each attribute defaults to and what that costs: service.name is airflow for every process, so a scheduler, a triggerer and a worker arrive under one name, and service.instance.id is unset, so processes running one component cannot be told apart. Distinguishing components is useful on its own, independently of running several of any one of them. Signed-off-by: 1fanwang <[email protected]> * Cover scheduler HA in the StatsD note and tighten the wording Every scheduler emits pool gauges under the same StatsD series, so with more than one the server keeps whichever value arrived last. Say so in the StatsD note rather than only pointing at OpenTelemetry. Signed-off-by: 1fanwang <[email protected]> * Use the same 'when' phrasing in the StatsD note Signed-off-by: 1fanwang <[email protected]> --------- (cherry picked from commit 9b43d6abc0fc0766b1b1a886657bb6befc33ee1d) Co-authored-by: Stefan Wang <[email protected]> Signed-off-by: 1fanwang <[email protected]> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
