The GitHub Actions job "Tests (AMD)" on airflow.git/backport-67527-to-v3-2-test 
has succeeded.
Run started by GitHub user vatsrahul1001 (triggered by vatsrahul1001).

Head commit for run:
98c9f3cfe22b2d5dc2f64de2edd5e71375071013 / vatsrahul1001 
<[email protected]>
Fix Callback.handle_event crash on OTel metrics with dict tag values

The triggerer crashes on the next deadline async callback when OpenTelemetry
metrics are enabled:

    File ".../airflow/jobs/triggerer_job_runner.py", line 659, in handle_events
        Trigger.submit_event(...)
    File ".../airflow/models/callback.py", line 234, in handle_event
        Stats.incr(**self.get_metric_info(status, self.output))
    File ".../airflow/_shared/observability/metrics/otel_logger.py", line 211, 
in incr
        counter.add(count, attributes=tags)
    File ".../opentelemetry/sdk/metrics/.../view_instrument_match.py", line 105
        aggr_key = frozenset(attributes.items())
    TypeError: unhashable type: 'dict'

`Callback.get_metric_info` builds the metric tags dict directly from the
callback's `result` and `self.data` (which includes `kwargs`). Both are
frequently dicts — for deadline async callbacks the `result` is the user
callback's return value, and `kwargs` is the captured callback kwargs. When
the metrics backend is OTel, the SDK builds the aggregation key as
`frozenset(attributes.items())`, which raises if any value is unhashable
(dict, list, set). The result is a triggerer crash and stalled triggers.

The bug is metrics-backend-dependent: statsd accepts non-primitive tag values
without complaint, so OSS users running default statsd never see it. OTel
backends (used in production by Astronomer Astro Cloud and any OSS deployment
that enables `[metrics] otel_*`) hit it consistently.

Reproduces against 3.2.1 and main; not a 3.2.x regression.

Sanitize tag values to primitives before returning from `get_metric_info`:
keep `str | int | float | bool | None` as-is, JSON-stringify anything else.
Using `default=str` in `json.dumps` so values like `datetime` fall back
cleanly instead of raising.

Adds a regression test that asserts every tag value is hashable and that
`frozenset(tags.items())` does not raise.

Reported by Astronomer Runtime team while testing 3.2.2rc2-based images.

(cherry picked from commit 9a34e73e729441556e50e08a1f577ae337a369b4)

Report URL: https://github.com/apache/airflow/actions/runs/26436515754

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to