xBis7 commented on PR #67328:
URL: https://github.com/apache/airflow/pull/67328#issuecomment-4660196660
> Do you have an idea? Looks for me like a new PR?
I would suggest adding them as part of this PR so that we can also verify
that your fix is the right one.
I would add a new test file
`providers/edge3/tests/unit/edge3/models/test_edge_worker.py` and then call
`set_metrics()` and test that both new and legacy metrics have been exported.
You don't have to assert on all the metrics or their tags. Just pick 1 metric,
e.g. if the worker name is `test_worker1`, then check that you can find both
`edge_worker.status` and `edge_worker.status.test_worker1`.
If we are in `AIRFLOW_V_3_3_PLUS` then `Stats` will be delegating to the
module-level functions of `stats` which will then do the dual emmission. For
example,
```python
@mock.patch("airflow.sdk._shared.observability.metrics.stats._get_backend")
def test_set_metrics(mock_get_backend):
mock_backend = mock.MagicMock(spec=StatsLogger)
mock_get_backend.return_value = mock_backend
set_metrics(
worker_name="test_worker1",
state=EdgeWorkerState.IDLE,
jobs_active=0,
concurrency=1,
free_concurrency=1,
queues=None,
sysinfo={"status": 1},
)
metric_names = [call.args[0] for call in
mock_backend.gauge.call_args_list]
print(f"captured metrics: {metric_names}")
assert "edge_worker.status" in metric_names
assert "edge_worker.status.test_worker1" in metric_names
```
and for older versions you should patch `Stats` itself.
Here are some examples of handling back-compat in providers
https://github.com/xBis7/airflow/blob/main/providers/openlineage/tests/unit/openlineage/plugins/test_adapter.py#L70
https://github.com/xBis7/airflow/blob/main/providers/openlineage/tests/unit/openlineage/plugins/test_adapter.py#L307-L308
https://github.com/xBis7/airflow/blob/main/providers/celery/tests/unit/celery/executors/test_celery_executor.py#L57-L65
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
