xBis7 commented on PR #74046: URL: https://github.com/apache/airflow/pull/74046#issuecomment-6199326699
@dstandish I've updated the PR and changed the approach. The most important metrics * `ti.start` * `ti.finish` * `ti_successes` and `ti_failures` * `operator_successes` and `operator_failures` * `task.duration` have been moved under the API server. I've also added 2 metrics missing from the scheduler for consistency. I've made some other small changes to the `task-sdk` so that we don't have to remove the rest of the metrics. The temporality environment variable defaults to `cumulative` if unset. For the `task-sdk`, now it defaults to `delta` if unset. Also, the OTel SDK assigns a random and unique `service.instance.id` per process. Just for the `task-sdk`, this is set to a fixed value so that all task runners are treated as the same process. For backends that support delta values natively, no config changes are needed. But for backends that don't, such as prometheus, the user will need to extend the `otel-collector.yml` file so that it converts the delta values back to cumulative. The required config changes have been added to the documentation. They are simple and they could be just copy-pasted and they will also allow custom metrics to continue working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
