o-nikolas commented on PR #68403:
URL: https://github.com/apache/airflow/pull/68403#issuecomment-6125046613
> Sorry, this got dropped in the mad prep for Airflow Summit last month.
No worries @ashb. This PR is just a typing change and thankfully doesn't
block functionality
> The main thing that I'm not sure, is that, given a worker belongs to
exactly one team, why does anything need to even know this on the worker side?
What would it do with that knowledge?
tl;dr the worker task process has no idea what team it's in unless the
server tells it (authoritatively).
In `supervise_task()` in-process (for execs like Celery/Edge/Local) or
`python -m airflow.sdk.execution_time.execute_workload` (in a pod for
Kubernetes). `ExecuteTask` carries no team field, so the only two inputs the
task process has are that workload and the `TIRunContext`. Worker-level config
doesn't always reach it (in case of Celery and Edge that have worker parents or
coordinators) and for something like LocalExecutor the team-scoped executors
are just instances inside the one scheduler process, so "a worker belongs to
exactly one team" doesn't hold there either. Essentially this is just a way to
deliver a reliable and authoritative indicator of what team the current task is
running for.
Some current uses:
- Metrics: we use it to add the team_name on task-side metrics
- Plugins: worker side plugins like macros use this to determine the right
team and access.
- Dag authors: `AIRFLOW_CTX_TEAM_NAME` / `context["dag_run"].team_name`
makes the team name available to dag authors
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
