seanghaeli opened a new issue, #76102:
URL: https://github.com/apache/airflow/issues/76102
### What happened
In multi-team mode, a callback run by an executor (for example a deadline
callback) that calls `Variable.get("my_var")` or `Connection.get("my_conn")`
gets the global `AIRFLOW_VAR_MY_VAR` / `AIRFLOW_CONN_MY_CONN`, even when its
team has `AIRFLOW_VAR__<TEAM>___MY_VAR` / `AIRFLOW_CONN__<TEAM>___MY_CONN` set.
#75998 fixes this for tasks and triggers by setting the current team at
startup (`task_runner.startup` and `TriggerRunner.init_comms`). Executor
callbacks run in their own subprocess
(`task-sdk/src/airflow/sdk/execution_time/callback_supervisor.py`), which never
sets it, so their lookups still reach the secrets backends without a team.
### What you think should happen instead
A callback for a team's Dag should look up secrets for that team, the same
as the team's tasks and triggers do.
### How to reproduce
1. Enable multi-team and give a Dag bundle to `team_a`.
2. Set `AIRFLOW_VAR_MY_VAR=global` and `AIRFLOW_VAR__TEAM_A___MY_VAR=team_a`
on the workers.
3. In a `team_a` Dag, add a deadline with an executor callback that logs
`Variable.get("my_var")`.
4. Let the deadline be missed. The callback logs `global`.
### Proposed fix
- Add the team to the callback workload (`ExecuteCallback`).
`ExecuteCallback.make` already has the Dag run, so it can pass
`dag_run.team_name`.
- Call `set_current_team_name` in the callback subprocess before the
callback runs.
- Add a test that a team callback's `Variable.get` returns the team's value
over the global one.
Depends on #75998, which adds `set_current_team_name`. Related to the Dag
processor follow-up discussed there.
### Operating System / Versions
main (after #75998)
### Are you willing to submit PR?
Yes, I am willing to submit a PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
