vincbeck commented on code in PR #75998:
URL: https://github.com/apache/airflow/pull/75998#discussion_r4428617862
##########
task-sdk/src/airflow/sdk/execution_time/context.py:
##########
@@ -205,6 +206,15 @@ async def _amask_connection_secrets(conn: Connection) ->
None:
"_preset_connections", default=None
)
+# A plain global rather than a ContextVar, so threads started by task code see
it too.
+_current_team_name: str | None = None
Review Comment:
Threads see the team but child processes don't. `PythonVirtualenvOperator`
and `ExternalPythonOperator` run the callable in a new interpreter that only
calls `reinit_supervisor_comms()`, so `_current_team_name` is `None` there.
Example: a team_a venv task calls `Variable.get("my_var")` with both
`AIRFLOW_VAR_MY_VAR` and `AIRFLOW_VAR__TEAM_A___MY_VAR` set, and gets the
global value. Fix: also put the team in an env var the child inherits (like
`__AIRFLOW_SUPERVISOR_FD`) and read it from there.
##########
task-sdk/src/airflow/sdk/execution_time/context.py:
##########
@@ -529,7 +554,7 @@ def _set_variable(key: str, value: Any, description: str |
None = None, serializ
SUPERVISOR_COMMS.send(PutVariable(key=key, value=value,
description=description))
# Invalidate cache after setting the variable
- SecretCache.invalidate_variable(key)
+ SecretCache.invalidate_variable(key, team_name=_current_team_name)
Review Comment:
The write-conflict check at the top of `_set_variable` (and
`_async_set_variable`) still calls `secrets_backend.get_variable(key=key)`
without the team. Example: a team_a task sets `my_var` while only
`AIRFLOW_VAR__TEAM_A___MY_VAR` exists. Reads return the env value, but no
warning is logged. With only the global env var set, it warns about a conflict
that doesn't apply. Fix: use `call_secrets_backend_method` there too.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
