zachliu opened a new issue, #49919:
URL: https://github.com/apache/airflow/issues/49919
### Apache Airflow version
3.0.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
When executing the cli `airflow sync-perm --include-dags -v`, I got
`KeyError: <Encoding.VAR: '__var'>` on some deprecated example DAGs. For
example, the `example_subdag_operator` DAG.
Full traceback:
```
INFO - [2025-04-24T17:20:08.865+0000] {serialized_dag.py:516} DEBUG -
Deserializing DAG: example_subdag_operator:
INFO - [2025-04-24T17:20:08.870+0000] {cli_action_loggers.py:97} DEBUG -
Calling callbacks: []:
INFO - Traceback (most recent call last)::
INFO - File "/usr/local/airflow/.local/bin/airflow", line 8, in <module>:
INFO - sys.exit(main()):
INFO - ^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py",
line 55, in main:
INFO - args.func(args):
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py",
line 48, in command:
INFO - return func(*args, **kwargs):
INFO - ^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py",
line 111, in wrapper:
INFO - return f(*args, **kwargs):
INFO - ^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py",
line 55, in wrapped_function:
INFO - return func(*args, **kwargs):
INFO - ^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/cli_commands/sync_perm_command.py",
line 39, in sync_perm:
INFO - appbuilder.sm.create_dag_specific_permissions():
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py",
line 920, in create_dag_specific_permissions:
INFO - dagbag.collect_dags_from_db():
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/models/dagbag.py",
line 622, in collect_dags_from_db:
INFO - self.dags = SerializedDagModel.read_all_dags():
INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py",
line 101, in wrapper:
INFO - return func(*args, session=session, **kwargs):
INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/models/serialized_dag.py",
line 517, in read_all_dags:
INFO - dag = row.dag:
INFO - ^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/models/serialized_dag.py",
line 551, in dag:
INFO - return SerializedDAG.from_dict(data):
INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 1926, in from_dict:
INFO - return cls.deserialize_dag(serialized_obj["dag"]):
INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 1716, in deserialize_dag:
INFO - deser =
SerializedBaseOperator.deserialize_operator(obj[Encoding.VAR]):
INFO -
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 1553, in deserialize_operator:
INFO - cls.populate_operator(op, encoded_op):
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 1448, in populate_operator:
INFO - v = cls.deserialize(v):
INFO - ^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 1636, in deserialize:
INFO - return BaseSerialization.deserialize(encoded_var=encoded_var):
INFO - ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^:
INFO - File
"/usr/local/airflow/.local/lib/python3.12/site-packages/airflow/serialization/serialized_objects.py",
line 853, in deserialize:
INFO - var = encoded_var[Encoding.VAR]:
INFO - ~~~~~~~~~~~^^^^^^^^^^^^^^:
INFO - KeyError: <Encoding.VAR: '__var'>:
```
### What you think should happen instead?
The core part of the cli mostly isn't changed between 3.0.0 and 2.9.5:
* 3.0.0
https://github.com/apache/airflow/blob/40cb51d135094c6eca7c69e75fb192f9d1113bde/providers/fab/src/airflow/providers/fab/auth_manager/security_manager/override.py#L908-L932
* 2.9.5
https://github.com/apache/airflow/blob/b93c3db6b1641b0840bd15ac7d05bc58ff2cccbf/airflow/providers/fab/auth_manager/security_manager/override.py#L1060-L1084
I suspect this is a serde issue
### How to reproduce
The workaround is to execute this sql command in the database:
```sql
DELETE FROM dag WHERE dag_id like '%example%' AND dag_id NOT IN (SELECT
dag_id FROM serialized_dag)
```
To reproduce this issue, we just need a DAG that exists in the `dag` table
but not in the `serialized_dag` table—typically a deprecated example DAG.
### Operating System
ubuntu debian
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other Docker-based deployment
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
