ms32035 opened a new issue, #59302: URL: https://github.com/apache/airflow/issues/59302
### Apache Airflow version
3.1.4
### If "Other Airflow 2/3 version" selected, which one?
_No response_
### What happened?
We encounter lots of DagProcessor crashes with the following stack trace
```
│ dag-processor Traceback (most recent call last):
│
│ dag-processor File "/home/airflow/.local/bin/airflow", line 7, in
<module>
│
│ dag-processor sys.exit(main())
│
│ dag-processor ^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/__main__.py", line
55, in main
│
│ dag-processor args.func(args)
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/cli_config.py",
line 49, in command
│
│ dag-processor return func(*args, **kwargs)
│
│ dag-processor ^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/cli.py", line
114, in wrapper
│
│ dag-processor return f(*args, **kwargs)
│
│ dag-processor ^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/providers_configuration_loader.py",
line 54, in wrapped_function
│
│ dag-processor return func(*args, **kwargs)
│
│ dag-processor ^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/dag_processor_command.py",
line 53, in dag_processor
│
│ dag-processor run_command_with_daemon_option(
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/daemon_utils.py",
line 86, in run_command_with_daemon_option
│
│ dag-processor callback()
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/cli/commands/dag_processor_command.py",
line 56, in <lambda>
│
│ dag-processor callback=lambda: run_job(job=job_runner.job,
execute_callable=job_runner._execute),
│
│ dag-processor
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/session.py",
line 100, in wrapper
│
│ dag-processor return func(*args, session=session, **kwargs)
│
│ dag-processor ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/job.py", line
368, in run_job
│
│ dag-processor return execute_job(job,
execute_callable=execute_callable)
│
│ dag-processor
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/job.py", line
397, in execute_job
│
│ dag-processor ret = execute_callable()
│
│ dag-processor ^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/jobs/dag_processor_job_runner.py",
line 61, in _execute
│
│ dag-processor self.processor.run()
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 282, in run
│
│ dag-processor return self._run_parsing_loop()
│
│ dag-processor ^^^^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 369, in _run_parsing_loop
│
│ dag-processor self._service_processor_sockets(timeout=poll_time)
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/dag_processing/manager.py",
line 411, in _service_processor_sockets
│
│ dag-processor need_more = socket_handler(key.fileobj)
│
│ dag-processor ^^^^^^^^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1723, in cb
│
│ dag-processor gen.send(line)
│
│ dag-processor File
"/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/supervisor.py",
line 1803, in process_log_messages_from_subprocess
│
│ dag-processor level = NAME_TO_LEVEL[event.pop("level")]
│
│ dag-processor ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
│
│ dag-processor KeyError: 'deprecation warning'
```
`NAME_TO_LEVEL` mapping comes from `structlog`
### What you think should happen instead?
Add support for `deprecation warning` in `structlog`
OR
change the code to
```
level = NAME_TO_LEVEL.get(event.pop("level"), "error")
```
or some similar defensive handling
### How to reproduce
I don't know exactly which dag file is causing it and what are the conditions
### Operating System
Debian GNU/Linux 12 (bookworm)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
