amordoch opened a new issue, #57426: URL: https://github.com/apache/airflow/issues/57426
### Apache Airflow Provider(s) standard ### Versions of Apache Airflow Providers apache-airflow-providers-standard==1.8.0 ### Apache Airflow version 3.1.0 ### Operating System Amazon Linux 2023 (arm64) ### Deployment Official Apache Airflow Helm Chart ### Deployment details OOTB Helm chart on EKS ### What happened Structlog 25.5.0 removed `Styles` from `structlog.dev`, which prevents virtualenv tasks from running if they have apache-airflow in their requirements: ```log INFO - Output: source=airflow.utils.process_utils loc=process_utils.py:192 INFO - Traceback (most recent call last): source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/venv-callh3vcy6hf/script.py", line 145, in <module> source=airflow.utils.process_utils loc=process_utils.py:196 INFO - arg_dict = pickle.load(file) source=airflow.utils.process_utils loc=process_utils.py:196 INFO - ^^^^^^^^^^^^^^^^^ source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/__init__.py", line 79, in <module> source=airflow.utils.process_utils loc=process_utils.py:196 INFO - settings.initialize() source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/settings.py", line 698, in initialize source=airflow.utils.process_utils loc=process_utils.py:196 INFO - LOGGING_CLASS_PATH = configure_logging() source=airflow.utils.process_utils loc=process_utils.py:196 INFO - ^^^^^^^^^^^^^^^^^^^ source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/logging_config.py", line 88, in configure_logging source=airflow.utils.process_utils loc=process_utils.py:196 INFO - from airflow._shared.logging import configure_logging, init_log_folder, translate_config_values source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/_shared/logging/__init__.py", line 27, in <module> source=airflow.utils.process_utils loc=process_utils.py:196 INFO - from .structlog import configure_logging, init_log_file, init_log_folder source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/_shared/logging/structlog.py", line 38, in <module> source=airflow.utils.process_utils loc=process_utils.py:196 INFO - from .percent_formatter import PercentFormatRender source=airflow.utils.process_utils loc=process_utils.py:196 INFO - File "/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/airflow/_shared/logging/percent_formatter.py", line 29, in <module> source=airflow.utils.process_utils loc=process_utils.py:196 INFO - from structlog.dev import ConsoleRenderer, Styles source=airflow.utils.process_utils loc=process_utils.py:196 INFO - ImportError: cannot import name 'Styles' from 'structlog.dev' (/tmp/.airflow-venv-cache/venv-cf3b86b0/lib/python3.12/site-packages/structlog/dev.py) source=airflow.utils.process_utils loc=process_utils.py:196 ERROR - Task failed with exception source=task loc=task_runner.py:994 CalledProcessError: Command '['/tmp/.airflow-venv-cache/venv-cf3b86b0/bin/python', '/tmp/venv-callh3vcy6hf/script.py', '/tmp/venv-callh3vcy6hf/script.in', '/tmp/venv-callh3vcy6hf/script.out', '/tmp/venv-callh3vcy6hf/string_args.txt', '/tmp/venv-callh3vcy6hf/termination.log', '/tmp/venv-callh3vcy6hf/airflow_context.json']' returned non-zero exit status 1. File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 920 in run File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/execution_time/task_runner.py", line 1307 in _execute_task File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 416 in wrapper File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/decorator.py", line 252 in execute File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 416 in wrapper File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 491 in execute File "/home/airflow/.local/lib/python3.12/site-packages/airflow/sdk/bases/operator.py", line 416 in wrapper File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 216 in execute File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 888 in execute_callable File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/standard/operators/python.py", line 578 in _execute_python_callable_in_subprocess File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/process_utils.py", line 177 in execute_in_subprocess File "/home/airflow/.local/lib/python3.12/site-packages/airflow/utils/process_utils.py", line 200 in execute_in_subprocess_with_kwargs ``` ### What you think should happen instead Airflow's pyproject.toml should specify structlog < 25.5.0, so that new venv's spun up by the virtual env operator don't run into this compatibility issue. ### How to reproduce 1. Create DAG with virtual env operator task and specify apache-airflow in the requirements 2. Execute DAG and observe this error ### Anything else _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
