DynamicAdsAmateur opened a new issue, #30855:
URL: https://github.com/apache/airflow/issues/30855
### Apache Airflow version
2.5.3
### What happened
Fresh Airflow 2.5.3 docker-compose installation (no k8s). CeleryExecutor
with (the only change in docker-compose):
AIRFLOW__CELERY__POOL: 'eventlet'
AIRFLOW__CELERY__WORKER_CONCURRENCY: '100'
(without these lines, meaning default settings - everything works properly)
Should I do some monkey patching?
`
[2023-04-24 22:19:42,485: WARNING/MainProcess] Dag 'NAME' not found in path
/opt/airflow/dags/NAME.py; trying path /opt/airflow/dags/NAME.py
[2023-04-24 22:19:42,486: INFO/MainProcess] Filling up the DagBag from
/opt/airflow/dags/NAME.py
[2023-04-24 22:19:42,493: ERROR/MainProcess] Failed to import:
/opt/airflow/dags/NAME.py
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/models/dagbag.py",
line 337, in parse
loader.exec_module(new_module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in
_call_with_frames_removed
File "/opt/airflow/dags/NAME.py", line 63, in <module>
s3 = boto3.resource('s3',
File
"/home/airflow/.local/lib/python3.10/site-packages/boto3/__init__.py", line
101, in resource
return _get_default_session().resource(*args, **kwargs)
File "/home/airflow/.local/lib/python3.10/site-packages/boto3/session.py",
line 446, in resource
client = self.client(
File "/home/airflow/.local/lib/python3.10/site-packages/boto3/session.py",
line 299, in client
return self._session.create_client(
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/session.py", line
976, in create_client
client = client_creator.create_client(
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/client.py", line
155, in create_client
client_args = self._get_client_args(
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/client.py", line
485, in _get_client_args
return args_creator.get_client_args(
File "/home/airflow/.local/lib/python3.10/site-packages/botocore/args.py",
line 129, in get_client_args
endpoint = endpoint_creator.create_endpoint(
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/endpoint.py", line
409, in create_endpoint
http_session = http_session_cls(
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/httpsession.py",
line 313, in __init__
self._manager = PoolManager(**self._get_pool_manager_kwargs())
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/httpsession.py",
line 331, in _get_pool_manager_kwargs
'ssl_context': self._get_ssl_context(),
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/httpsession.py",
line 340, in _get_ssl_context
return create_urllib3_context()
File
"/home/airflow/.local/lib/python3.10/site-packages/botocore/httpsession.py",
line 129, in create_urllib3_context
context.options |= options
File "/usr/local/lib/python3.10/ssl.py", line 620, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/local/lib/python3.10/ssl.py", line 620, in options
super(SSLContext, SSLContext).options.__set__(self, value)
File "/usr/local/lib/python3.10/ssl.py", line 620, in options
super(SSLContext, SSLContext).options.__set__(self, value)
[Previous line repeated 459 more times]
RecursionError: maximum recursion depth exceeded while calling a Python
object
[2023-04-24 22:19:42,497: ERROR/MainProcess]
[3afe76d4-26e5-439c-9bfe-b2ac20acc0d2] Failed to execute task Dag 'NAME' could
not be found; either it does not exist or it failed to parse..
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py",
line 130, in _execute_in_fork
args.func(args)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/cli_parser.py",
line 52, in command
return func(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line
108, in wrapper
return f(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/cli/commands/task_command.py",
line 379, in task_run
dag = get_dag(args.subdir, args.dag_id)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/cli.py", line
231, in get_dag
raise AirflowException(
airflow.exceptions.AirflowException: Dag 'NAME' could not be found; either
it does not exist or it failed to parse.
[2023-04-24 22:19:42,566: ERROR/MainProcess] Task
airflow.executors.celery_executor.execute_command[3afe76d4-26e5-439c-9bfe-b2ac20acc0d2]
raised unexpected: AirflowException('Celery command failed on host:
e64b4e680265 with celery_task_id 3afe76d4-26e5-439c-9bfe-b2ac20acc0d2')
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.10/site-packages/celery/app/trace.py", line
451, in trace_task
R = retval = fun(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/celery/app/trace.py", line
734, in __protected_call__
return self.run(*args, **kwargs)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py",
line 96, in execute_command
_execute_in_fork(command_to_exec, celery_task_id)
File
"/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/celery_executor.py",
line 111, in _execute_in_fork
raise AirflowException(msg)
airflow.exceptions.AirflowException: Celery command failed on host:
e64b4e680265 with celery_task_id 3afe76d4-26e5-439c-9bfe-b2ac20acc0d2
`
### What you think should happen instead
_No response_
### How to reproduce
Official docker-compose and following variables
AIRFLOW__CELERY__POOL: 'eventlet'
AIRFLOW__CELERY__WORKER_CONCURRENCY: '100'
### Operating System
Ubuntu
### Versions of Apache Airflow Providers
_No response_
### Deployment
Docker-Compose
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
