wolfier opened a new issue #20969: URL: https://github.com/apache/airflow/issues/20969
### Apache Airflow version 2.2.3 (latest released) ### What happened Including an environment variable named `APP` with value other than the [default_app_name](https://airflow.apache.org/docs/apache-airflow/stable/configurations-ref.html#celery-app-name) causes the worker to terminate at start up. ``` [2022-01-18 03:14:00 +0000] [7575] [INFO] Starting gunicorn 20.1.0 [2022-01-18 03:14:00 +0000] [7575] [INFO] Listening at: http://0.0.0.0:8793 (7575) [2022-01-18 03:14:00 +0000] [7575] [INFO] Using worker: sync [2022-01-18 03:14:00 +0000] [7577] [INFO] Booting worker with pid: 7577 [2022-01-18 03:14:00 +0000] [7578] [INFO] Booting worker with pid: 7578 [2022-01-18 03:14:00 +0000] [7575] [INFO] Handling signal: term [2022-01-18 03:14:00,790] {cli_action_loggers.py:84} DEBUG - Calling callbacks: [] Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 53, in convert return find_app(value) File "/usr/local/lib/python3.9/site-packages/celery/app/utils.py", line 384, in find_app sym = symbol_by_name(app, imp=imp) File "/usr/local/lib/python3.9/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name module = imp(module_name, package=package, **kwargs) File "/usr/local/lib/python3.9/site-packages/celery/utils/imports.py", line 100, in import_from_cwd return imp(module, package=package) File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked ModuleNotFoundError: No module named 'datapipeline' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/airflow", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.9/site-packages/airflow/__main__.py", line 48, in main args.func(args) [2022-01-18 03:14:00 +0000] [7577] [INFO] Worker exiting (pid: 7577) File "/usr/local/lib/python3.9/site-packages/airflow/cli/cli_parser.py", line 48, in command return func(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/utils/cli.py", line 92, in wrapper [2022-01-18 03:14:00 +0000] [7578] [INFO] Worker exiting (pid: 7578) return f(*args, **kwargs) File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/celery_command.py", line 188, in worker _run_worker(options=options, skip_serve_logs=skip_serve_logs) File "/usr/local/lib/python3.9/site-packages/airflow/cli/commands/celery_command.py", line 94, in _run_worker celery_app.worker_main(options) File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 391, in worker_main self.start(argv=argv) File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 371, in start celery.main(args=argv, standalone_mode=False) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1052, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/local/lib/python3.9/site-packages/click/core.py", line 914, in make_context self.parse_args(ctx, args) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1615, in parse_args rest = super().parse_args(ctx, args) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1370, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 2347, in handle_parse_result value = self.process_value(ctx, value) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 2303, in process_value value = self.type_cast_value(ctx, value) File "/usr/local/lib/python3.9/site-packages/click/core.py", line 2291, in type_cast_value return convert(value) File "/usr/local/lib/python3.9/site-packages/click/types.py", line 75, in __call__ return self.convert(value, param, ctx) File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 60, in convert self.fail(UNABLE_TO_LOAD_APP_MODULE_NOT_FOUND.format(e.name)) File "/usr/local/lib/python3.9/site-packages/click/types.py", line 128, in fail raise BadParameter(message, ctx=ctx, param=param) click.exceptions.BadParameter: Unable to load celery application. The module datapipeline was not found. [2022-01-18 03:14:00,798] {base.py:314} DEBUG - No metrics to flush. Continuing. [2022-01-18 03:14:00,798] {base.py:322} DEBUG - No distributions to flush. Continuing. [2022-01-18 03:14:00,798] {base.py:332} DEBUG - No events to flush. Continuing. [2022-01-18 03:14:00,799] {settings.py:340} DEBUG - Disposing DB connection pool (PID 7475) [2022-01-18 03:14:00 +0000] [7575] [INFO] Shutting down: Master ``` ### What you expected to happen I would imagine Celery recognizing only `CELERY_` prefixed environment variable. However, `APP` is a global option. ### How to reproduce 1. Export `APP` environment variable with value other than the default app name 2. Start a Celery worker ### Operating System N/A ### Versions of Apache Airflow Providers _No response_ ### Deployment Astronomer ### Deployment details _No response_ ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
