nagaran1 commented on issue #11225:
URL: https://github.com/apache/airflow/issues/11225#issuecomment-1833290322
> @sudarshan2906 - can you show the `celery_config_options` line in your
`airflow.cfg`? I'm doing something similar, but getting an error on the
scheduler saying:
>
> > airflow.exceptions.AirflowConfigException: The object could not be
loaded. Please check "celery_config_options" key in "celery" section. Current
value: "celery_config.MY_CELERY_CONFIG".
>
> My `celery_config` is in my AIRFLOW_HOME, and currently it's pretty basic:
>
> ```
> from airflow.config_templates.default_celery import DEFAULT_CELERY_CONFIG
>
> MY_CELERY_CONFIG = {
> **DEFAULT_CELERY_CONFIG
> }
> ```
>
> EDIT to add, an exception higher up in the stack trace: `[2021-08-19
09:19:06,303] {configuration.py:468} ERROR - No module named 'celery_config'`,
so it's just not finding the file.
>
> The specific line in my airflow.cfg: `celery_config_options =
celery_config.MY_CELERY_CONFIG`
>
> Thanks!
The custom celery config file should be kept under PYTHONPATH. Here are the
built-in PYTHONPATH.
1. The dags folder: It is configured with option dags_folder in section
[core].
2. The config folder: It is configured by setting AIRFLOW_HOME variable
({AIRFLOW_HOME}/config) by default.
3. The plugins Folder: It is configured with option plugins_folder in
section [core].
I have faced similar issue, but after placing the custom config file in
({AIRFLOW_HOME}/config) it worked fine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
