venkatnpedada opened a new issue, #27498:
URL: https://github.com/apache/airflow/issues/27498
### Apache Airflow version
2.4.2
### What happened
Im trying to use custom logger to print the logs to console instead of the
local directory. Im suing official airflow helm chart
step 1: Create custom logger class to
${AIRFLOW_HOME}/config/airflow_local_settings.py by adding below in
airflowLocalSettings property
```
airflowLocalSettings: |-
from copy import deepcopy
from airflow.config_templates.airflow_local_settings import
DEFAULT_LOGGING_CONFIG
import sys
LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)
LOGGING_CONFIG["handlers"]["processor"] = {
"class": "logging.StreamHandler",
"formatter": "airflow",
"stream": sys.stdout,
}
```
step 2: Set logging_config_class property to
config.log_config.LOGGING_CONFIG in airflow.cfg by adding below in property
```
config:
logging:
logging_config_class: airflow_local_settings.LOGGING_CONFIG
```
While deploying the helm chart got below error from running migrations in
"wait-for-airflow-migrations" container
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/airflow/logging_config.py",
line 40, in configure_logging
logging_config = import_string(logging_class_path)
File
"/usr/local/lib/python3.7/dist-packages/airflow/utils/module_loading.py", line
32, in import_string
module = import_module(module_path)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'airflow_local_settings'
```
Suspecting the helm chart not mounted the airflowLocalSettings config on
this container and only mounted the actual airflow.cfg -
https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L138-L142
### What you think should happen instead
official airflow helm chart should support using the custom logging as
provided in the above without any issue. We might need to mount the
airflowLocalSettings config aswell in the init container
`wait-for-airflow-migrations`
https://github.com/apache/airflow/blob/main/chart/templates/scheduler/scheduler-deployment.yaml#L210-L215
```
{{- if .Values.airflowLocalSettings }}
- name: config
mountPath: {{ template "airflow_local_setting_path" . }}
subPath: airflow_local_settings.py
readOnly: true
{{- end }}
```
### How to reproduce
step 1: Create custom logger class to
${AIRFLOW_HOME}/config/airflow_local_settings.py by adding below in
airflowLocalSettings property
```
airflowLocalSettings: |-
from copy import deepcopy
from airflow.config_templates.airflow_local_settings import
DEFAULT_LOGGING_CONFIG
import sys
LOGGING_CONFIG = deepcopy(DEFAULT_LOGGING_CONFIG)
LOGGING_CONFIG["handlers"]["processor"] = {
"class": "logging.StreamHandler",
"formatter": "airflow",
"stream": sys.stdout,
}
```
step 2: Set logging_config_class property to
config.log_config.LOGGING_CONFIG in airflow.cfg by adding below in property
```
config:
logging:
logging_config_class: airflow_local_settings.LOGGING_CONFIG
```
While deploying the helm chart got below error from running migrations in
"wait-for-airflow-migrations" container
```
Traceback (most recent call last):
File "/usr/local/lib/python3.7/dist-packages/airflow/logging_config.py",
line 40, in configure_logging
logging_config = import_string(logging_class_path)
File
"/usr/local/lib/python3.7/dist-packages/airflow/utils/module_loading.py", line
32, in import_string
module = import_module(module_path)
File "/usr/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'airflow_local_settings'
```
### Operating System
Debian GNU/Linux 10 (buster)
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
