mik-laj commented on issue #11286: URL: https://github.com/apache/airflow/issues/11286#issuecomment-705213994
Overall, logger setup is one of the nightmares faced by people who want to start using Airflow. We have deffects in documentation, and debugging is also painful. I wanted to correct these situations I added a new field to the airflow info command. https://github.com/apache/airflow/pull/10771 This command will check which task handler you currently have. This is only available for Airflow 2.0, so if you want a similar effect for Airflow 1.10 you have to run python and then execute the following script. ``` >>> import airflow # Initialize airflow and logger configuration >>> import logging >>> logging.getLogger('airflow.task').handlers [<GCSTaskHandler (NOTSET)>] ``` If you see a message similar to the one below, you have a default configuration. `[<FileTaskHandler (NOTSET)>]` also see that the `airflow config `command is not working for us, so you can also use the workaround to see the current configuration. ``` from airflow import conf import pprint pprint.pprint(conf.getsection('core')) OrderedDict([('dags_folder', '/opt/airflow/dags/repo/dags'), ('base_log_folder', '/opt/airflow/logs'), ('remote_logging', False), ('remote_log_conn_id', ''), ('remote_base_log_folder', ''), ('encrypt_s3_logs', False), ('logging_level', 'INFO'), ('fab_logging_level', 'WARN'), ('logging_config_class', ''), ('colored_console_log', False), ``` I use environment variables to configure and everything works fine. ``` AIRFLOW_CONN_GOOGLE_CLOUD_DEFAULT = "google-cloud-platform://" // Configure remote logging // https://airflow.readthedocs.io/en/latest/logging-monitoring/logging-tasks.html#writing-logs-to-google-cloud-storage AIRFLOW__CORE__REMOTE_LOGGING = "True" AIRFLOW__CORE__REMOTE_BASE_LOG_FOLDER = "gs://${var.gcs_logging_bucket}/" AIRFLOW__CORE__REMOTE_LOG_CONN_ID = "google_cloud_default" ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
