jedcunningham commented on code in PR #39309: URL: https://github.com/apache/airflow/pull/39309#discussion_r1583339899
########## RELEASE_NOTES.rst: ########## @@ -21,6 +21,96 @@ .. towncrier release notes start +Airflow 2.9.1 (2024-05-03) +-------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +Rename the ``name`` attribute of the StackdriverTaskHandler to ``gcp_log_name`` to avoid name overriding by the the ``DictConfigurator`` (#38071). +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +Airflow relies on the ``logging.config.dictConfig`` (`documentation <https://docs.python.org/3/library/logging.config.html>`_) method +to `setup the logging stack <https://github.com/apache/airflow/blob/a58441ca1b263cae61a5bb653e6839f0dd29b08e/airflow/logging_config.py#L69>`_. +However, during this setup, it iterates through the handlers and +`explicitly sets their name <https://github.com/python/cpython/blob/2a4cbf17af19a01d942f9579342f77c39fbd23c4/Lib/logging/config.py#L578>`_: + +.. code-block:: python + + for name in sorted(handlers): + try: + handler = self.configure_handler(handlers[name]) + handler.name = name + handlers[name] = handler + except Exception as e: + # [...] + pass + +So, before this fix: + +#. You setup the remote logging through the environment variables ``AIRFLOW__LOGGING__REMOTE_LOGGING="true"`` and ``AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER="stackdriver://host/path"``. +#. Airflow instantiates a ``StackdriverTaskHandler`` with the name of ``"path"`` +#. **BUT** the ``dictConfig`` call overrides the name of the handler with the key of the handlers configuration (i.e. `task <https://github.com/apache/airflow/blob/a58441ca1b263cae61a5bb653e6839f0dd29b08e/airflow/config_templates/airflow_local_settings.py#L350>`_). +#. Hence, the next calls to the ``emit`` method of the handler will generate logs to the wrong destination (``task`` instead of ``path``). + +Changing the field, from ``name`` to ``gcp_log_name`` prevents the overriding from the ``dictConfig``. Review Comment: ```suggestion Stackdriver logging bugfix requires Google provider `10.17.0` or later (#38071) """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" If you use Stackdriver logging, you must use Google provider version `10.17.0` or later. Airflow `2.9.1` now passes `gcp_log_name` to the `StackdriverTaskHandler` instead of `name`, and this will fail on earlier provider versions. This fixes a bug where the log name configured in `[logging] remove_base_log_folder` was overridden when Airflow configured logging, resulting in task logs going to the wrong destination. ``` I think this is a better user facing message of why and what the user actually needs to do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
