eladkal commented on code in PR #38527: URL: https://github.com/apache/airflow/pull/38527#discussion_r1541365729
########## RELEASE_NOTES.rst: ########## @@ -21,6 +21,285 @@ .. towncrier release notes start +Airflow 2.9.0 (2024-04-04) +-------------------------- + +Significant Changes +^^^^^^^^^^^^^^^^^^^ + +Following Listener API methods are considered stable and can be used for production system (were experimental feature in older Airflow versions) (#36376): +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +Lifecycle events: + +- ``on_starting`` +- ``before_stopping`` + +DagRun State Change Events: + +- ``on_dag_run_running`` +- ``on_dag_run_success`` +- ``on_dag_run_failed`` + +TaskInstance State Change Events: + +- ``on_task_instance_running`` +- ``on_task_instance_success`` +- ``on_task_instance_failed`` + +Support for Microsoft SQL-Server for Airflow Meta Database has been removed (#36514) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +After `discussion <https://lists.apache.org/thread/r06j306hldg03g2my1pd4nyjxg78b3h4>`__ +and a `voting process <https://lists.apache.org/thread/pgcgmhf6560k8jbsmz8nlyoxosvltph2>`__, +the Airflow's PMC and Committers have reached a resolution to no longer maintain MsSQL as a supported Database Backend. + +As of Airflow 2.9.0 support of MsSQL has been removed for Airflow Database Backend. + +A migration script which can help migrating the database *before* upgrading to Airflow 2.9.0 is available in +`airflow-mssql-migration repo on Github <https://github.com/apache/airflow-mssql-migration>`_. +Note that the migration script is provided without support and warranty. + +This does not affect the existing provider packages (operators and hooks), DAGs can still access and process data from MsSQL. + +Dataset URIs are now validated on input (#37005) +"""""""""""""""""""""""""""""""""""""""""""""""" + +Datasets must use a URI that conform to rules laid down in AIP-60, and the value +will be automatically normalized when the DAG file is parsed. See +`documentation on Datasets <https://airflow.apache.org/docs/apache-airflow/stable/authoring-and-scheduling/datasets.html>`_ for +a more detailed description on the rules. + +You may need to change your Dataset identifiers if they look like a URI, but are +used in a less mainstream way, such as relying on the URI's auth section, or +have a case-sensitive protocol name. + +The method ``get_permitted_menu_items`` in ``BaseAuthManager`` has been renamed ``filter_permitted_menu_items`` (#37627) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" + +Add REST API actions to Audit Log events (#37734) +""""""""""""""""""""""""""""""""""""""""""""""""" + +The Audit Log ``event`` name for REST API events will be prepended with ``api.`` or ``ui.``, depending on if it came from the Airflow UI or externally. + +Airflow 2.9.0 is the first release that officially supports Python 3.12 (#38025) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +There are a few caveats though: + +* Pendulum2 does not support Python 3.12. For Python 3.12 you need to use + `Pendulum 3 <https://pendulum.eustace.io/blog/announcing-pendulum-3-0-0.html>`_ + +* Minimum SQLAlchemy version supported when Pandas is installed for Python 3.12 is ``1.4.36`` released in + April 2022. Airflow 2.9.0 increases the minimum supported version of SQLAlchemy to ``1.4.36`` for all + Python versions. + +Not all Providers support Python 3.12. At the initial release of Airflow 2.9.0 the following providers +are released without support for Python 3.12: + + * ``apache.beam`` - pending on `Apache Beam support for 3.12 <https://github.com/apache/beam/issues/29149>`_ + * ``papermill`` - pending on Releasing Python 3.12 compatible papermill client version + `including this merged issue <https://github.com/nteract/papermill/pull/771>`_ + +Prevent large string objects from being stored in the Rendered Template Fields (#38094) +""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +There's now a limit to the length of data that can be stored in the Rendered Template Fields. +The limit is set to 4096 characters. If the data exceeds this limit, it will be truncated. You can change this limit +by setting the ``[core]max_template_field_length`` configuration option in your airflow config. + +Change xcom table column value type to longblob for MySQL backend (#38401) +"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" +Xcom table column ``value`` type has changed from ``blob`` to ``longblob``. This will allow you to store relatively big data in Xcom but process can take a significant amount of time if you have a lot of large data stored in Xcom. + +To downgrade from revision: ``b4078ac230a1``, ensure that you don't have Xcom values larger than 65,535 bytes. Otherwise, you'll need to clean those rows or run ``airflow db clean xcom`` to clean the Xcom table. + +New Features +"""""""""""" +- Allow users to write dag_id and task_id in their national characters, added display name for dag / task (v2) (#38446) +- Prevent large objects from being stored in the RTIF (#38094) +- Use current time to calculate duration when end date is not present. (#38375) +- Add average duration mark line in task and dagrun duration charts. (#38214, #38434) +- Add button to manually create dataset events (#38305) +- Add ``Matomo`` as an option for analytics_tool. (#38221) +- Experimental: Support custom weight_rule implementation to calculate the TI priority_weight (#38222) +- Adding ability to automatically set DAG to off after X times it failed sequentially (#36935) +- Add dataset conditions to next run datasets modal (#38123) +- Add task log grouping to UI (#38021) +- Add dataset_expression to grid dag details (#38121) +- Introduce mechanism to support multiple executor configuration (#37635) +- Add color formatting for ANSI chars in logs from task executions (#37985) +- Add the dataset_expression as part of DagModel and DAGDetailSchema (#37826) +- Add TaskFail entries to Gantt chart (#37918) +- Allow longer rendered_map_index (#37798) +- Inherit the run_ordering from DatasetTriggeredTimetable for DatasetOrTimeSchedule (#37775) +- Implement AIP-60 Dataset URI formats (#37005) +- Introducing Logical Operators for dataset conditional logic (#37101) +- Add post endpoint for dataset events (#37570) +- Show custom instance names for a mapped task in UI (#36797) +- Add excluded/included events to get_event_logs api (#37641) +- Add datasets to dag graph (#37604) +- Show dataset events above task/run details in grid view (#37603) +- Introduce new config variable to control whether DAG processor outputs to stdout (#37439) +- Make Datasets ``hashable`` (#37465) +- Add conditional logic for dataset triggering (#37016) +- Implement task duration page in react. (#35863) +- Add ``queuedEvent`` endpoint to get/delete DatasetDagRunQueue (#37176) +- Support multiple XCom output in the BaseOperator (#37297) +- AIP-58: Add object storage backend for xcom (#37058) +- Introduce ``DatasetOrTimeSchedule`` (#36710) +- Add ``on_skipped_callback`` in to ``BaseOperator`` (#36374) Review Comment: ```suggestion - Add ``on_skipped_callback`` to ``BaseOperator`` (#36374) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
