Dear Airflow Community, I am thrilled to announce the availability of *Apache Airflow 3.0.0.alpha4* for testing! Airflow 3.0 marks a significant milestone as the first major release in over four years, introducing improvements that enhance user experience, task execution, and system scalability.
First, a few caveats: - This is an *alpha release*, so do not run it in production. It may contain significant issues, and you will likely need to reset your database between this and subsequent alpha, beta, or release candidate versions. (Consider yourself warned!) - This release is intended for Airflow developers and early adopters to test the build and start preparing for Airflow 3.0.0. *This is not an official release*βthat will happen when we create a release candidate and hold a vote. - The expected timeline for the first release candidate is the week of 2025-03-31, but we encourage early feedback to help stabilize the release. *What's New in Airflow 3?* Apache Airflow 3.0.0 introduces significant enhancements and breaking changes. Notable Features include: *DAG Versioning & Bundles* - Airflow now tracks DAG versions, offering better visibility into historical DAG changes and execution states. - DAG Bundles ensure tasks run with the correct code version, even as DAGs evolve. *Modern Web Application* - The UI has been rebuilt using React and a complete API-driven structure, improving maintainability and extensibility. - A new React-based plugin system supports custom widgets, improved workflow visibility, and integration with external tools. *Task Execution Interface* - Airflow 3.0 adopts a client/server architecture, decoupling task execution from the internal meta-database via API-based interaction. - This allows for remote execution, multi-language support, enhanced security, and better dependency management. - The Edge Executor enables seamless remote task execution without direct database connections. *Data Assets & Asset-Centric Syntax* - Airflow 3.0 enhances dataset management by introducing Data Assets, expanding beyond tables and files to include ML models and more. - Assets can be explicitly defined using the @asset decorator, simplifying tracking and dependencies. *External Event-Driven Scheduling* - Airflow now supports event-driven DAG triggers from external sources like message queues. - This builds upon dataset scheduling and enhances integration with the external data ecosystem. For a more comprehensive list of new features, see the 3.0.0.alpha4 release notes <https://github.com/apache/airflow/blob/3.0.0a4/RELEASE_NOTES.rst#airflow-3-0-0alpha4-2025-02-20> . For a list of the breaking changes, visit: Apache Airflow Breaking Changes <https://cwiki.apache.org/confluence/x/9pCMEw> *Known Issues in 3.0.0.alpha4* *AIP-72 - Task Execution Interface* - DB isolation in tasks is not enforced yet. - The following do not work on LocalExecutor, CeleryExecutor, or KubernetesExecutor: - Skip-based tasks (e.g. branch or skip operators). - Task callbacks. *AIP-38 - Modern Web Application* - The new UI has limited functionality at this time and is still being enhanced until GA. - FastAPI API server (including the new UI and public API) does not yet have authentication. - UI performance may degrade with a large number of DAGs or very big DAGs. - Notable areas of functionality not yet implemented in the UI: DAG Versioning, Data Assets, Backfills, DAG Gantt View. *Where to Get It?* The *alpha snapshot* is available at: https://dist.apache.org/repos/dist/dev/airflow/3.0.0a4/ - Binary Python "sdist" snapshot: apache-airflow-3.0.0a4-bin.tar.gz - Binary Python wheel snapshot: apache_airflow-3.0.0a4-py3-none-any.whl This snapshot has *not been published to PyPI*. Also present are *alpha releases* for: - apache-airflow-task-sdk - apache-airflow-providers-standard - apache-airflow-providers-fab - apache-airflow-providers-celery These *are also not published to PyPI*. *Public Keys & Verification* Public keys for verification are available at: https://downloads.apache.org/airflow/KEYS *Installation Instructions* To build a virtualenv that installs this alpha release and required packages: uv venv uv pip install --find-links https://dist.apache.org/repos/dist/dev/airflow/ 3.0.0a4/ \ apache-airflow==3.0.0a4 \ apache-airflow-task-sdk==1.0.0a4 \ apache-airflow-providers-standard==0.1.0a4 \ apache-airflow-providers-fab==2.0.0a4 \ apache-airflow-providers-celery==3.11.0a4 *Changes Youβll Need to Consider* - The standalone DAG processor is now required, start it with: airflow dag-processor - The new UI and public API start with: airflow fastapi-api - UI is available at http://localhost:9091/webapp. - Depending on your deployment setup, you may need to set the [workers] execution_api_server_url config option. Default: http://localhost:9091/execution/ *Get Involved* We encourage the community to test this release and report any issues or feedback. Your contributions help us ensure a stable and reliable Airflow 3.0.0 release. A huge *thank you* to all the contributors who have worked on this milestone release! Thanks, Jed
