Awesome! Thank you Jed and everyone who contributed to making this happen! Vikram
On Thu, Feb 20, 2025 at 2:40β―PM Jed Cunningham <jedcunning...@apache.org> wrote: > Dear Airflow Community, > > I am thrilled to announce the availability of *Apache Airflow 3.0.0.alpha4* > for testing! Airflow 3.0 marks a significant milestone as the first major > release in over four years, introducing improvements that enhance user > experience, task execution, and system scalability. > > First, a few caveats: > > - This is an *alpha release*, so do not run it in production. It may > contain significant issues, and you will likely need to reset your database > between this and subsequent alpha, beta, or release candidate versions. > (Consider yourself warned!) > - This release is intended for Airflow developers and early adopters to > test the build and start preparing for Airflow 3.0.0. *This is not an > official release*βthat will happen when we create a release candidate and > hold a vote. > - The expected timeline for the first release candidate is the week of > 2025-03-31, but we encourage early feedback to help stabilize the release. > > *What's New in Airflow 3?* > > Apache Airflow 3.0.0 introduces significant enhancements and breaking > changes. Notable Features include: > > > *DAG Versioning & Bundles* > - Airflow now tracks DAG versions, offering better visibility into > historical DAG changes and execution states. > - DAG Bundles ensure tasks run with the correct code version, even as DAGs > evolve. > > > *Modern Web Application* > - The UI has been rebuilt using React and a complete API-driven structure, > improving maintainability and extensibility. > - A new React-based plugin system supports custom widgets, improved > workflow visibility, and integration with external tools. > > > *Task Execution Interface* > - Airflow 3.0 adopts a client/server architecture, decoupling task > execution from the internal meta-database via API-based interaction. > - This allows for remote execution, multi-language support, enhanced > security, and better dependency management. > - The Edge Executor enables seamless remote task execution without direct > database connections. > > > *Data Assets & Asset-Centric Syntax* > - Airflow 3.0 enhances dataset management by introducing Data Assets, > expanding beyond tables and files to include ML models and more. > - Assets can be explicitly defined using the @asset decorator, simplifying > tracking and dependencies. > > > *External Event-Driven Scheduling* > - Airflow now supports event-driven DAG triggers from external sources like > message queues. > - This builds upon dataset scheduling and enhances integration with the > external data ecosystem. > > For a more comprehensive list of new features, see the 3.0.0.alpha4 release > notes > < > https://github.com/apache/airflow/blob/3.0.0a4/RELEASE_NOTES.rst#airflow-3-0-0alpha4-2025-02-20 > > > . > > For a list of the breaking changes, visit: Apache Airflow Breaking Changes > <https://cwiki.apache.org/confluence/x/9pCMEw> > > *Known Issues in 3.0.0.alpha4* > > *AIP-72 - Task Execution Interface* > - DB isolation in tasks is not enforced yet. > - The following do not work on LocalExecutor, CeleryExecutor, or > KubernetesExecutor: > - Skip-based tasks (e.g. branch or skip operators). > - Task callbacks. > > > *AIP-38 - Modern Web Application* > - The new UI has limited functionality at this time and is still being > enhanced until GA. > - FastAPI API server (including the new UI and public API) does not yet > have authentication. > - UI performance may degrade with a large number of DAGs or very big DAGs. > - Notable areas of functionality not yet implemented in the UI: DAG > Versioning, Data Assets, Backfills, DAG Gantt View. > > > *Where to Get It?* > The *alpha snapshot* is available at: > https://dist.apache.org/repos/dist/dev/airflow/3.0.0a4/ > > - Binary Python "sdist" snapshot: apache-airflow-3.0.0a4-bin.tar.gz > - Binary Python wheel snapshot: apache_airflow-3.0.0a4-py3-none-any.whl > > This snapshot has *not been published to PyPI*. > > Also present are *alpha releases* for: > > - apache-airflow-task-sdk > - apache-airflow-providers-standard > - apache-airflow-providers-fab > - apache-airflow-providers-celery > > These *are also not published to PyPI*. > > > *Public Keys & Verification* > Public keys for verification are available at: > https://downloads.apache.org/airflow/KEYS > > > *Installation Instructions* > To build a virtualenv that installs this alpha release and required > packages: > > uv venv > > uv pip install --find-links > https://dist.apache.org/repos/dist/dev/airflow/ > 3.0.0a4/ \ > apache-airflow==3.0.0a4 \ > apache-airflow-task-sdk==1.0.0a4 \ > apache-airflow-providers-standard==0.1.0a4 \ > apache-airflow-providers-fab==2.0.0a4 \ > apache-airflow-providers-celery==3.11.0a4 > > > *Changes Youβll Need to Consider* > - The standalone DAG processor is now required, start it with: airflow > dag-processor > - The new UI and public API start with: airflow fastapi-api > - UI is available at http://localhost:9091/webapp. > - Depending on your deployment setup, you may need to set the [workers] > execution_api_server_url config option. Default: > http://localhost:9091/execution/ > > > *Get Involved* > We encourage the community to test this release and report any issues or > feedback. Your contributions help us ensure a stable and reliable Airflow > 3.0.0 release. > > A huge *thank you* to all the contributors who have worked on this > milestone release! > > Thanks, > Jed >
