Amazing news! Kudos to everyone who contributed and made this happen! On Thu, Mar 20, 2025 at 9:52 PM Kaxil Naik <kaxiln...@gmail.com> wrote:
> Docker image is published, use the following: > > docker pull apache/airflow:3.0.0b4 > > On Fri, 21 Mar 2025 at 01:58, Vikram Koka <vik...@astronomer.io.invalid> > wrote: > > > Awesome! > > > > Thank you Kaxil and everyone who contributed to this release! > > > > Vikram > > > > On Thu, Mar 20, 2025 at 12:55 PM Jarek Potiuk <ja...@potiuk.com> wrote: > > > > > Wowowowowwowow > > > > > > On Thu, Mar 20, 2025 at 8:54 PM Kaxil Naik <kaxiln...@gmail.com> > wrote: > > > > > > > Dear Airflow Community, > > > > > > > > I am thrilled to announce the availability of Apache Airflow > > 3.0.0.beta4 > > > > for testing! > > > > > > > > Airflow 3.0 marks a significant milestone as the first major > > > > release in over four years, introducing improvements that enhance > user > > > > experience, task execution, and system scalability. > > > > > > > > First, a few caveats: > > > > > > > > This is a beta release, so do not run it in production. It may > contain > > > > significant issues, and you will likely need to reset your database > > > between > > > > this and subsequent release candidate versions. (Consider yourself > > > > warned!) > > > > > > > > This release is intended for Airflow developers only to test the > build > > > and > > > > start preparing for Airflow 3.0.0. This is not an official > release—that > > > > will happen when we create a release candidate and hold a vote. The > > > > expected timeline for the first release candidate is the week of > > > > 2025-03-31, but we encourage early feedback to help stabilize the > > > release. > > > > > > > > Where to get it? The beta snapshot is available at: > > > > https://dist.apache.org/repos/dist/dev/airflow/3.0.0b4 > > > > > > > > - apache-airflow-3.0.0b3-bin.tar.gz: Binary Python "sdist" > snapshot. > > > > - apache_airflow-3.0.0b3-py3-none-any.whl: Binary Python wheel > > > snapshot. > > > > > > > > This snapshot has also been published to PyPI at > > > > https://pypi.org/project/apache-airflow/3.0.0b4/ > > > > > > > > Also present are beta releases for various other packages which you > > might > > > > also need for 3.0.0b4 to work. These have also been published to > PyPI.: > > > > > > > > - apache-airflow-task-sdk > > > > - apache-airflow-providers-celery > > > > - apache-airflow-providers-cncf-kubernetes > > > > - apache-airflow-providers-common-compat > > > > - apache-airflow-providers-common-io > > > > - apache-airflow-providers-common-messaging > > > > - apache-airflow-providers-common-sql > > > > - apache-airflow-providers-fab > > > > - apache-airflow-providers-openlineage > > > > - apache-airflow-providers-standard > > > > > > > > > > > > > > > > *Public Keys & Verification* > > > > > > > > Public keys for verification are available at: > > > > https://www.apache.org/dist/airflow/KEYS > > > > > > > > Jed has signed the artifacts. > > > > > > > > *Instructions*: > > > > > > > > You can build a virtualenv that installs this beta, and other > required > > > > packages (e.g. task sdk), like this: > > > > > > > > ``` > > > > uv venv > > > > > > > > uv pip install \ > > > > apache-airflow==3.0.0b4 \ > > > > apache-airflow-providers-celery==3.10.4b1 \ > > > > apache-airflow-providers-cncf-kubernetes==10.4.0b1 \ > > > > apache-airflow-providers-common-compat==1.6.0b1 \ > > > > apache-airflow-providers-common-io==1.5.2b1 \ > > > > apache-airflow-providers-common-sql==1.24.1b1 \ > > > > apache-airflow-providers-fab==2.0.0b1 \ > > > > apache-airflow-providers-openlineage==2.1.2b1 \ > > > > apache-airflow-providers-standard==0.2.0b1 \ > > > > apache-airflow-task-sdk==1.0.0b4 > > > > ``` > > > > > > > > The Dockerfiles will also be published in an hour. Once they are > done, > > I > > > > will reply to this email. > > > > > > > > > > > > *What's new in Airflow 3?* > > > > Airflow 3.0.0 introduces significant enhancements and breaking > changes. > > > > > > > > Notable Features > > > > > > > > DAG versioning & Bundles > > > > > > > > Airflow now tracks DAG versions, offering better visibility into > > > historical > > > > DAG changes and execution states. The introduction of DAG Bundles > > ensures > > > > tasks run with the correct code version, even as DAGs evolve. > > > > > > > > Modern Web Application > > > > > > > > The UI has been rebuilt using React and a complete API-driven > > structure, > > > > improving maintainability and extensibility. It includes a new > > > > component-based design system and an enhanced information > > architecture. A > > > > new React-based plugin system supports custom widgets, improved > > workflow > > > > visibility, and integration with external tools. > > > > > > > > Task Execution Interface > > > > > > > > Airflow 3.0 adopts a client / server architecture, decoupling task > > > > execution from the internal meta-database via API-based interaction. > > This > > > > allows for remote execution across networks, multi-language support, > > > > enhanced security, and better dependency management. The Edge > Executor > > > > further enables seamless remote task execution without direct > database > > > > connections. > > > > > > > > Data Assets & Asset-Centric Syntax > > > > > > > > Airflow 3.0 enhances dataset management by introducing Data Assets, > > > > expanding beyond tables and files to include ML models and more. > Assets > > > can > > > > be explicitly defined using the @asset decorator, simplifying > tracking > > > and > > > > dependencies. > > > > > > > > External Event-Driven Scheduling > > > > > > > > Airflow now supports event-driven DAG triggers from external sources > > like > > > > message queues and blob stores. This builds upon dataset scheduling > and > > > > enhances integration with the external data ecosystem. > > > > > > > > For a more comprehensive list of new features, please see the > > 3.0.0beta3 > > > > release notes: > > > > > > > > > > > > > > > > > > https://github.com/apache/airflow/blob/3.0.0b3/RELEASE_NOTES.rst#airflow-3-0-0b3-2025-03-13 > > > > > > > > For a list of the breaking changes, please see visit: > > > > https://cwiki.apache.org/confluence/x/9pCMEw > > > > > > > > Known limitations in 3.0.0.beta4: > > > > > > > > > > > > AIP-72 - Task Execution Interface > > > > - The following capabilities are not yet supported in the beta: > Task > > > > callbacks. > > > > > > > > AIP-38 - Modern Web Application > > > > - The new UI has richer functionality than beta3. However, > feedback > > on > > > > the UX flow is > > > > appreciated at this time. > > > > > > > > Notable areas that are usable but not 100% complete: Backfills, > > > > Connections, Assets, Dag Versioning. > > > > > > > > Below are some of the changes you’ll need to consider in order to run > > > this > > > > beta release: > > > > > > > > - The standalone DAG processor is now required, and can be started > > > with > > > > `airflow dag-processor`. > > > > - The new UI and public API is started by running `airflow > > > api-server`, > > > > and the UI is available on port `8080`. > > > > - Depending on your deployment setup, you may need to set the > > > `[workers] > > > > execution_api_server_url` config option. This defaults to ` > > > > http://localhost:8080/execution/` > <http://localhost:8080/execution/> > > <http://localhost:8080/execution/> <http://localhost:8080/execution/> > > > <http://localhost:8080/execution/> < > > > > http://localhost:8080/execution/> < > > > > http://localhost:8080/execution/>. > > > > > > > > Get Involved > > > > > > > > We encourage the community to test this release and report any issues > > or > > > > feedback. Your contributions help us ensure a stable and reliable > > Airflow > > > > 3.0.0 release. Please report issues using Github at > > > > https://github.com/apache/airflow/issues and mark that this is an > > issue > > > in > > > > 3.0.0. For an updated list of all known issues in the beta can also > be > > > > found in the above link with the label “affected_version:3.0.0beta”. > > > > > > > > A huge thank you to all the contributors who have worked on this > > > milestone > > > > release! > > > > > > > > Thanks, > > > > Kaxil > > > > > > > > > > -- Bugra Ozturk
