Great job! Looking forward to the upcoming release :) On Sat, Feb 1, 2025 at 5:40 PM Pierre Jeambrun <pierrejb...@gmail.com> wrote:
> Amazing news! Well done everybody :) > > On Sat, Feb 1, 2025 at 2:25 PM Buğra Öztürk <ozturkbugr...@gmail.com> > wrote: > > > Awesome news! It feels amazing to be part of this great community! > > Kudos all for the immense progress and enormous effort! > > > > On Sat, Feb 1, 2025 at 12:32 PM Shubham Raj <shubhamraj....@gmail.com> > > wrote: > > > > > Woooww!! It’s really great news. > > > > > > On Sat, 1 Feb 2025 at 16:46, Amogh Desai <amoghdesai....@gmail.com> > > wrote: > > > > > > > Really good news! > > > > > > > > The timelines seemed very unrealistic when AF3 was announced during > > > Airflow > > > > Summit 2024, > > > > but the community decked all hands together and helped get to this > > stage, > > > > and that > > > > too with really good quality levels. > > > > > > > > Exciting times ahead for Airflow, and I am really proud to be part of > > > this > > > > amazing community and > > > > collaborating on Airflow 3! > > > > > > > > Thanks & Regards, > > > > Amogh Desai > > > > > > > > > > > > On Sat, Feb 1, 2025 at 1:31 PM Jarek Potiuk <ja...@potiuk.com> > wrote: > > > > > > > > > Woooho! > > > > > > > > > > On Sat, Feb 1, 2025 at 2:21 AM Vikram Koka > > > <vik...@astronomer.io.invalid > > > > > > > > > > wrote: > > > > > > > > > > > This is fantastic! Congratulations to everyone on this huge > > > milestone. > > > > > > Thank you all for your hard work in getting us to this point. I > am > > > > truly > > > > > > proud to be a part of this team. > > > > > > > > > > > > Best regards, > > > > > > Vikram > > > > > > > > > > > > > > > > > > On Fri, Jan 31, 2025 at 5:17 PM Jed Cunningham < > > > > jedcunning...@apache.org > > > > > > > > > > > > wrote: > > > > > > > > > > > > > Dear Airflow Community, > > > > > > > > > > > > > > I am thrilled to announce the availability of Apache Airflow > > > > > 3.0.0.alpha1 > > > > > > > for testing! Airflow 3.0 marks a significant milestone as the > > first > > > > > major > > > > > > > release in over four years, introducing improvements that > enhance > > > > user > > > > > > > experience, task execution, and system scalability. > > > > > > > > > > > > > > First, a few caveats: > > > > > > > > > > > > > > This is an alpha release, so do not run it in production. It > may > > > > > contain > > > > > > > significant issues, and you will likely need to reset your > > database > > > > > > between > > > > > > > this and subsequent alpha, beta or release candidate versions. > > > > > (Consider > > > > > > > yourself warned!) > > > > > > > > > > > > > > This release is intended for Airflow developers and early > > adopters > > > to > > > > > > test > > > > > > > the build and start preparing for Airflow 3.0.0. This is not an > > > > > official > > > > > > > release—that will happen when we create a release candidate and > > > hold > > > > a > > > > > > > vote. The expected timeline for the first release candidate is > > the > > > > week > > > > > > of > > > > > > > 2025-03-31, but we encourage early feedback to help stabilize > the > > > > > > release. > > > > > > > > > > > > > > What's new in 3.0.0.alpha1? > > > > > > > > > > > > > > Airflow 3.0.0 introduces significant enhancements and breaking > > > > changes. > > > > > > > > > > > > > > Notable Features > > > > > > > > > > > > > > DAG versioning & Bundles > > > > > > > Airflow now tracks DAG versions, offering better visibility > into > > > > > > historical > > > > > > > DAG changes and execution states. The introduction of DAG > Bundles > > > > > ensures > > > > > > > tasks run with the correct code version, even as DAGs evolve. > > > > > > > > > > > > > > Modern Web Application > > > > > > > The UI has been rebuilt using React and a complete API-driven > > > > > structure, > > > > > > > improving maintainability and extensibility. It includes a new > > > > > > > component-based design system and an enhanced information > > > > > architecture. A > > > > > > > new React-based plugin system supports custom widgets, improved > > > > > workflow > > > > > > > visibility, and integration with external tools. > > > > > > > > > > > > > > Task Execution Interface > > > > > > > Airflow 3.0 adopts a client / server architecture, decoupling > > task > > > > > > > execution from the internal meta-database via API-based > > > interaction. > > > > > This > > > > > > > allows for remote execution across networks, multi-language > > > support, > > > > > > > enhanced security, and better dependency management. The Edge > > > > Executor > > > > > > > further enables seamless remote task execution without direct > > > > database > > > > > > > connections. > > > > > > > > > > > > > > Data Assets & Asset-Centric Syntax > > > > > > > Airflow 3.0 enhances dataset management by introducing Data > > Assets, > > > > > > > expanding beyond tables and files to include ML models and > more. > > > > Assets > > > > > > can > > > > > > > be explicitly defined using the @asset decorator, simplifying > > > > tracking > > > > > > and > > > > > > > dependencies. > > > > > > > > > > > > > > External Event-Driven Scheduling > > > > > > > > > > > > > > Airflow now supports event-driven DAG triggers from external > > > sources > > > > > like > > > > > > > message queues. This builds upon dataset scheduling and > enhances > > > > > > > integration with the external data ecosystem. > > > > > > > > > > > > > > For a more comprehensive list of new features, please see the > > > > > 3.0.0apha1 > > > > > > > release notes: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/airflow/blob/3.0.0a1/RELEASE_NOTES.rst#airflow-300alpha1-2025-01-31 > > > > > > > > > > > > > > For a list of the breaking changes, please see visit: > > > > > > > https://cwiki.apache.org/confluence/x/9pCMEw > > > > > > > > > > > > > > Known issues in 3.0.0.alpha1: > > > > > > > > > > > > > > - > > > > > > > > > > > > > > AIP-72 - Task Execution Interface > > > > > > > - > > > > > > > > > > > > > > DB isolation in tasks is not enforced yet > > > > > > > - > > > > > > > > > > > > > > KubernetesExecutor does not yet use the TaskExecution > > > Interface > > > > > > > - > > > > > > > > > > > > > > The following does not work on LocalExecutor or > > > CeleryExecutor: > > > > > > > Dynamic Task Mapping, Skip based tasks (e.g. branch or > skip > > > > > > > operators), and > > > > > > > Task callbacks. > > > > > > > - > > > > > > > > > > > > > > OpenLineage listeners are not yet functional in this > alpha > > > > > release. > > > > > > > - > > > > > > > > > > > > > > AIP-38 - Modern Web Application > > > > > > > - > > > > > > > > > > > > > > The new UI has limited functionality at this time and is > > > still > > > > > > being > > > > > > > enhanced until GA. However, feedback on the UX flow is > > > > > > > appreciated at this > > > > > > > time. > > > > > > > - > > > > > > > > > > > > > > The underlying FastAPI API server, including the new UI > and > > > > > public > > > > > > > API, does not yet have auth. > > > > > > > - > > > > > > > > > > > > > > UI may be slow if you have a lot of DAGs or very big DAGs > > > > > > > - > > > > > > > > > > > > > > Notable areas of functionality that are not yet > functional > > in > > > > the > > > > > > UI: > > > > > > > DAG Versioning, Data Assets, Backfills, and the DAG Gantt > > > View. > > > > > > > - > > > > > > > > > > > > > > AIP-63 - DAG Versioning > > > > > > > - > > > > > > > > > > > > > > Callbacks are not yet working on the DAG Processor (e.g. > > DAG > > > > > level > > > > > > > callbacks) > > > > > > > - > > > > > > > > > > > > > > The concept of a separate `--subdir` in the DAG processor > > is > > > > > being > > > > > > > replaced with separate DAG bundles, but currently the DAG > > > > > > > processor always > > > > > > > parses all DAG bundles. > > > > > > > - > > > > > > > > > > > > > > AIP-82 - Event Based Scheduling > > > > > > > - > > > > > > > > > > > > > > Scheduling using some triggers, for example file-based, > can > > > > lead > > > > > to > > > > > > > duplicate DAG runs. > > > > > > > > > > > > > > > > > > > > > Where to get it? > > > > > > > > > > > > > > The alpha snapshot is available at: > > > > > > > https://dist.apache.org/repos/dist/dev/airflow/3.0.0a1/ > > > > > > > > > > > > > > - > > > > > > > > > > > > > > apache-airflow-3.0.0a1-bin.tar.gz: Binary Python "sdist" > > > snapshot. > > > > > > > - > > > > > > > > > > > > > > apache_airflow-3.0.0a1-py3-none-any.whl: Binary Python wheel > > > > > snapshot. > > > > > > > > > > > > > > > > > > > > > This snapshot has not been published to PyPI. > > > > > > > > > > > > > > Also present are alpha releases for apache-airflow-task-sdk, > > > > > > > apache-airflow-providers-standard, > apache-airflow-providers-fab, > > > and > > > > > > > apache-airflow-providers-celery, which you will also need for > the > > > > > 3.0.0a1 > > > > > > > to work. These have also not been published to PyPI. > > > > > > > > > > > > > > Public Keys & Verification > > > > > > > > > > > > > > Public keys for verification are available at: > > > > > > > https://www.apache.org/dist/airflow/KEYS > > > > > > > > > > > > > > Instructions: > > > > > > > > > > > > > > You can build a virtualenv that installs this alpha, and other > > > > required > > > > > > > packages (e.g. task sdk), like this: > > > > > > > > > > > > > > ``` > > > > > > > > > > > > > > uv venv > > > > > > > > > > > > > > uv pip install --find-links > > > > > > > https://dist.apache.org/repos/dist/dev/airflow/3.0.0a1/ > > > > > > > apache-airflow==3.0.0a1 apache-airflow-task-sdk==1.0.0a1 > > > > > > > apache-airflow-providers-standard==0.1.0a1 > > > > > > > apache-airflow-providers-fab==2.0.0a1 > > > > > > > apache-airflow-providers-celery==3.11.0a1 > > > > > > > > > > > > > > ``` > > > > > > > > > > > > > > Below are some of the changes you’ll need to consider in order > to > > > run > > > > > > this > > > > > > > Alpha release: > > > > > > > > > > > > > > - > > > > > > > > > > > > > > The standalone DAG processor is now required, and can be > > started > > > > > with > > > > > > > `airflow dag-processor`. > > > > > > > - > > > > > > > > > > > > > > The new UI and public API is started by running `airflow > > > > > fastapi-api`, > > > > > > > and the UI is available on port `9091` and path `/webapp`. > > > > > > > - > > > > > > > > > > > > > > Depending on your deployment setup, you may need to set the > > > > > `[workers] > > > > > > > execution_api_server_url` config option. This defaults to ` > > > > > > > http://localhost:9091/execution/` > <http://localhost:9091/execution/> > > <http://localhost:9091/execution/> > > > <http://localhost:9091/execution/> > > > > <http://localhost:9091/execution/> > > > > > <http://localhost:9091/execution/> < > http://localhost:9091/execution/ > > > > > > > > > <http://localhost:9091/execution/>. > > > > > > > > > > > > > > > > > > > > > Get Involved > > > > > > > > > > > > > > We encourage the community to test this release and report any > > > issues > > > > > or > > > > > > > feedback. Your contributions help us ensure a stable and > reliable > > > > > Airflow > > > > > > > 3.0.0 release. > > > > > > > > > > > > > > A huge thank you to all the contributors who have worked on > this > > > > > > milestone > > > > > > > release! > > > > > > > > > > > > > > Best, > > > > > > > Utkarsh and Jed > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Bugra Ozturk > > >
