The big new part of AIP-72 (Task Execution Interface) that was included in this Alpha was mapped tasks β so you should now be able to use and test mapped tasks (and task groups) in Celery and Local executor.
-ash > On 6 Feb 2025, at 21:03, Vikram Koka <vik...@astronomer.io.INVALID> wrote: > > Awesome! > Great to see the consistent progress week over week. > > Thank you all for the wonderful teamwork, > > Vikram > > > On Thu, Feb 6, 2025 at 11:28β―AM Jarek Potiuk <ja...@potiuk.com> wrote: > >> wooohooo..... ! >> >> On Thu, Feb 6, 2025 at 8:20β―PM Utkarsh Sharma >> <utkarsh.sha...@astronomer.io.invalid> wrote: >> >>> Dear Airflow Community, >>> >>> I am thrilled to announce the availability of *Apache Airflow >> 3.0.0.alpha2* >>> for testing! Airflow 3.0 marks a significant milestone as the first major >>> release in over four years, introducing improvements that enhance user >>> experience, task execution, and system scalability. >>> First, a few caveats: >>> >>> - This is an *alpha release*, so do not run it in production. It may >>> contain significant issues, and you will likely need to reset your >>> database >>> between this and subsequent alpha, beta, or release candidate >>> versions. *(Consider >>> yourself warned!)* >>> - This release is intended for *Airflow developers and early adopters* >>> to test the build and start preparing for Airflow 3.0.0. This is *not >> an >>> official release*βthat will happen when we create a release candidate >>> and hold a vote. >>> - The *expected timeline* for the first release candidate is *the week >>> of 2025-03-31*, but we encourage early feedback to help stabilize the >>> release. >>> >>> What's New in 3.0.0.alpha2? >>> >>> Apache Airflow 3.0.0 introduces significant enhancements and breaking >>> changes. >>> *Notable Features* *DAG Versioning & Bundles* >>> >>> - Airflow now tracks *DAG versions*, offering better visibility into >>> historical DAG changes and execution states. >>> - *DAG Bundles* ensure tasks run with the correct code version, even >> as >>> DAGs evolve. >>> >>> *Modern Web Application* >>> >>> - The UI has been *rebuilt using React* and a complete *API-driven >>> structure*, improving maintainability and extensibility. >>> - A new *React-based plugin system* supports custom widgets, improved >>> workflow visibility, and integration with external tools. >>> >>> *Task Execution Interface* >>> >>> - Airflow 3.0 adopts a *client/server architecture*, decoupling task >>> execution from the internal meta-database via API-based interaction. >>> - This allows for *remote execution*, *multi-language support*, >> enhanced >>> security, and better dependency management. >>> - The *Edge Executor* enables seamless remote task execution without >>> direct database connections. >>> >>> *Data Assets & Asset-Centric Syntax* >>> >>> - Airflow 3.0 enhances dataset management by introducing *Data >> Assets*, >>> expanding beyond tables and files to include ML models and more. >>> - Assets can be explicitly defined using the @asset decorator, >>> simplifying tracking and dependencies. >>> >>> *External Event-Driven Scheduling* >>> >>> - Airflow now supports *event-driven DAG triggers* from external >> sources >>> like message queues. >>> - This builds upon dataset scheduling and enhances integration with >> the >>> external data ecosystem. >>> >>> For a more comprehensive list of new features, see the 3.0.0.alpha2 >> release >>> notes >>> < >>> >> https://github.com/apache/airflow/blob/3.0.0a2/RELEASE_NOTES.rst#airflow-300alpha2-2025-01-31 >>>> >>> . >>> >>> For a list of the breaking changes, visit: Apache Airflow Breaking >> Changes >>> <https://cwiki.apache.org/confluence/x/9pCMEw> >>> *Known Issues in 3.0.0.alpha2* *AIP-72 - Task Execution Interface* >>> >>> - *DB isolation in tasks* is not enforced yet. >>> - *KubernetesExecutor* does not yet use the Task Execution Interface. >>> - The following do not work on *LocalExecutor* or *CeleryExecutor*: >>> - Skip-based tasks (e.g. branch or skip operators). >>> - Task callbacks. >>> - *OpenLineage listeners* are not yet functional in this alpha >> release. >>> >>> *AIP-38 - Modern Web Application* >>> >>> - The new UI has *limited functionality* at this time and is still >> being >>> enhanced until GA. >>> - *FastAPI API server* (including the new UI and public API) does not >>> yet have authentication. >>> - UI performance may degrade with a *large number of DAGs* or *very >> big >>> DAGs*. >>> - Notable areas of functionality *not yet implemented* in the UI: >>> - DAG Versioning, Data Assets, Backfills, DAG Gantt View. >>> >>> *AIP-63 - DAG Versioning* >>> >>> - Callbacks are *not yet working* on the DAG Processor (e.g., >> DAG-level >>> callbacks). >>> - The concept of a separate --subdir in the DAG processor is being >>> replaced with *DAG Bundles*, but currently, the DAG processor always >>> parses all bundles. >>> >>> *AIP-82 - Event-Based Scheduling* >>> >>> - Some triggers (e.g., *file-based triggers*) may lead to *duplicate >> DAG >>> runs*. >>> >>> *Where to Get It?* >>> >>> The *alpha snapshot* is available at: Apache Airflow 3.0.0.alpha2 >>> Distribution <https://dist.apache.org/repos/dist/dev/airflow/3.0.0a2/> >>> >>> - *Binary Python "sdist" snapshot:* apache-airflow-3.0.0a2-bin.tar.gz >>> - *Binary Python wheel snapshot:* >>> apache_airflow-3.0.0a2-py3-none-any.whl >>> >>> This snapshot has *not been published to PyPI*. >>> >>> Also present are *alpha releases* for: >>> >>> - apache-airflow-task-sdk >>> - apache-airflow-providers-standard >>> - apache-airflow-providers-fab >>> - apache-airflow-providers-celery >>> >>> These *are also not published to PyPI*. >>> *Public Keys & Verification* >>> >>> Public keys for verification are available at: Apache Airflow KEYS >>> <https://www.apache.org/dist/airflow/KEYS> >>> *Installation Instructions* >>> >>> To build a virtualenv that installs this alpha release and required >>> packages: >>> >>> uv venv >>> >>> uv pip install --find-links >>> https://dist.apache.org/repos/dist/dev/airflow/3.0.0a2/ \ >>> apache-airflow==3.0.0a2 \ >>> apache-airflow-task-sdk==1.0.0a2 \ >>> apache-airflow-providers-standard==0.1.0a2 \ >>> apache-airflow-providers-fab==2.0.0a2 \ >>> apache-airflow-providers-celery==3.11.0a2 >>> >>> *Changes Youβll Need to Consider* >>> >>> - The *standalone DAG processor* is now required, start it with: >>> >>> airflow dag-processor >>> >>> - The new UI and public API start with: >>> >>> airflow fastapi-api >>> >>> UI is available at http://localhost:9091/webapp. >>> - Depending on your deployment setup, you may need to set the >> [workers] >>> execution_api_server_url config option. Default: >>> >>> http://localhost:9091/execution/ >>> >>> >>> *Get Involved* >>> >>> We encourage the community to *test this release* and report any issues >> or >>> feedback. Your contributions help us ensure a *stable and reliable >> Airflow >>> 3.0.0 release*. >>> >>> A huge *thank you* to all the contributors who have worked on this >>> milestone release! >>> >>> *Best,* >>> Utkarsh and Jed >>> >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@airflow.apache.org For additional commands, e-mail: dev-h...@airflow.apache.org
