Yes. As of last week you can do `--use-airflow-version <PR_NUMBER>` in case anyone missed it :). Thank Lue Zhe You for this QOL improvement again :).
│ --use-airflow-version Use (reinstall at entry) Airflow version from PyPI. It can also be version (to install from PyPI), `none`, `wheel`, or `sdist` to install from `dist` folder, `owner/repo:branch` to install from GitHub repo, or a │ │ PR number (e.g., `57219`) to install from a pull request. Uses --mount-sources `remove` if not specified, but `providers-and-tests` or `tests` can be specified for `--mount-sources` when `--use-airflow-version` │ │ is used. On Wed, Nov 5, 2025 at 11:15 AM Jarek Potiuk <[email protected]> wrote: > As a small pointer - the logic is implemented here: > https://github.com/apache/airflow/blob/main/scripts/in_container/install_airflow_and_providers.py > : > > It also handles installing airflow from git url and pr # (added recently > by Liu Zhe You) and compiles assets in case we install from those. It has > now a good 1000+ lines of code to handle all cases :D :D. > > Good luck :) > > J. > > > On Wed, Nov 5, 2025 at 11:08 AM Jarek Potiuk <[email protected]> wrote: > >> BTW. We probably (and feel free :) could make it work when I think of it. >> currently --use-installation-from-dist works in two steps:. >> >> a) installs specified airflow (or airflow from dist if found in dist) in >> a single operation without constraints >> >> b) installs all other distributions without constraints >> >> >> We could probably make it three step and handle this edge case by >> installing airflow separately (with constraints) then providers and all >> other things >> >> I will leave it as exercise for the readers though - the logic there is >> already very complex and handles quite a few other cases and this one might >> break some of them. :) >> >> śr., 5 lis 2025, 10:57 użytkownik Jarek Potiuk <[email protected]> >> napisał: >> >>> The problem is `--use-distributions-from-dist`. The >>> `--use-airflow-version` and --use-distributions-from-dist does not use the >>> `--use-airflow-version` constraints because it will then limit whatever is >>> instlalled from dist (constraints from 3.1.1 have older version of edge >>> provider and if you try to install new edge provider from "dist" and >>> constraints pointing to the 3.1.1 provider they will conflict. >>> >>> This is our classic installation pattern where you install airflow + >>> whatever was released with constraints and **then** in a separate step you >>> install any updates. Using both constraints and something that deviates >>> from them in the same command is not going to work. >>> >>> So the right way of installing the provider for old versions is: >>> >>> * breeze start-airflow --use-airflow-version 3.1.1 -> uses constraints >>> * install new provider in terminal inside >>> * restart the components in other tmux terminals >>> >>> The --use-distributions-from-dist also works but by the fact that we >>> want to install all the distributions together (including one that deviates >>> from original constraints) - we cannot use constraints for it. >>> >>> J. >>> >>> >>> >>> On Tue, Nov 4, 2025 at 11:09 PM Jens Scheffler <[email protected]> >>> wrote: >>> >>>> Hi Jarek, >>>> >>>> what I actually did to be preceise: >>>> >>>> 1) Checked out to tag "providers-edge3/1.4.1rc2" >>>> >>>> 2) Build edge3 and docker on that tag (I need docker as well for an >>>> extended test...) via >>>> breeze release-management prepare-provider-distributions edge3 docker >>>> --skip-tag-check >>>> >>>> 3) Checkout (latest) main to ensure we use feshest breeze >>>> >>>> 4) Start system via breeze using: >>>> breeze start-airflow --python 3.12 --load-example-dags --backend >>>> postgres --executor EdgeExecutor --answer y --use-airflow-version 3. >>>> 1.1 --use-distributions-from-dist >>>> >>>> Re tested tonight again, results: >>>> >>>> * 3.0.6 - OK >>>> * 3.1.0 - not compatible with edge3 1.4.1 :-D >>>> * 3.1.1 - fails in structlog >>>> * 3.1.2rc1 - OK >>>> * 3.1.2rc2 - OK >>>> >>>> Made no manual install. But as 3.1.2 is OK I am okay anyway. >>>> >>>> On 11/4/25 15:36, Jarek Potiuk wrote: >>>> >> Don't know why I catched the "wrong" constraints" via installing with >>>> > breeze - might be constraints are not considered when installed a >>>> specific >>>> > airflow version and then providers are upgraded? >>>> > >>>> > It could be that later when you upgraded some providers, this also get >>>> > bumped or maybe some uv sync in the image happened. When I used >>>> > `--use-airflow-version 3.1.1` and **then** installed just >>>> > `apache-airflow-providers-edge3==1.4.1rc2` in breeze - everything >>>> worked >>>> > fine. It also can be you have some customisation scripts in breeze >>>> that do >>>> > some upgrades afterwards and you have not noticed. >>>> > >>>> > On Tue, Nov 4, 2025 at 3:34 PM Jarek Potiuk<[email protected]> wrote: >>>> > >>>> >> It's actually already fixed in main and cherry-picked to 3.1.2 - > >>>> so in >>>> >> 3.1.2 it's OK to have structlog 25.5.0. See >>>> >> https://github.com/apache/airflow/pull/57335 and it's been >>>> cherry-picked >>>> >> and merged as part ofhttps://github.com/apache/airflow/pull/57640 >>>> >> >>>> >> J, >>>> >> >>>> >> >>>> >> On Tue, Nov 4, 2025 at 3:04 PM Scheffler Jens (XC-AS/EAE-ADA-T) via >>>> dev < >>>> >> [email protected]> wrote: >>>> >> >>>> >>> Hi Amough, >>>> >>> >>>> >>> you are right but constraints in Airflow 3.1.2rc[1|2] both pin >>>> >>> structlog==25.5.0 >>>> >>> see >>>> >>> >>>> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.2rc2/constraints-3.10.txt >>>> >>> >>>> >>> So wanted to raise this if not detected that we need to fix >>>> constraints >>>> >>> before releasing 3.1.2 as well. >>>> >>> Don't know why I catched the "wrong" constraints" via installing >>>> with >>>> >>> breeze - might be constraints are not considered when installed a >>>> specific >>>> >>> airflow version and then providers are upgraded? >>>> >>> >>>> >>> Jens >>>> >>> >>>> >>> -----Original Message----- >>>> >>> From: Amogh Desai<[email protected]> >>>> >>> Sent: Dienstag, 4. November 2025 10:07 >>>> >>> To:[email protected] >>>> >>> Subject: Re: [VOTE] Airflow Providers prepared on November 03, 2025 >>>> >>> >>>> >>> Jens, we have constrained structlog to disallow 25.5.0: >>>> >>> >>>> >>> >>>> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.0/constraints-3.10.txt >>>> >>> >>>> >>> So, that should not be a problem for you :), maybe it was installed >>>> in >>>> >>> your environment. >>>> >>> >>>> >>> Thanks & Regards, >>>> >>> Amogh Desai >>>> >>> >>>> >>> >>>> >>> On Tue, Nov 4, 2025 at 3:59 AM Jens Scheffler<[email protected]> >>>> >>> wrote: >>>> >>> >>>> >>>> +1 (binding) - Checked SVN, Check in Docker, Reproducible package >>>> >>>> +build, >>>> >>>> Licenses, Signatures >>>> >>>> >>>> >>>> Installed the edge provider with current common-compat in Airflow >>>> >>>> 2.11.0, 3.0.6, 3.1.2rc1 and main and tested with EdgeExecutor and >>>> >>>> Integration Test Dag, all loking good. >>>> >>>> >>>> >>>> Will test detailled issue resolution (subpath etc) tomorrow but >>>> until >>>> >>>> now all good to release. Best release we had for Edge3 ever :-D >>>> >>>> >>>> >>>> Note: Was not able to install providers with Airflow 3.1.1 as >>>> >>>> structlog==25.5.0 installed by breeze has some incompatability and >>>> >>>> breze start failed with - assume this is not blocking providers >>>> >>>> release - assume this is because constraints are installed from >>>> main >>>> >>>> and not 3.1.1 in breeze: >>>> >>>> >>>> >>>> Traceback (most recent call last): >>>> >>>> File "/usr/python/bin/airflow", line 4, in <module> >>>> >>>> from airflow.__main__ import main >>>> >>>> File >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/__init__.py", >>>> >>>> line 79, in <module> >>>> >>>> settings.initialize() >>>> >>>> File >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/settings.py", >>>> >>>> line 698, in initialize >>>> >>>> LOGGING_CLASS_PATH = configure_logging() >>>> >>>> ^^^^^^^^^^^^^^^^^^^ >>>> >>>> File >>>> >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/logging_config.py", >>>> >>>> line 88, in configure_logging >>>> >>>> from airflow._shared.logging import configure_logging, >>>> >>>> init_log_folder, translate_config_values >>>> >>>> File >>>> >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/__in >>>> >>>> it__.py", >>>> >>>> >>>> >>>> line 27, in <module> >>>> >>>> from .structlog import configure_logging, init_log_file, >>>> >>>> init_log_folder >>>> >>>> File >>>> >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/stru >>>> >>>> ctlog.py", >>>> >>>> >>>> >>>> line 38, in <module> >>>> >>>> from .percent_formatter import PercentFormatRender >>>> >>>> File >>>> >>>> >>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/perc >>>> >>>> ent_formatter.py", >>>> >>>> >>>> >>>> line 29, in <module> >>>> >>>> from structlog.dev import ConsoleRenderer, Styles >>>> >>>> ImportError: cannot import name 'Styles' from 'structlog.dev' >>>> >>>> (/usr/python/lib/python3.12/site-packages/structlog/dev.py) >>>> >>>> >>>> >>>> Error: check_environment returned 1. Exiting. >>>> >>>> >>>> >>>> Call that I used for testing: breeze start-airflow --python 3.12 >>>> >>>> --load-example-dags --backend postgres --executor EdgeExecutor >>>> >>>> --answer y --use-airflow-version 3.1.1 >>>> --use-distributions-from-dist >>>> >>>> >>>> >>>> Tested also with Airflow 3.1.0 and saw as intended that edge3 >>>> provider >>>> >>>> 1.4.1rc2 can not installed with Airflow 3.1.0 as UI plugin >>>> incompatible. >>>> >>>> >>>> >>>> Using Python 3.12.12 environment at: /usr/python >>>> >>>> × No solution found when resolving dependencies: >>>> >>>> ╰─▶ Because only apache-airflow-providers-edge3==1.4.1 is >>>> available >>>> >>>> and apache-airflow-providers-edge3==1.4.1 depends on one of: >>>> >>>> apache-airflow>=2.10.0,<3.1.0 >>>> >>>> apache-airflow>3.1.0 >>>> >>>> we can conclude that all versions of >>>> >>>> apache-airflow-providers-edge3 depend on one of: >>>> >>>> apache-airflow>=2.10.0,<3.1.0 >>>> >>>> apache-airflow>3.1.0 >>>> >>>> >>>> >>>> And because you require apache-airflow==3.1.0 and >>>> >>>> apache-airflow-providers-edge3, we can conclude that your >>>> requirements >>>> >>>> are unsatisfiable. >>>> >>>> >>>> >>>> On 11/3/25 13:06, Elad Kalif wrote: >>>> >>>>> Hey all, >>>> >>>>> >>>> >>>>> I have just cut the ad-hoc wave Airflow Providers packages. This >>>> >>>>> email is calling a vote on the release, which will last for 72 >>>> hours >>>> >>>>> - which means that it will end on November 06, 2025 12:05 PM UTC >>>> and >>>> >>>>> until 3 binding +1 votes have been received. >>>> >>>>> >>>> >>>>> Consider this my (binding) +1. >>>> >>>>> >>>> >>>>> Airflow Providers are available at: >>>> >>>>> https://di/ >>>> >>>>> st.apache.org >>>> %2Frepos%2Fdist%2Fdev%2Fairflow%2Fproviders%2F&data=05% >>>> >>>>> 7C02%7CJens.Scheffler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1b819 >>>> >>>>> >>>> e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681481189% >>>> >>>>> >>>> 7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs >>>> >>>>> >>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata= >>>> >>>>> u%2BGuEPt6aFORkaSjATUG2TVoDJiUJmNDiTM9Txe4cQc%3D&reserved=0 >>>> >>>>> >>>> >>>>> *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the binary >>>> >>>>> Python "sdist" release - they are also official "sources" for >>>> the >>>> >>>> Provider >>>> >>>>> distributions. >>>> >>>>> >>>> >>>>> *apache_airflow_providers_<PROVIDER>-*.whl are the binary >>>> >>>>> Python "wheel" release. >>>> >>>>> >>>> >>>>> The test procedure for PMC members is described in >>>> >>>>> >>>> >>>> https://gith/ >>>> >>>> ub.com >>>> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE >>>> >>>> >>>> RS.md%23verify-the-release-candidate-by-pmc-members&data=05%7C02%7CJen >>>> >>>> s.Scheffler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e >>>> >>>> >>>> 1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681516854%7CUnknown%7CTW >>>> >>>> >>>> FpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs >>>> >>>> >>>> IkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6sgJ88Z0t%2F1Qnhxr >>>> >>>> wPTnej7SfgLNf6hI7qYxnJBl6RU%3D&reserved=0 >>>> >>>>> The test procedure for and Contributors who would like to test >>>> this >>>> >>>>> RC is described in: >>>> >>>>> >>>> >>>> https://gith/ >>>> >>>> ub.com >>>> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE >>>> >>>> >>>> RS.md%23verify-the-release-candidate-by-contributors&data=05%7C02%7CJe >>>> >>>> ns.Scheffler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51 >>>> >>>> >>>> e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681546634%7CUnknown%7CT >>>> >>>> >>>> WFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI >>>> >>>> >>>> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=j%2B4anx9P7T5uTdc >>>> >>>> omIOr%2BOr2%2FoDHQnntn1Pvhn33BHs%3D&reserved=0 >>>> >>>>> >>>> >>>>> Public keys are available at: >>>> >>>>> https://di/ >>>> >>>>> st.apache.org >>>> %2Frepos%2Fdist%2Frelease%2Fairflow%2FKEYS&data=05%7C02 >>>> >>>>> %7CJens.Scheffler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1b819e4d% >>>> >>>>> >>>> 7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681574562%7CUn >>>> >>>>> >>>> known%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAi >>>> >>>>> >>>> OiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=enB4 >>>> >>>>> BNSldvybyXLUzIJIz5jE2tCnZ904nXCrgzoZp%2Bg%3D&reserved=0 >>>> >>>>> >>>> >>>>> Please vote accordingly: >>>> >>>>> >>>> >>>>> [ ] +1 approve >>>> >>>>> [ ] +0 no opinion >>>> >>>>> [ ] -1 disapprove with the reason >>>> >>>>> >>>> >>>>> Only votes from PMC members are binding, but members of the >>>> >>>>> community are encouraged to test the release and vote with >>>> >>> "(non-binding)". >>>> >>>>> Please note that the version number excludes the 'rcX' string. >>>> >>>>> This will allow us to rename the artifact without modifying the >>>> >>>>> artifact checksums when we actually release. >>>> >>>>> >>>> >>>>> The status of testing the providers by the community is kept here: >>>> >>>>> https://gi/ >>>> >>>>> thub.com >>>> %2Fapache%2Fairflow%2Fissues%2F57739&data=05%7C02%7CJens.Sch >>>> >>>>> effler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e190 >>>> >>>>> >>>> 7c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681602068%7CUnknown%7CTWF >>>> >>>>> >>>> pbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI >>>> >>>>> >>>> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PPDrnZgPIfBJWRp >>>> >>>>> 5VyEq1Ij2NGJXs9qK3b9mJA7Cwjk%3D&reserved=0 >>>> >>>>> >>>> >>>>> The issue is also the easiest way to see important PRs included in >>>> >>>>> the RC candidates. >>>> >>>>> Detailed changelog for the providers will be published in the >>>> >>>> documentation >>>> >>>>> after the >>>> >>>>> RC candidates are released. >>>> >>>>> >>>> >>>>> You can find the RC packages in PyPI following these links: >>>> >>>>> >>>> >>>>> https://py/ >>>> >>>>> pi.org >>>> %2Fproject%2Fapache-airflow-providers-edge3%2F1.4.1rc2%2F&data >>>> >>>>> =05%7C02%7CJens.Scheffler%40de.bosch.com >>>> %7Cf6343926cc624857a11208de1 >>>> >>>>> >>>> b819e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681628 >>>> >>>>> >>>> 771%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw >>>> >>>>> >>>> MCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sd >>>> >>>>> ata=HWxblv2u1%2B6Pr3QrB5N1s9k7wnOcgLZNi7qsgW3t9i0%3D&reserved=0 >>>> >>>>> >>>> >>>>> Cheers, >>>> >>>>> Elad Kalif >>>> >>>>> >>>> >>>> >>>> --------------------------------------------------------------------- >>>> >>>> To unsubscribe, e-mail:[email protected] >>>> >>>> For additional commands, e-mail:[email protected] >>>> >>>> >>>> >>>> >>>> >>> >>>> --------------------------------------------------------------------- >>>> >>> To unsubscribe, e-mail:[email protected] >>>> >>> For additional commands, e-mail:[email protected] >>>> >>> >>> >>>
