+1 (binding)
* Checked
* SVN
* Docker
* reproducibility
* licences
* signatures
* checksums
Best,
Wei
> On Nov 5, 2025, at 6:20 PM, Jarek Potiuk <[email protected]> wrote:
>
> Yes. As of last week you can do `--use-airflow-version <PR_NUMBER>` in case
> anyone missed it :). Thank Lue Zhe You for this QOL improvement again :).
>
> │ --use-airflow-version Use
> (reinstall at entry) Airflow version from PyPI. It can also be version (to
> install from PyPI), `none`, `wheel`, or `sdist` to install from `dist`
> folder, `owner/repo:branch` to install from GitHub repo, or a
> │
> │
> PR number (e.g., `57219`) to install from a pull request. Uses
> --mount-sources `remove` if not specified, but `providers-and-tests` or
> `tests` can be specified for `--mount-sources` when `--use-airflow-version`
> │
> │
> is used.
>
> On Wed, Nov 5, 2025 at 11:15 AM Jarek Potiuk <[email protected]> wrote:
>
>> As a small pointer - the logic is implemented here:
>> https://github.com/apache/airflow/blob/main/scripts/in_container/install_airflow_and_providers.py
>> :
>>
>> It also handles installing airflow from git url and pr # (added recently
>> by Liu Zhe You) and compiles assets in case we install from those. It has
>> now a good 1000+ lines of code to handle all cases :D :D.
>>
>> Good luck :)
>>
>> J.
>>
>>
>> On Wed, Nov 5, 2025 at 11:08 AM Jarek Potiuk <[email protected]> wrote:
>>
>>> BTW. We probably (and feel free :) could make it work when I think of it.
>>> currently --use-installation-from-dist works in two steps:.
>>>
>>> a) installs specified airflow (or airflow from dist if found in dist) in
>>> a single operation without constraints
>>>
>>> b) installs all other distributions without constraints
>>>
>>>
>>> We could probably make it three step and handle this edge case by
>>> installing airflow separately (with constraints) then providers and all
>>> other things
>>>
>>> I will leave it as exercise for the readers though - the logic there is
>>> already very complex and handles quite a few other cases and this one might
>>> break some of them. :)
>>>
>>> śr., 5 lis 2025, 10:57 użytkownik Jarek Potiuk <[email protected]>
>>> napisał:
>>>
>>>> The problem is `--use-distributions-from-dist`. The
>>>> `--use-airflow-version` and --use-distributions-from-dist does not use the
>>>> `--use-airflow-version` constraints because it will then limit whatever is
>>>> instlalled from dist (constraints from 3.1.1 have older version of edge
>>>> provider and if you try to install new edge provider from "dist" and
>>>> constraints pointing to the 3.1.1 provider they will conflict.
>>>>
>>>> This is our classic installation pattern where you install airflow +
>>>> whatever was released with constraints and **then** in a separate step you
>>>> install any updates. Using both constraints and something that deviates
>>>> from them in the same command is not going to work.
>>>>
>>>> So the right way of installing the provider for old versions is:
>>>>
>>>> * breeze start-airflow --use-airflow-version 3.1.1 -> uses constraints
>>>> * install new provider in terminal inside
>>>> * restart the components in other tmux terminals
>>>>
>>>> The --use-distributions-from-dist also works but by the fact that we
>>>> want to install all the distributions together (including one that deviates
>>>> from original constraints) - we cannot use constraints for it.
>>>>
>>>> J.
>>>>
>>>>
>>>>
>>>> On Tue, Nov 4, 2025 at 11:09 PM Jens Scheffler <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi Jarek,
>>>>>
>>>>> what I actually did to be preceise:
>>>>>
>>>>> 1) Checked out to tag "providers-edge3/1.4.1rc2"
>>>>>
>>>>> 2) Build edge3 and docker on that tag (I need docker as well for an
>>>>> extended test...) via
>>>>> breeze release-management prepare-provider-distributions edge3 docker
>>>>> --skip-tag-check
>>>>>
>>>>> 3) Checkout (latest) main to ensure we use feshest breeze
>>>>>
>>>>> 4) Start system via breeze using:
>>>>> breeze start-airflow --python 3.12 --load-example-dags --backend
>>>>> postgres --executor EdgeExecutor --answer y --use-airflow-version 3.
>>>>> 1.1 --use-distributions-from-dist
>>>>>
>>>>> Re tested tonight again, results:
>>>>>
>>>>> * 3.0.6 - OK
>>>>> * 3.1.0 - not compatible with edge3 1.4.1 :-D
>>>>> * 3.1.1 - fails in structlog
>>>>> * 3.1.2rc1 - OK
>>>>> * 3.1.2rc2 - OK
>>>>>
>>>>> Made no manual install. But as 3.1.2 is OK I am okay anyway.
>>>>>
>>>>> On 11/4/25 15:36, Jarek Potiuk wrote:
>>>>>>> Don't know why I catched the "wrong" constraints" via installing with
>>>>>> breeze - might be constraints are not considered when installed a
>>>>> specific
>>>>>> airflow version and then providers are upgraded?
>>>>>>
>>>>>> It could be that later when you upgraded some providers, this also get
>>>>>> bumped or maybe some uv sync in the image happened. When I used
>>>>>> `--use-airflow-version 3.1.1` and **then** installed just
>>>>>> `apache-airflow-providers-edge3==1.4.1rc2` in breeze - everything
>>>>> worked
>>>>>> fine. It also can be you have some customisation scripts in breeze
>>>>> that do
>>>>>> some upgrades afterwards and you have not noticed.
>>>>>>
>>>>>> On Tue, Nov 4, 2025 at 3:34 PM Jarek Potiuk<[email protected]> wrote:
>>>>>>
>>>>>>> It's actually already fixed in main and cherry-picked to 3.1.2 - >
>>>>> so in
>>>>>>> 3.1.2 it's OK to have structlog 25.5.0. See
>>>>>>> https://github.com/apache/airflow/pull/57335 and it's been
>>>>> cherry-picked
>>>>>>> and merged as part ofhttps://github.com/apache/airflow/pull/57640
>>>>>>>
>>>>>>> J,
>>>>>>>
>>>>>>>
>>>>>>> On Tue, Nov 4, 2025 at 3:04 PM Scheffler Jens (XC-AS/EAE-ADA-T) via
>>>>> dev <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Hi Amough,
>>>>>>>>
>>>>>>>> you are right but constraints in Airflow 3.1.2rc[1|2] both pin
>>>>>>>> structlog==25.5.0
>>>>>>>> see
>>>>>>>>
>>>>> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.2rc2/constraints-3.10.txt
>>>>>>>>
>>>>>>>> So wanted to raise this if not detected that we need to fix
>>>>> constraints
>>>>>>>> before releasing 3.1.2 as well.
>>>>>>>> Don't know why I catched the "wrong" constraints" via installing
>>>>> with
>>>>>>>> breeze - might be constraints are not considered when installed a
>>>>> specific
>>>>>>>> airflow version and then providers are upgraded?
>>>>>>>>
>>>>>>>> Jens
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Amogh Desai<[email protected]>
>>>>>>>> Sent: Dienstag, 4. November 2025 10:07
>>>>>>>> To:[email protected]
>>>>>>>> Subject: Re: [VOTE] Airflow Providers prepared on November 03, 2025
>>>>>>>>
>>>>>>>> Jens, we have constrained structlog to disallow 25.5.0:
>>>>>>>>
>>>>>>>>
>>>>> https://raw.githubusercontent.com/apache/airflow/constraints-3.1.0/constraints-3.10.txt
>>>>>>>>
>>>>>>>> So, that should not be a problem for you :), maybe it was installed
>>>>> in
>>>>>>>> your environment.
>>>>>>>>
>>>>>>>> Thanks & Regards,
>>>>>>>> Amogh Desai
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Nov 4, 2025 at 3:59 AM Jens Scheffler<[email protected]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> +1 (binding) - Checked SVN, Check in Docker, Reproducible package
>>>>>>>>> +build,
>>>>>>>>> Licenses, Signatures
>>>>>>>>>
>>>>>>>>> Installed the edge provider with current common-compat in Airflow
>>>>>>>>> 2.11.0, 3.0.6, 3.1.2rc1 and main and tested with EdgeExecutor and
>>>>>>>>> Integration Test Dag, all loking good.
>>>>>>>>>
>>>>>>>>> Will test detailled issue resolution (subpath etc) tomorrow but
>>>>> until
>>>>>>>>> now all good to release. Best release we had for Edge3 ever :-D
>>>>>>>>>
>>>>>>>>> Note: Was not able to install providers with Airflow 3.1.1 as
>>>>>>>>> structlog==25.5.0 installed by breeze has some incompatability and
>>>>>>>>> breze start failed with - assume this is not blocking providers
>>>>>>>>> release - assume this is because constraints are installed from
>>>>> main
>>>>>>>>> and not 3.1.1 in breeze:
>>>>>>>>>
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File "/usr/python/bin/airflow", line 4, in <module>
>>>>>>>>> from airflow.__main__ import main
>>>>>>>>> File
>>>>>>>>> "/usr/python/lib/python3.12/site-packages/airflow/__init__.py",
>>>>>>>>> line 79, in <module>
>>>>>>>>> settings.initialize()
>>>>>>>>> File
>>>>>>>>> "/usr/python/lib/python3.12/site-packages/airflow/settings.py",
>>>>>>>>> line 698, in initialize
>>>>>>>>> LOGGING_CLASS_PATH = configure_logging()
>>>>>>>>> ^^^^^^^^^^^^^^^^^^^
>>>>>>>>> File
>>>>>>>>>
>>>>> "/usr/python/lib/python3.12/site-packages/airflow/logging_config.py",
>>>>>>>>> line 88, in configure_logging
>>>>>>>>> from airflow._shared.logging import configure_logging,
>>>>>>>>> init_log_folder, translate_config_values
>>>>>>>>> File
>>>>>>>>>
>>>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/__in
>>>>>>>>> it__.py",
>>>>>>>>>
>>>>>>>>> line 27, in <module>
>>>>>>>>> from .structlog import configure_logging, init_log_file,
>>>>>>>>> init_log_folder
>>>>>>>>> File
>>>>>>>>>
>>>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/stru
>>>>>>>>> ctlog.py",
>>>>>>>>>
>>>>>>>>> line 38, in <module>
>>>>>>>>> from .percent_formatter import PercentFormatRender
>>>>>>>>> File
>>>>>>>>>
>>>>> "/usr/python/lib/python3.12/site-packages/airflow/_shared/logging/perc
>>>>>>>>> ent_formatter.py",
>>>>>>>>>
>>>>>>>>> line 29, in <module>
>>>>>>>>> from structlog.dev import ConsoleRenderer, Styles
>>>>>>>>> ImportError: cannot import name 'Styles' from 'structlog.dev'
>>>>>>>>> (/usr/python/lib/python3.12/site-packages/structlog/dev.py)
>>>>>>>>>
>>>>>>>>> Error: check_environment returned 1. Exiting.
>>>>>>>>>
>>>>>>>>> Call that I used for testing: breeze start-airflow --python 3.12
>>>>>>>>> --load-example-dags --backend postgres --executor EdgeExecutor
>>>>>>>>> --answer y --use-airflow-version 3.1.1
>>>>> --use-distributions-from-dist
>>>>>>>>>
>>>>>>>>> Tested also with Airflow 3.1.0 and saw as intended that edge3
>>>>> provider
>>>>>>>>> 1.4.1rc2 can not installed with Airflow 3.1.0 as UI plugin
>>>>> incompatible.
>>>>>>>>>
>>>>>>>>> Using Python 3.12.12 environment at: /usr/python
>>>>>>>>> × No solution found when resolving dependencies:
>>>>>>>>> ╰─▶ Because only apache-airflow-providers-edge3==1.4.1 is
>>>>> available
>>>>>>>>> and apache-airflow-providers-edge3==1.4.1 depends on one of:
>>>>>>>>> apache-airflow>=2.10.0,<3.1.0
>>>>>>>>> apache-airflow>3.1.0
>>>>>>>>> we can conclude that all versions of
>>>>>>>>> apache-airflow-providers-edge3 depend on one of:
>>>>>>>>> apache-airflow>=2.10.0,<3.1.0
>>>>>>>>> apache-airflow>3.1.0
>>>>>>>>>
>>>>>>>>> And because you require apache-airflow==3.1.0 and
>>>>>>>>> apache-airflow-providers-edge3, we can conclude that your
>>>>> requirements
>>>>>>>>> are unsatisfiable.
>>>>>>>>>
>>>>>>>>> On 11/3/25 13:06, Elad Kalif wrote:
>>>>>>>>>> Hey all,
>>>>>>>>>>
>>>>>>>>>> I have just cut the ad-hoc wave Airflow Providers packages. This
>>>>>>>>>> email is calling a vote on the release, which will last for 72
>>>>> hours
>>>>>>>>>> - which means that it will end on November 06, 2025 12:05 PM UTC
>>>>> and
>>>>>>>>>> until 3 binding +1 votes have been received.
>>>>>>>>>>
>>>>>>>>>> Consider this my (binding) +1.
>>>>>>>>>>
>>>>>>>>>> Airflow Providers are available at:
>>>>>>>>>> https://di/
>>>>>>>>>> st.apache.org
>>>>> %2Frepos%2Fdist%2Fdev%2Fairflow%2Fproviders%2F&data=05%
>>>>>>>>>> 7C02%7CJens.Scheffler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1b819
>>>>>>>>>>
>>>>> e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681481189%
>>>>>>>>>>
>>>>> 7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIs
>>>>>>>>>>
>>>>> IlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=
>>>>>>>>>> u%2BGuEPt6aFORkaSjATUG2TVoDJiUJmNDiTM9Txe4cQc%3D&reserved=0
>>>>>>>>>>
>>>>>>>>>> *apache-airflow-providers-<PROVIDER>-*.tar.gz* are the binary
>>>>>>>>>> Python "sdist" release - they are also official "sources" for
>>>>> the
>>>>>>>>> Provider
>>>>>>>>>> distributions.
>>>>>>>>>>
>>>>>>>>>> *apache_airflow_providers_<PROVIDER>-*.whl are the binary
>>>>>>>>>> Python "wheel" release.
>>>>>>>>>>
>>>>>>>>>> The test procedure for PMC members is described in
>>>>>>>>>>
>>>>>>>>> https://gith/
>>>>>>>>> ub.com
>>>>> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE
>>>>>>>>>
>>>>> RS.md%23verify-the-release-candidate-by-pmc-members&data=05%7C02%7CJen
>>>>>>>>> s.Scheffler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e
>>>>>>>>>
>>>>> 1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681516854%7CUnknown%7CTW
>>>>>>>>>
>>>>> FpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIs
>>>>>>>>>
>>>>> IkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=6sgJ88Z0t%2F1Qnhxr
>>>>>>>>> wPTnej7SfgLNf6hI7qYxnJBl6RU%3D&reserved=0
>>>>>>>>>> The test procedure for and Contributors who would like to test
>>>>> this
>>>>>>>>>> RC is described in:
>>>>>>>>>>
>>>>>>>>> https://gith/
>>>>>>>>> ub.com
>>>>> %2Fapache%2Fairflow%2Fblob%2Fmain%2Fdev%2FREADME_RELEASE_PROVIDE
>>>>>>>>>
>>>>> RS.md%23verify-the-release-candidate-by-contributors&data=05%7C02%7CJe
>>>>>>>>> ns.Scheffler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51
>>>>>>>>>
>>>>> e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681546634%7CUnknown%7CT
>>>>>>>>>
>>>>> WFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI
>>>>>>>>>
>>>>> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=j%2B4anx9P7T5uTdc
>>>>>>>>> omIOr%2BOr2%2FoDHQnntn1Pvhn33BHs%3D&reserved=0
>>>>>>>>>>
>>>>>>>>>> Public keys are available at:
>>>>>>>>>> https://di/
>>>>>>>>>> st.apache.org
>>>>> %2Frepos%2Fdist%2Frelease%2Fairflow%2FKEYS&data=05%7C02
>>>>>>>>>> %7CJens.Scheffler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1b819e4d%
>>>>>>>>>>
>>>>> 7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681574562%7CUn
>>>>>>>>>>
>>>>> known%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAi
>>>>>>>>>>
>>>>> OiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=enB4
>>>>>>>>>> BNSldvybyXLUzIJIz5jE2tCnZ904nXCrgzoZp%2Bg%3D&reserved=0
>>>>>>>>>>
>>>>>>>>>> Please vote accordingly:
>>>>>>>>>>
>>>>>>>>>> [ ] +1 approve
>>>>>>>>>> [ ] +0 no opinion
>>>>>>>>>> [ ] -1 disapprove with the reason
>>>>>>>>>>
>>>>>>>>>> Only votes from PMC members are binding, but members of the
>>>>>>>>>> community are encouraged to test the release and vote with
>>>>>>>> "(non-binding)".
>>>>>>>>>> Please note that the version number excludes the 'rcX' string.
>>>>>>>>>> This will allow us to rename the artifact without modifying the
>>>>>>>>>> artifact checksums when we actually release.
>>>>>>>>>>
>>>>>>>>>> The status of testing the providers by the community is kept here:
>>>>>>>>>> https://gi/
>>>>>>>>>> thub.com
>>>>> %2Fapache%2Fairflow%2Fissues%2F57739&data=05%7C02%7CJens.Sch
>>>>>>>>>> effler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1b819e4d%7C0ae51e190
>>>>>>>>>>
>>>>> 7c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681602068%7CUnknown%7CTWF
>>>>>>>>>>
>>>>> pbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiI
>>>>>>>>>>
>>>>> sIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=PPDrnZgPIfBJWRp
>>>>>>>>>> 5VyEq1Ij2NGJXs9qK3b9mJA7Cwjk%3D&reserved=0
>>>>>>>>>>
>>>>>>>>>> The issue is also the easiest way to see important PRs included in
>>>>>>>>>> the RC candidates.
>>>>>>>>>> Detailed changelog for the providers will be published in the
>>>>>>>>> documentation
>>>>>>>>>> after the
>>>>>>>>>> RC candidates are released.
>>>>>>>>>>
>>>>>>>>>> You can find the RC packages in PyPI following these links:
>>>>>>>>>>
>>>>>>>>>> https://py/
>>>>>>>>>> pi.org
>>>>> %2Fproject%2Fapache-airflow-providers-edge3%2F1.4.1rc2%2F&data
>>>>>>>>>> =05%7C02%7CJens.Scheffler%40de.bosch.com
>>>>> %7Cf6343926cc624857a11208de1
>>>>>>>>>>
>>>>> b819e4d%7C0ae51e1907c84e4bbb6d648ee58410f4%7C0%7C0%7C638978440681628
>>>>>>>>>>
>>>>> 771%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAw
>>>>>>>>>>
>>>>> MCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sd
>>>>>>>>>> ata=HWxblv2u1%2B6Pr3QrB5N1s9k7wnOcgLZNi7qsgW3t9i0%3D&reserved=0
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>> Elad Kalif
>>>>>>>>>>
>>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>>> To unsubscribe, e-mail:[email protected]
>>>>>>>>> For additional commands, e-mail:[email protected]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail:[email protected]
>>>>>>>> For additional commands, e-mail:[email protected]
>>>>>>>>
>>>>
>>>>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
