potiuk commented on pull request #20509: URL: https://github.com/apache/airflow/pull/20509#issuecomment-1029180353
> @mik-laj Makes sense. Why is the latest Airflow constraints pointing to an older version of the providers? Longer answer: Often new providers introduce dependency changes that might or might not work with Airlfow, the constraints are "golden" set of dependencies that we know that will work (can be installed and passes all the tests) - not only when we release Airflow, but only in the future when new dependencies are released. This is what we - as community commit to, and this is also what we maintain (and every new release of airlfow witll automatically have constratints with the latest released providers. Constraints are there to make sure "You can always install Airflow with those constraints and we guarantee they will work fine". For the newly released providers - we make a lot of work to make sure they work with previously released airflow versions - for example we run tests to make sure they still can be used with Airflow 2.1.0. However - what we DO NOT guarantee - that you can install the providers together with oher released providers on the older airflow version using single "pip install" operation. Due to complexity of dependencies and their live nature we cannot guarantee that all combinations of providers can be installed "together" in a "single" operation. We can guarantee however that you should be able to upgrade providers on your own (as long as this version is compatible with your Airlfow version) and we even give you the ways how to do it: see [installation and upgrade scenarios](https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#installation-and-upgrade-scenarios). But - in this case - depending on the set and versions of providers you already have installed, you are responsible to resolve all the conflict dependencies if and when they occur. We cannot guarantee that all combinations and versions of all airflow versions and providers will work. That's probably millions of combinations and I am pretty sure some of them won't work. But when you reduce it to the set of providers that you have and you use PIP to resolve those installs (Without using constraints) you should be able to get the right combination in vast majority of cases. But - it's your work and responsibilty not ours. We do explicitly and deliberately do not put that responsibility on our shoulders. It's yours to bear. With great powers (flexibility of installing newer version of providers without upgrading airflow) , comes great responsibilities, The easiest way to keep up-to-date is just to upgrade Airflow to latest version - as soon as it released or soon after. This is the option you always have (or the option or upgrade providers independently between the releases - manually). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
