potiuk commented on issue #17453:
URL: https://github.com/apache/airflow/issues/17453#issuecomment-894621792
Yeah it's for more 'adventurous' ones :). It is not really 'documented' yet
and It does not have the same 'guarantees' as the airflow constraints. But it
might prevent some obvious problems like this one when we have a new 'breaking'
release of dependency.
And if some main changes happened since the last provider release, they
**might** break.
The way how it should be done (and I might actually implement it in our
processes) is to to tag the 'main' constraints at the moment we release the
providers (with separate tag per provider). That can be easily automated, and I
might add it to our process. We can even track down historical constraints and
re-tag the history for all provider releases. Should be easy.
It has a few caveats and i think it is basically almost unsolvable problem -
it will only fully work when you have the same set of provider versions as
those at the moment of tagging. It is possible that there is a combination of
providers that has conflicting dependencies in different versions and that
provider a in version 1.0.0 will never work with provider b in version 2.0.0.
So when you install provider a and you already have provider b, they might
conflict.
However, the last problem is somewhat solvable if we tag constraints with
provider versions. Our constraints (the default ones - because we actually have
three sets of constraints - PyPI providers, no-providers, source-providers)
contain actually all the providers with versions. So if you have a conflicting
provider (and once we bring the tags in) we should be able to do something like:
`pip install apache-airflow-provider-google==5.0.0
apache-airflow-provider-amazon --constraint
http....apache-airflow-provider-google/5.0.0/constraints-3.6.txt`
And it will not only install Google provider in the right version and all
it's dependencies in the right version but it will also update the Amazon
provider (and it's dependencies) to the version that was released at the same
time as the Google 5.0.0 (thus - with guaranteed, not conflicting dependencies)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
