SameerMesiah97 commented on PR #72542: URL: https://github.com/apache/airflow/pull/72542#issuecomment-5592192017
> Thanks for raising this. I agree that the Amazon release must depend on the first `cncf-kubernetes` release containing `KubernetesPodExecOperator`, but this is the exact cross-provider scenario covered by Airflow's documented [`# use next version` mechanism](https://github.com/apache/airflow/blob/main/contributing-docs/13_airflow_dependencies_and_extras.rst#managing-dependencies-in-pyprojecttoml-files:~:text=They%20should%20exclusively,release%20is%20prepared), which is already present in this PR. The Release Manager will update the optional dependency and coordinate both releases, so I would prefer this documented mechanism over conditional imports unless the AWS maintainers specifically prefer that approach. > > > Sometimes, when you add a new feature to a common distribution, you might add a feature to it or change the API in the way that other packages can use it. This is especially true for common packages such as apache-airflow-providers-common-compat, but can happen for other packages (for example apache-airflow-providers-apache-beam is used by apache-airflow-providers-google to use Apache Beam hooks to communicate with Google Dataflow). In such case, when you are adding a feature to a common package remember that the feature you just add will only be released in the FUTURE release of such common package and you cannot add >==x.y.z dependency to it where x.y.z is the version you are going to release in the future. Ultimately, this should happen (and happens) when the Release Manager prepares both packages together. Let us repeat - such changes in versions between different airflow package should NOT be added to the dependencies manually by the contributor. They should exclusively b e added by the Release Manager. when preparing the release of both packages together. We have a custom mechanism to support such additions, where it is contributor's responsibility to mark dependency with a special comment - simply communicating with the Release Manager that such dependency should be updated to the next version when the release is prepared. If you see such a need to use newly added feature and using it at the same time in a different distribution - make sure to add this comment in the line where dependency you want to use the new feature from is defined I think I may have been a bit too presumptuous in my previous comment. Since the `cncf-provider` is an optional dependency, it means that only those users who use the EKS modules will be affected. Regular AWS provider users can simply ignore the `cncf-provider`. It is still a large bump in the minimum version for a dependency but given that EKS is more niche, it might be acceptable. Lets see what the AWS maintainers think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
