JDarDagran commented on PR #39946: URL: https://github.com/apache/airflow/pull/39946#issuecomment-2141809110
> > Does this PR assume all cross-provider dependencies should be expressed as extras (in terms of python packaging, added in additional-extras for each provider)? > > Unfortunately the way it works in Python, those "extra" limits are not enforceable, and It makes little sense to add them (and verify). There is really no good way to speciffy "if you want to install this optional dependency, it has to follow those limits" - there are multiple scenarios where even with those limits set on extra, user will be able to perform a sequence of upgrades to break it, and at any point in their process even `pip check` will show that everything is fine. So I am not sure we even should attempt to do such a check. Fully agreed. Do you still think that would be worth adding different kind of tests too? I'll stick to OpenLineage as there have been cases for not keeping backwards-compatibility we did not detect. https://github.com/apache/airflow/pull/39530 also brings such and would love to have something that runs checks against this. My initial idea is to maintain a static list of providers that can be downgraded to the latest released version, similar to how Pydantic and SQLAlchemy have special tests for downgrading. This way, if changes are made to a provider that affect compatibility with OpenLineage, tests will fail if the changes are not backwards-compatible. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
