potiuk commented on code in PR #47281: URL: https://github.com/apache/airflow/pull/47281#discussion_r1977884408
########## generated/provider_dependencies.json: ########## Review Comment: I just found it - because I tried individually a few providers - that it was actually **missing** - because some of the code in amazon provider uses opensearch. This is the side-effect (and a very positive one) of the new approach - where we separated providers out to their independent projects, and with `uv` approach for virtualenv syncing that we will start detecting those kinds of missing dependencies. So far we tested every provider in the same venv (in breeze) and most of the time in localvenv - and in order to make things easier we usually installed more extras than needed (all of them with devel_ci or `--all-extras`) - this means that we often already had some dependencies installed - and we used them even if they were not our dependencies. After this PR, you will be able to do: ```bash cd provider/amazon uv run pytest ``` This is what I did for amazon and found out that some amazon provider code uses opensearch directly (not opensearch provider - but directly opensearch package). And what happens here is that ONLY dependencies that are either in `tests_common` (test deps) , `airflow`, `task_sdk` and dependencies of `amazon` provider (and dependent providers - when there are cross-provider dependencies) will be installed. In this case `uv sync` will **REMOVE** all the other dependencies not explicitly mentioned as amazon provider dependency. Note that current set of dependencies is too wide now - because we have `airflow" as a dependency and `task_sdk` requires always "standard", "common.sql" and "fab" providers to be installed - but this is because some of the tests and imports are not cleaned yet and we have not yet moved providers to **only** use `task_sdk`. But after this change (and moving `task_sdk` to `task-sdk` and airflow code to `airflow-core` subproject - which are the next logica steps, this cleanup will be easy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
