potiuk commented on pull request #18671: URL: https://github.com/apache/airflow/pull/18671#issuecomment-933331600
> To my understanding the postgres dependency is relevant only for users of Redshift so if you need the AWS provider for Athena or some other service does it make sense to force install postgres provider as well? We do not force it, actually. It's only when you install `apache-airflow-providers-amazon[postgres]' this additionally installls apache-airflow-providers-postgres. The import of "airflow.providers.postgres" is only used when you use one of the Redshift Operators. We consider all such "cross-provider" dependencies as optional and that's why we have them as "extras" in providers and there are more "indications" that there is optional dependency, than actual "forcing" it. > I'd prefer if we could be able to specify the dependency directly in the package metadata because that's what metadata is for, isn't it 😛 But I'm obviously biased toward solving things this way. Yeah, ideally I agree and we could do it, but I am not sure if it justifies cancelling the RC and releasing a new rc (maybe we can simply add it in the future version ?). Correct me if I am wrong @uranusjr - even if we put (>2.3.0) there, this is not a "hard" requirement. Extras are very "soft" dependencies and they only work temporarily while installation is running - they have no long lasting effects on the installation. You could easily both remove and change version to < 2.3.0 of postgres because the "extras" limitation is not stored persistently (we do not even know that the extra was That's why I think this scenario where it causes problems is quite unlikely. First - I imaginge most people will install those providers together: `pip install apache-airflow[postgres,amazon,....]` or via separately instaling 'apache-airflow-providers-amazon' and `apache-airflow-providers-postgres`. Both of those will work fine as latest versions of both providers will be used. Secondly - When you install them via images - users will have them both installed and consistent by default. Thirdly - also if you use constraints (which is the only supported method) all will be fine. It's only when you upgrade the amazon provider without upgrading the postgres one - then you can have the problem (and even then you would have to remember to run `pip install -U apache-airflow-providers-amazon[postgres]` for the limitation to be actually used). We have no way to "force" upgrading "postgres" if the user upgrades `pip install -U apache-airlfow-providers-amazon` (because postgres is optional extra). And always answer here will be "use constraints" as we do for all other dependencies now. So I really think just doc update mentioning this "implicit" dependency for now will be enough. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
