potiuk commented on issue #15463: URL: https://github.com/apache/airflow/issues/15463#issuecomment-826112107
OK @zachliu @uranusjr -> I think I solved it. The PR to switch to PIP 21.* is here: https://github.com/apache/airflow/pull/15513 The culprit was apache-beam and it's 'gcp' extra. We had "apache-beam[gcp]" specified in our requirements for apache-beam, but in fact the `gcp` is only needed for it if you want to use it together with `google` package. Which is already an extra on the 'apache-airflow-providers-beam` package. So I moved the `extra` from "airflow" level to "provider" level. I believe this will work fine by PIP automatically finding and installing one of the earlier version of `apache-airflow-providers-google` which should have compatible set of requirements (i will do more tests with it later). Even if someone would like to use latest version of Google providers with Dataflow (Google's managed Apache Beam service), this will still work fine, as google provider already has `Dataflow` operators. And It is anyhow a bit unsolvable problem until Apache Beam migrates to google client > 2*, so I think this solution will allow to eat and have the biggest part of the cake we can in this case. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
