Huge DEPENDENCY dump of course :) On Thu, May 18, 2023 at 11:09 AM Jarek Potiuk <ja...@potiuk.com> wrote:
> Hello everyone, > > Due to yesterday's dependency changes I recommend everyone to rebase your > PRs to the latest main and run `breeze ci-image build` if you are using > breeze (and likely upgrading/recreating your local venvs). > > We have quite a huge dependency bump in main - thanks to the finally > merged dependency upgrade for Google provider: > https://github.com/apache/airflow/pull/30067. This took quite some time > (several months, countless rebases, some 25+ google libraries to upgrade in > a single PR and a lot of patience was needed as there were a lot of changes > we had to wait for. I also removed vendored-in `google-ads` as the upgrade > allows us to use the regular google-ads library, we have also switched to > the latest v13 API for ads - ahead of July's sunsetting of v12 API. That > should end the "drama" we had with ads-api being sunset and some of our > users being cut-off. > > Note that the new Google provider is Airflow 2.4+ only, so if you want to > make use of it, users will have to upgrade Airflow to at least Airflow 2.4. > > Elad is going to make an ad-hoc release for google provider right after we > release the current wave. I expect a bit more heavy testing (Google team is > going to take it to a spin) and some "teething" problems so it might take a > few RCs to get it released. But generally speaking it looks good. > > This also opens up 3.11 migration (PR in progress here: > https://github.com/apache/airflow/pull/27264 and I expect it to land this > week so that Airflow 2.7 will have Python 3.11 support finally > > Thanks to Łukasz Wyszomirski who has done a lot of coordinating work for a > long time and Augusto Hidalgo who led it to completion - but also some > unnamed :) people in the Google team who contributed to it. > > That was a big effort that Google team undertool and we had to wait for > others to complete (for example apache.beam) but it finally landed - > freeing some long awaited dependency upgrades (protobuf 3 -> 4 being the > most problematic) > > We also had a few "stabilizing" updates after (things got a bit slower for > tests and we will optimize it in the future) but generally speaking all > looks good > > For the record - here is what changed (if you see strange errors - they > might come from one of those): > > 43a44 > > Shapely==1.8.5.post1 > 56c57 > < alembic==1.11.0 > --- > > alembic==1.11.1 > 63c64 > < apache-beam==2.46.0 > --- > > apache-beam==2.47.0 > 108c109 > < black==23.1a1 > --- > > black==23.3.0 > 163c164 > < elasticsearch-dbapi==0.2.10 > --- > > elasticsearch-dbapi==0.2.2 > 165c166 > < elasticsearch==7.13.4 > --- > > elasticsearch==7.14.2 > 190,191c191,193 > < google-api-core==2.8.2 > < google-api-python-client==1.12.11 > --- > > google-ads==21.1.0 > > google-api-core==2.11.0 > > google-api-python-client==2.86.0 > 193,205c195,207 > < google-auth-oauthlib==0.8.0 > < google-auth==2.18.0 > < google-cloud-aiplatform==1.16.1 > < google-cloud-appengine-logging==1.1.3 > < google-cloud-audit-log==0.2.4 > < google-cloud-automl==2.8.0 > < google-cloud-bigquery-datatransfer==3.7.0 > < google-cloud-bigquery-storage==2.14.1 > < google-cloud-bigquery==2.34.4 > > google-cloud-memcache==1.7.1 > > google-cloud-monitoring==2.14.2 > > google-cloud-orchestration-airflow==1.7.1 > > google-cloud-os-login==2.9.1 > > google-cloud-pubsub==2.17.0 > > google-cloud-redis==2.12.1 > > google-cloud-resource-manager==1.10.1 > > google-cloud-secret-manager==2.16.1 > > google-cloud-spanner==3.34.0 > > google-cloud-speech==2.19.0 > 228,233c230,235 > < google-cloud-tasks==2.10.1 > < google-cloud-texttospeech==1.0.3 > < google-cloud-translate==1.7.2 > < google-cloud-videointelligence==1.16.3 > < google-cloud-vision==1.0.2 > < google-cloud-workflows==1.7.1 > --- > > google-cloud-tasks==2.13.1 > > google-cloud-texttospeech==2.14.1 > > google-cloud-translate==3.11.1 > > google-cloud-videointelligence==2.11.1 > > google-cloud-vision==3.4.1 > > google-cloud-workflows==1.10.1 > 236c238 > < googleapis-common-protos==1.56.4 > --- > > googleapis-common-protos==1.59.0 > 240c242 > < grpc-google-iam-v1==0.12.4 > --- > > grpc-google-iam-v1==0.12.6 > 242c244 > < grpcio-status==1.48.2 > --- > > grpcio-status==1.54.2 > 358,359c360,361 > < packaging==21.3 > < pandas-gbq==0.17.9 > --- > > packaging==23.1 > > pandas-gbq==0.19.2 > 368c370 > < pdpyras==5.0.1 > --- > > pdpyras==5.0.3 > 387,388c389,390 > < proto-plus==1.19.6 > < protobuf==3.20.0 > --- > > proto-plus==1.22.2 > > protobuf==4.22.5 > 396c398 > < pyarrow==9.0.0 > --- > > pyarrow==11.0.0 > 412c414 > < pymongo==3.13.0 > --- > > pymongo==4.3.3 > 513c515 > < tableauserverclient==0.24 > --- > > tableauserverclient==0.25 > 557c559 > < uritemplate==3.0.1 > --- > > uritemplate==4.1.1 > > J. > >
