eladkal opened a new issue, #39842: URL: https://github.com/apache/airflow/issues/39842
### Body I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers? The guidelines on how to test providers can be found in [Verify providers by contributors](https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors) Let us know in the comment, whether the issue is addressed. Those are providers that require testing as there were some substantial changes introduced: ## Provider [alibaba: 2.8.1rc1](https://pypi.org/project/apache-airflow-providers-alibaba/2.8.1rc1) - [ ] [Fix oss key prefix bug in oss_task_handler, alibaba-provider (#39627)](https://github.com/apache/airflow/pull/39627): @EricGao888 Linked issues: ## Provider [amazon: 8.23.0rc1](https://pypi.org/project/apache-airflow-providers-amazon/8.23.0rc1) - [ ] [Amazon Bedrock - Retrieve and RetrieveAndGenerate (#39500)](https://github.com/apache/airflow/pull/39500): @ferruzzi - [ ] [Introduce Amazon Comprehend Service (#39592)](https://github.com/apache/airflow/pull/39592): @gopidesupavan - [ ] [[OpenLineage] fix: empty dataset name for AthenaOperator (#39677)](https://github.com/apache/airflow/pull/39677): @kacpermuda - [ ] [Fix default value for aws batch operator retry strategy (#39608)](https://github.com/apache/airflow/pull/39608): @0x26res - [ ] [Sagemaker trigger: pass the job name as part of the event (#39671)](https://github.com/apache/airflow/pull/39671): @vincbeck Linked issues: - [ ] [Linked Issue #39503](https://github.com/apache/airflow/issues/39503): @mxuan0 - [ ] [Handle task adoption for batch executor (#39590)](https://github.com/apache/airflow/pull/39590): @vincbeck Linked issues: - [ ] [Linked Issue #37786](https://github.com/apache/airflow/pull/37786): @ferruzzi - [ ] [NeptuneStopDbClusterOperator - Handle invalid cluster states (#38287)](https://github.com/apache/airflow/pull/38287): @ellisms Linked issues: - [ ] [Linked Issue #38120](https://github.com/apache/airflow/issues/38120): @ferruzzi - [ ] [Fix automatic termination issue in EmrOperator by ensuring `waiter_max_attempts` is set for deferrable triggers (#38658)](https://github.com/apache/airflow/pull/38658): @beobest2 - [ ] [Resolving EMR deprecated warnings (#39743)](https://github.com/apache/airflow/pull/39743): @dirrao Linked issues: - [ ] [Linked Issue #39485](https://github.com/apache/airflow/issues/39485): @Taragolis ## Provider [apache.beam: 5.7.1rc1](https://pypi.org/project/apache-airflow-providers-apache-beam/5.7.1rc1) - [ ] [Fix deferrable mode for BeamRunJavaPipelineOperator (#39371)](https://github.com/apache/airflow/pull/39371): @MaksYermak ## Provider [apache.druid: 3.10.1rc1](https://pypi.org/project/apache-airflow-providers-apache-druid/3.10.1rc1) - [ ] [Pass SSL arg to all requests in DruidOperator (#39066)](https://github.com/apache/airflow/pull/39066): @danielbe11 Linked issues: - [ ] [Linked Issue #37673](https://github.com/apache/airflow/pull/37673): @danielbe11 ## Provider [apache.hive: 8.1.1rc1](https://pypi.org/project/apache-airflow-providers-apache-hive/8.1.1rc1) - [ ] [Improvising high availability field name in hive hook (#39658)](https://github.com/apache/airflow/pull/39658): @amoghrajesh ## Provider [apache.kafka: 1.4.1rc1](https://pypi.org/project/apache-airflow-providers-apache-kafka/1.4.1rc1) - [ ] [Group id is mandatory configurtion option for confluent_kafka 2.4.0+ (#39559)](https://github.com/apache/airflow/pull/39559): @potiuk ## Provider [apprise: 1.3.1rc1](https://pypi.org/project/apache-airflow-providers-apprise/1.3.1rc1) - [ ] [Fix Apprise mypy checks added in 1.8.0 (#39580)](https://github.com/apache/airflow/pull/39580): @potiuk ## Provider [atlassian.jira: 2.6.1rc1](https://pypi.org/project/apache-airflow-providers-atlassian-jira/2.6.1rc1) - [ ] [Bump version of atlassian-python-api (#39714)](https://github.com/apache/airflow/pull/39714): @eladkal ## Provider [celery: 3.7.1rc1](https://pypi.org/project/apache-airflow-providers-celery/3.7.1rc1) - [ ] [ECS Executor: Set tasks to RUNNING state once active (#39212)](https://github.com/apache/airflow/pull/39212): @o-nikolas - [ ] [Remove compat code for 2.7.0 - its now the min Airflow version (#39591)](https://github.com/apache/airflow/pull/39591): @jedcunningham ## Provider [cncf.kubernetes: 8.3.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/8.3.0rc1) - [ ] [Add timeout when watching pod events in k8s executor (#39551)](https://github.com/apache/airflow/pull/39551): @dstandish - [ ] [Add retry logic for KubernetesCreateResourceOperator and KubernetesJobOperator (#39201)](https://github.com/apache/airflow/pull/39201): @MaksYermak - [ ] [Fix deprecated calls in `cncf.kubernetes` provider (#39381)](https://github.com/apache/airflow/pull/39381): @Taragolis Linked issues: - [ ] [Linked Issue #38642](https://github.com/apache/airflow/issues/38642): @Taragolis - [ ] [Linked Issue #39356](https://github.com/apache/airflow/issues/39356): @bmoon4 - [ ] [Move Kubernetes cli to provider package (#39587)](https://github.com/apache/airflow/pull/39587): @romsharon98 - [ ] [Remove compat code for 2.7.0 - its now the min Airflow version (#39591)](https://github.com/apache/airflow/pull/39591): @jedcunningham - [ ] [Replace pod_manager.read_pod_logs with client.read_namespaced_pod_log in KubernetesPodOperator._write_logs (#39112)](https://github.com/apache/airflow/pull/39112): @Lee-W ## Provider [common.io: 1.3.2rc1](https://pypi.org/project/apache-airflow-providers-common-io/1.3.2rc1) - [ ] [[openlineage] fix: OpenLineage in FileTransferOperator for Airflow 2.8 (#39755)](https://github.com/apache/airflow/pull/39755): @kacpermuda Linked issues: - [ ] [Linked Issue #36410](https://github.com/apache/airflow/pull/36410): @bolkedebruin - [ ] [Linked Issue #39606](https://github.com/apache/airflow/pull/39606): @potiuk ## Provider [common.sql: 1.14.0rc1](https://pypi.org/project/apache-airflow-providers-common-sql/1.14.0rc1) - [ ] [Add `parameters` as template field for SqlSensor (#39588)](https://github.com/apache/airflow/pull/39588): @josh-fell - [ ] [DbAPiHook: Don't log a warning message if placeholder is None and make sure warning message is formatted correctly (#39690)](https://github.com/apache/airflow/pull/39690): @dabla Linked issues: - [ ] [Linked Issue #39678](https://github.com/apache/airflow/issues/39678): @romanzdk - [ ] [DbAPiHook executemany parameter of insert_rows should not be deprecated (#39630)](https://github.com/apache/airflow/pull/39630): @dabla - [ ] [Add typing for SqlSensor (#39773)](https://github.com/apache/airflow/pull/39773): @josh-fell ## Provider [databricks: 6.5.0rc1](https://pypi.org/project/apache-airflow-providers-databricks/6.5.0rc1) - [ ] [add deferrable support to `DatabricksNotebookOperator` (#39295)](https://github.com/apache/airflow/pull/39295): @rawwar Linked issues: - [ ] [Linked Issue #39178](https://github.com/apache/airflow/pull/39178): @pankajkoti - [ ] [[FEAT] get all failed tasks errors in exception (#39354)](https://github.com/apache/airflow/pull/39354): @gaurav7261 - [ ] [Better typing for BaseOperator `defer` (#39742)](https://github.com/apache/airflow/pull/39742): @jedcunningham ## Provider [dbt.cloud: 3.8.1rc1](https://pypi.org/project/apache-airflow-providers-dbt-cloud/3.8.1rc1) - [ ] [Scheduler to handle incrementing of try_number (#39336)](https://github.com/apache/airflow/pull/39336): @dstandish - [ ] [Validate dbt `trigger_reason` field to be less than 255 characters (#38896)](https://github.com/apache/airflow/pull/38896): @boraberke - [ ] [Add (optional) dependency between dbt-cloud and openlineage providers (#39366)](https://github.com/apache/airflow/pull/39366): @tatiana ## Provider [docker: 3.12.0rc1](https://pypi.org/project/apache-airflow-providers-docker/3.12.0rc1) - [ ] [Add args to docker service ContainerSpec (#39464)](https://github.com/apache/airflow/pull/39464): @guydriesen - [ ] [Add support to define Resources on DockerSwarmOperator (#39027)](https://github.com/apache/airflow/pull/39027): @guydriesen - [ ] [Limit requests in botocore upgrade test (#39747)](https://github.com/apache/airflow/pull/39747): @potiuk Linked issues: - [ ] [Linked Issue #39740](https://github.com/apache/airflow/pull/39740): @eladkal - [ ] [Pin requests due to incompatibility with docker-py (#39740)](https://github.com/apache/airflow/pull/39740): @eladkal ## Provider [elasticsearch: 5.4.1rc1](https://pypi.org/project/apache-airflow-providers-elasticsearch/5.4.1rc1) - [ ] [Make elastic search index_pattern more configurable (#38423)](https://github.com/apache/airflow/pull/38423): @pankajastro - [ ] [Scheduler to handle incrementing of try_number (#39336)](https://github.com/apache/airflow/pull/39336): @dstandish ## Provider [fab: 1.1.1rc1](https://pypi.org/project/apache-airflow-providers-fab/1.1.1rc1) - [ ] [Simplify action name retrieval in FAB auth manager (#39358)](https://github.com/apache/airflow/pull/39358): @jedcunningham - [ ] [Add `jmespath` as an explicit dependency (#39350)](https://github.com/apache/airflow/pull/39350): @jedcunningham ## Provider [google: 10.19.0rc1](https://pypi.org/project/apache-airflow-providers-google/10.19.0rc1) - [ ] [Introduce anonymous credentials in GCP base hook (#39695)](https://github.com/apache/airflow/pull/39695): @shahar1 Linked issues: - [ ] [Linked Issue #35755](https://github.com/apache/airflow/issues/35755): @jvaesteves - [ ] [Remove parent_model version suffix if passed to Vertex AI operators (#39640)](https://github.com/apache/airflow/pull/39640): @e-galan Linked issues: - [ ] [Linked Issue #38417](https://github.com/apache/airflow/pull/38417): @shahar1 - [ ] [Fix BigQueryCursor execute method if the location is missing (#39659)](https://github.com/apache/airflow/pull/39659): @e-galan - [ ] [Fix acknowledged functionality in deferrable mode for PubSubPullSensor (#39711)](https://github.com/apache/airflow/pull/39711): @MaksYermak - [ ] [Reroute AutoML operator links to Google Translation links (#39668)](https://github.com/apache/airflow/pull/39668): @e-galan - [ ] [Pin google-cloud-bigquery to < 3.21.0 (#39583)](https://github.com/apache/airflow/pull/39583): @pankajastro - [ ] [openlineage: remove `openlineage.common` dependencies in Google and Snowflake providers. (#39614)](https://github.com/apache/airflow/pull/39614): @JDarDagran - [ ] [Deprecate AutoML Tables operators (#39752)](https://github.com/apache/airflow/pull/39752): @e-galan Linked issues: - [ ] [Linked Issue #38673](https://github.com/apache/airflow/pull/38673): @molcay - [ ] [Resolve deprecation warnings in Azure FileShare-to-GCS tests (#39599)](https://github.com/apache/airflow/pull/39599): @josh-fell Linked issues: - [ ] [Linked Issue #39485](https://github.com/apache/airflow/issues/39485): @Taragolis - [ ] [[openlineage] typo: wrong OpenLineage facet key in spec (#39782)](https://github.com/apache/airflow/pull/39782): @kacpermuda - [ ] [removed stale code (#39744)](https://github.com/apache/airflow/pull/39744): @msoni1369 ## Provider [hashicorp: 3.7.1rc1](https://pypi.org/project/apache-airflow-providers-hashicorp/3.7.1rc1) - [ ] [inherit key "verify" from env into session so that kv_client can read it properly (#38614)](https://github.com/apache/airflow/pull/38614): @chz8494 ## Provider [microsoft.mssql: 3.7.1rc1](https://pypi.org/project/apache-airflow-providers-microsoft-mssql/3.7.1rc1) - [ ] [use `cached_property` to reuse `get_connections` value in `mssql` provider (#39575)](https://github.com/apache/airflow/pull/39575): @rawwar Linked issues: - [ ] [Linked Issue #39572](https://github.com/apache/airflow/pull/39572): @rawwar - [ ] [remove extra return in `get_conn` of mssql provider (#39582)](https://github.com/apache/airflow/pull/39582): @rawwar ## Provider [openlineage: 1.8.0rc1](https://pypi.org/project/apache-airflow-providers-openlineage/1.8.0rc1) - [ ] [Scheduler to handle incrementing of try_number (#39336)](https://github.com/apache/airflow/pull/39336): @dstandish - [ ] [[openlineage] fix: Prevent error when extractor can't be imported (#39736)](https://github.com/apache/airflow/pull/39736): @kacpermuda Linked issues: - [ ] [Linked Issue #38982](https://github.com/apache/airflow/pull/38982): @anandhimurali - [ ] [openlineage: improve spawned OpenLineage process in scheduler (#39735)](https://github.com/apache/airflow/pull/39735): @JDarDagran Linked issues: - [ ] [Linked Issue #39520](https://github.com/apache/airflow/pull/39520): @kacpermuda - [ ] [[openlineage] Update conf retrieval docstring and adjust pool_size (#39721)](https://github.com/apache/airflow/pull/39721): @kacpermuda - [ ] [openlineage: remove `openlineage.common` dependencies in Google and Snowflake providers. (#39614)](https://github.com/apache/airflow/pull/39614): @JDarDagran - [ ] [openlineage: use `ProcessPoolExecutor` over `ThreadPoolExecutor`. (#39235)](https://github.com/apache/airflow/pull/39235): @JDarDagran Linked issues: - [ ] [Linked Issue #39232](https://github.com/apache/airflow/issues/39232): @JDarDagran - [ ] [feat: Add custom provider runtime checks (#39609)](https://github.com/apache/airflow/pull/39609): @kacpermuda Linked issues: - [ ] [Linked Issue #39336](https://github.com/apache/airflow/pull/39336): @dstandish - [ ] [Linked Issue #39513](https://github.com/apache/airflow/pull/39513): @potiuk - [ ] [openlineage: notify that logged exception was caught (#39493)](https://github.com/apache/airflow/pull/39493): @mobuchowski - [ ] [[OpenLineage] Add more debug logs to facilitate debugging (#39136)](https://github.com/apache/airflow/pull/39136): @kacpermuda - [ ] [openlineage: add missing `dag_state_change_process_pool_size` in `provider.yaml`. (#39674)](https://github.com/apache/airflow/pull/39674): @JDarDagran Linked issues: - [ ] [Linked Issue #39235](https://github.com/apache/airflow/pull/39235): @JDarDagran - [ ] [Run unit tests with airflow installed from packages (#39513)](https://github.com/apache/airflow/pull/39513): @potiuk - [ ] [Reapply templates for all providers (#39554)](https://github.com/apache/airflow/pull/39554): @potiuk Linked issues: - [ ] [Linked Issue #39552](https://github.com/apache/airflow/pull/39552): @jedcunningham - [ ] [Linked Issue #39341](https://github.com/apache/airflow/pull/39341): @Taragolis ## Provider [pinecone: 2.0.0rc3](https://pypi.org/project/apache-airflow-providers-pinecone/2.0.0rc3) - [ ] [Pinecone provider support for `pinecone-client`>=3 (#37307)](https://github.com/apache/airflow/pull/37307): @rawwar Linked issues: - [ ] [Linked Issue #37158](https://github.com/apache/airflow/issues/37158): @Taragolis - [ ] [Fix the pinecone system test (#39365)](https://github.com/apache/airflow/pull/39365): @sunank200 - [ ] [Reapply templates for all providers (#39554)](https://github.com/apache/airflow/pull/39554): @potiuk Linked issues: - [ ] [Linked Issue #39552](https://github.com/apache/airflow/pull/39552): @jedcunningham - [ ] [Linked Issue #39341](https://github.com/apache/airflow/pull/39341): @Taragolis - [ ] [Fix the argument type of input_vectors in pinecone upsert (#39688)](https://github.com/apache/airflow/pull/39688): @sunank200 Linked issues: - [ ] [Linked Issue #37307](https://github.com/apache/airflow/pull/37307): @rawwar ## Provider [postgres: 5.11.1rc1](https://pypi.org/project/apache-airflow-providers-postgres/5.11.1rc1) - [ ] [fix: use `sqlalchemy_url` property in `get_uri` for postgresql provider (#38831)](https://github.com/apache/airflow/pull/38831): @rawwar Linked issues: - [ ] [Linked Issue #38195](https://github.com/apache/airflow/issues/38195): @Taragolis - [ ] [Linked Issue #38187](https://github.com/apache/airflow/issues/38187): @DEworkerDE ## Provider [sftp: 4.10.1rc1](https://pypi.org/project/apache-airflow-providers-sftp/4.10.1rc1) - [ ] [Fix SFTPSensor.newer_than not working with jinja logical ds/ts expression (#39056)](https://github.com/apache/airflow/pull/39056): @grrolland Linked issues: - [ ] [Linked Issue #36629](https://github.com/apache/airflow/issues/36629): @fpopic ## Provider [snowflake: 5.5.1rc1](https://pypi.org/project/apache-airflow-providers-snowflake/5.5.1rc1) - [ ] [openlineage: remove `openlineage.common` dependencies in Google and Snowflake providers. (#39614)](https://github.com/apache/airflow/pull/39614): @JDarDagran - [ ] [Remove unused `copy_into_postifx` param from docstring (#39454)](https://github.com/apache/airflow/pull/39454): @josh-fell ## Provider [yandex: 3.11.1rc1](https://pypi.org/project/apache-airflow-providers-yandex/3.11.1rc1) - [ ] [ AIP-21: yandexcloud: rename files, emit deprecation warning (#39618)](https://github.com/apache/airflow/pull/39618): @uzhastik - [ ] [yandex provider: bump version for yq http client package (#39548)](https://github.com/apache/airflow/pull/39548): @uzhastik - [ ] [add doc about Yandex Query operator (#39445)](https://github.com/apache/airflow/pull/39445): @uzhastik <!-- NOTE TO RELEASE MANAGER: You can move here the providers that have doc-only changes or for which changes are trivial, and you could assess that they are OK. --> All users involved in the PRs: @pankajastro @dabla @sunank200 @e-galan @shahar1 @boraberke @JDarDagran @0x26res @ellisms @EricGao888 @grrolland @danielbe11 @tatiana @kacpermuda @gaurav7261 @gopidesupavan @mobuchowski @Lee-W @eladkal ### Committer - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
