eladkal opened a new issue, #39346: URL: https://github.com/apache/airflow/issues/39346
### Body I have a kind request for all the contributors to the latest provider packages release. Could you please help us to test the RC versions of the providers? The guidelines on how to test providers can be found in [Verify providers by contributors](https://github.com/apache/airflow/blob/main/dev/README_RELEASE_PROVIDER_PACKAGES.md#verify-the-release-candidate-by-contributors) Let us know in the comment, whether the issue is addressed. Those are providers that require testing as there were some substantial changes introduced: ## Provider [airbyte: 3.8.0rc1](https://pypi.org/project/apache-airflow-providers-airbyte/3.8.0rc1) - [ ] [fix(airbyte/hooks): add schema and port to prevent InvalidURL error (#38860)](https://github.com/apache/airflow/pull/38860): @josix Linked issues: - [ ] [Linked Issue #37449](https://github.com/apache/airflow/issues/37449): @henriquemeloo ## Provider [amazon: 8.21.0rc1](https://pypi.org/project/apache-airflow-providers-amazon/8.21.0rc1) - [ ] [Added logging_config,snapstart,ephemeral_storage parameters to aws lambdacreatefunctionoperator (#39300)](https://github.com/apache/airflow/pull/39300): @gopidesupavan - [ ] [Fix bug in GlueJobOperator where consecutive runs fail when a local script file is used (#38960)](https://github.com/apache/airflow/pull/38960): @moritzsanne Linked issues: - [ ] [Linked Issue #38959](https://github.com/apache/airflow/issues/38959): @moritzsanne - [ ] [Update `is_authorized_custom_view` from auth manager to handle custom actions (#39167)](https://github.com/apache/airflow/pull/39167): @vincbeck Linked issues: - [ ] [Linked Issue #39144](https://github.com/apache/airflow/issues/39144): @jedcunningham - [ ] [Updated logic to allow AWS Batch Hook `get_job_description` retries to be more effective (#38998)](https://github.com/apache/airflow/pull/38998): @shahar1 Linked issues: - [ ] [Linked Issue #37552](https://github.com/apache/airflow/pull/37552): @evgenyslab - [ ] [Amazon Bedrock - Model Throughput Provisioning (#38850)](https://github.com/apache/airflow/pull/38850): @ferruzzi - [ ] [Adding MSGraphOperator in Microsoft Azure provider (#38111)](https://github.com/apache/airflow/pull/38111): @dabla - [ ] [Allow importing the aws executors with a shorter path (#39093)](https://github.com/apache/airflow/pull/39093): @o-nikolas Linked issues: - [ ] [Linked Issue #38524](https://github.com/apache/airflow/pull/38524): @LipuFei - [ ] [Remove flag from AWS auth manager to use it (#39033)](https://github.com/apache/airflow/pull/39033): @vincbeck ## Provider [apache.beam: 5.7.0rc1](https://pypi.org/project/apache-airflow-providers-apache-beam/5.7.0rc1) - [ ] [Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator (#38716)](https://github.com/apache/airflow/pull/38716): @zstrathe Linked issues: - [ ] [Linked Issue #38713](https://github.com/apache/airflow/issues/38713): @zstrathe ## Provider [apache.livy: 3.8.0rc1](https://pypi.org/project/apache-airflow-providers-apache-livy/3.8.0rc1) - [ ] [Fix bug in LivyOperator when its trigger times out (#38916)](https://github.com/apache/airflow/pull/38916): @mateuslatrova - [ ] [Fix 'polling_interval' parameter docs in LivyOperator (#38979)](https://github.com/apache/airflow/pull/38979): @mateuslatrova - [ ] [Update docstring `LivyOperator` retry_args and deferrable docs (#39266)](https://github.com/apache/airflow/pull/39266): @boraberke Linked issues: - [ ] [Linked Issue #29047](https://github.com/apache/airflow/pull/29047): @sunank200 ## Provider [apache.spark: 4.8.0rc2](https://pypi.org/project/apache-airflow-providers-apache-spark/4.8.0rc2) - [ ] [Rename SparkSubmitOperator argument queue as yarn_queue (#38852)](https://github.com/apache/airflow/pull/38852): @pateash Linked issues: - [ ] [Linked Issue #38461](https://github.com/apache/airflow/issues/38461): @djuarezg - [ ] [Linked Issue #35911](https://github.com/apache/airflow/issues/35911): @pateash ## Provider [cncf.kubernetes: 8.2.0rc1](https://pypi.org/project/apache-airflow-providers-cncf-kubernetes/8.2.0rc1) - [ ] [Add missing informative logs in KPO trigger until container has not finished (#37546)](https://github.com/apache/airflow/pull/37546): @pankajkoti - [ ] [Fix templated env_vars field in `KubernetesPodOperator` to allow for compatibility with `XComArgs` (#39139)](https://github.com/apache/airflow/pull/39139): @nyoungstudios - [ ] [Fix SparkKubernetesOperator when using initContainers (#38119)](https://github.com/apache/airflow/pull/38119): @ShelRoman - [ ] [Refresh properties on KubernetesPodOperator when k8s fails due to token expiration (#39325)](https://github.com/apache/airflow/pull/39325): @paolo-moriello - [ ] [Remove unnecessary validation from cncf provider. (#39238)](https://github.com/apache/airflow/pull/39238): @VShkaberda - [ ] [Moves airflow import in deprecated pod_generator to local (#39062)](https://github.com/apache/airflow/pull/39062): @potiuk Linked issues: - [ ] [Linked Issue #31442](https://github.com/apache/airflow/issues/31442): @potiuk - [ ] [KPO xcom sidecar PodDefault usage (#38951)](https://github.com/apache/airflow/pull/38951): @jedcunningham ## Provider [common.sql: 1.13.0rc1](https://pypi.org/project/apache-airflow-providers-common-sql/1.13.0rc1) - [ ] [Add `sqlalchemy_url` property to `DbApiHook` class (#38871)](https://github.com/apache/airflow/pull/38871): @rawwar Linked issues: - [ ] [Linked Issue #38195](https://github.com/apache/airflow/issues/38195): @Taragolis - [ ] [Always use the executemany method when inserting rows in DbApiHook as it's way much faster (#38715)](https://github.com/apache/airflow/pull/38715): @dabla - [ ] [Fix `DbApiHook.insert_rows` when `rows` is a generator (#38972)](https://github.com/apache/airflow/pull/38972): @vincbeck - [ ] [Fix `update-common-sql-api-stubs` pre-commit check (#38915)](https://github.com/apache/airflow/pull/38915): @Taragolis - [ ] [openlineage, snowflake: do not run external queries for Snowflake when (#39113)](https://github.com/apache/airflow/pull/39113): @mobuchowski ## Provider [databricks: 6.4.0rc1](https://pypi.org/project/apache-airflow-providers-databricks/6.4.0rc1) - [ ] [Add `DatabricksNotebookOperator` (#39178)](https://github.com/apache/airflow/pull/39178): @pankajkoti - [ ] [Add notification settings paramaters (#39175)](https://github.com/apache/airflow/pull/39175): @SubhamSinghal - [ ] [[FEAT] raise exception with main notebook error in DatabricksRunNowDeferrableOperator (#39110)](https://github.com/apache/airflow/pull/39110): @gaurav7261 ## Provider [datadog: 3.6.0rc1](https://pypi.org/project/apache-airflow-providers-datadog/3.6.0rc1) - [ ] [Add comment about versions updated by release manager (#37488)](https://github.com/apache/airflow/pull/37488): @potiuk - [ ] [Add docs for RC2 wave of providers for 2nd round of Jan 2024 (#37019)](https://github.com/apache/airflow/pull/37019): @potiuk ## Provider [dbt.cloud: 3.8.0rc1](https://pypi.org/project/apache-airflow-providers-dbt-cloud/3.8.0rc1) - [ ] [fix wrong payload set when reuse_existing_run set to True in DbtCloudRunJobOperator (#39271)](https://github.com/apache/airflow/pull/39271): @Lee-W - [ ] [migrate to dbt v3 api for project endpoints (#39214)](https://github.com/apache/airflow/pull/39214): @Lee-W ## Provider [fab: 1.1.0rc1](https://pypi.org/project/apache-airflow-providers-fab/1.1.0rc1) - [ ] [Remove plugins permissions from Viewer role (#39254)](https://github.com/apache/airflow/pull/39254): @csp33 Linked issues: - [ ] [Linked Issue #39218](https://github.com/apache/airflow/issues/39218): @RocFang - [ ] [Update `is_authorized_custom_view` from auth manager to handle custom actions (#39167)](https://github.com/apache/airflow/pull/39167): @vincbeck Linked issues: - [ ] [Linked Issue #39144](https://github.com/apache/airflow/issues/39144): @jedcunningham ## Provider [google: 10.18.0rc1](https://pypi.org/project/apache-airflow-providers-google/10.18.0rc1) - [ ] [Add templated fields for Google Cloud Generative AI Operators (#39174)](https://github.com/apache/airflow/pull/39174): @CYarros10 - [ ] [Add logic to handle on_kill for BigQueryInsertJobOperator when deferrable=True (#38912)](https://github.com/apache/airflow/pull/38912): @sunank200 - [ ] [Create CloudComposerRunAirflowCLICommandOperator operator (#38965)](https://github.com/apache/airflow/pull/38965): @MaksYermak - [ ] [Add the deferrable mode to the Dataflow sensors (#37693)](https://github.com/apache/airflow/pull/37693): @e-galan - [ ] [Deferrable mode for Custom Training Job operators (#38584)](https://github.com/apache/airflow/pull/38584): @e-galan - [ ] [Enhancement for SSL-support in CloudSQLExecuteQueryOperator (#38894)](https://github.com/apache/airflow/pull/38894): @moiseenkov - [ ] [Create GKESuspendJobOperator and GKEResumeJobOperator operators (#38677)](https://github.com/apache/airflow/pull/38677): @MaksYermak - [ ] [Add support for role arn for aws creds in storage transfer job operators (#38911)](https://github.com/apache/airflow/pull/38911): @wlinamchurch - [ ] [Fix deferrable mode for DataflowTemplatedJobStartOperator and DataflowStartFlexTemplateOperator (#39018)](https://github.com/apache/airflow/pull/39018): @e-galan - [ ] [Fix batching for BigQueryToPostgresOperator (#39233)](https://github.com/apache/airflow/pull/39233): @moiseenkov - [ ] [Fix DataprocSubmitJobOperator in deferrable mode=True when task is marked as failed. (#39230)](https://github.com/apache/airflow/pull/39230): @sunank200 - [ ] [Fix GCSObjectExistenceSensor operator to return the same XCOM value in deferrable and non-deferrable mode (#39206)](https://github.com/apache/airflow/pull/39206): @VladaZakharova - [ ] [Bugfix BigQueryToMsSqlOperator (#39171)](https://github.com/apache/airflow/pull/39171): @moiseenkov - [ ] [fix: add retry logic in case of google auth refresh credential error (#38961)](https://github.com/apache/airflow/pull/38961): @dondaum Linked issues: - [ ] [Linked Issue #38532](https://github.com/apache/airflow/issues/38532): @nathadfield - [ ] [fix: BigQueryCheckOperator skip value and error check in deferrable mode (#38408)](https://github.com/apache/airflow/pull/38408): @kacpermuda Linked issues: - [ ] [Linked Issue #37885](https://github.com/apache/airflow/issues/37885): @kacpermuda - [ ] [[OpenLineage] Fix datasets in GCSDeleteObjectsOperator (#39059)](https://github.com/apache/airflow/pull/39059): @kacpermuda Linked issues: - [ ] [Linked Issue #35838](https://github.com/apache/airflow/pull/35838): @kacpermuda - [ ] [[OpenLineage] Use prefixes instead of file paths for datasets in GCSToGCSOperator (#39058)](https://github.com/apache/airflow/pull/39058): @kacpermuda Linked issues: - [ ] [Linked Issue #31350](https://github.com/apache/airflow/pull/31350): @mobuchowski - [ ] [[OpenLineage] Fix datasets in GCSTimeSpanFileTransformOperator (#39064)](https://github.com/apache/airflow/pull/39064): @kacpermuda Linked issues: - [ ] [Linked Issue #35838](https://github.com/apache/airflow/pull/35838): @kacpermuda - [ ] [Improve DataprocCreateClusterOperator in Triggers for Enhanced Error Handling and Resource Cleanup (#39130)](https://github.com/apache/airflow/pull/39130): @sunank200 - [ ] [Adding MSGraphOperator in Microsoft Azure provider (#38111)](https://github.com/apache/airflow/pull/38111): @dabla - [ ] [Apply PROVIDE_PROJECT_ID mypy workaround across Google provider (#39129)](https://github.com/apache/airflow/pull/39129): @potiuk - [ ] [Handle ImportError for KubernetesDeleteJobOperator in google provider (#39036)](https://github.com/apache/airflow/pull/39036): @vatsrahul1001 Linked issues: - [ ] [Linked Issue #37793](https://github.com/apache/airflow/pull/37793): @MaksYermak ## Provider [hashicorp: 3.7.0rc2](https://pypi.org/project/apache-airflow-providers-hashicorp/3.7.0rc2) - [ ] [Add assume_role_kwargs in hashicorp backend config (#39279)](https://github.com/apache/airflow/pull/39279): @pankajastro - [ ] [Expose AWS IAM missing param in Hashicorp secret (#38536)](https://github.com/apache/airflow/pull/38536): @pankajastro ## Provider [http: 4.11.0rc1](https://pypi.org/project/apache-airflow-providers-http/4.11.0rc1) - [ ] [Allow trust env parameter to be defined in extra options of HTTP Connection (#39161)](https://github.com/apache/airflow/pull/39161): @dabla ## Provider [jdbc: 4.3.0rc1](https://pypi.org/project/apache-airflow-providers-jdbc/4.3.0rc1) - [ ] [Suppress jaydebeapi.Error when setAutoCommit or getAutoCommit is unsupported by JDBC driver (#38707)](https://github.com/apache/airflow/pull/38707): @dabla - [ ] [Add stacklevel into the `suppress_and_warn` warning (#39263)](https://github.com/apache/airflow/pull/39263): @Taragolis ## Provider [microsoft.azure: 10.1.0rc1](https://pypi.org/project/apache-airflow-providers-microsoft-azure/10.1.0rc1) - [ ] [add dns_config and diagnostics parameters to AzureContainerInstancesOperator (#39156)](https://github.com/apache/airflow/pull/39156): @Kylea650 Linked issues: - [ ] [Linked Issue #39149](https://github.com/apache/airflow/issues/39149): @Kylea650 - [ ] [Add stacklevel into the `AzureSynapsePipelineHook` deprecation warnings (#39192)](https://github.com/apache/airflow/pull/39192): @Taragolis - [ ] [Adding MSGraphOperator in Microsoft Azure provider (#38111)](https://github.com/apache/airflow/pull/38111): @dabla - [ ] [Pass through more parameters to AzureBlobFileSystem to mirror handlin… (#38747)](https://github.com/apache/airflow/pull/38747): @tomrutter Linked issues: - [ ] [Linked Issue #38746](https://github.com/apache/airflow/issues/38746): @tomrutter - [ ] [Implement run-method on KiotaRequestAdapterHook and move logic away from triggerer to hook (#39237)](https://github.com/apache/airflow/pull/39237): @dabla - [ ] [Fix: Only quote the keys of the query_parameters in MSGraphOperator (#39207)](https://github.com/apache/airflow/pull/39207): @dabla ## Provider [odbc: 4.6.0rc1](https://pypi.org/project/apache-airflow-providers-odbc/4.6.0rc1) - [ ] [Always use the executemany method when inserting rows in DbApiHook as it's way much faster (#38715)](https://github.com/apache/airflow/pull/38715): @dabla ## Provider [openai: 1.2.0rc1](https://pypi.org/project/apache-airflow-providers-openai/1.2.0rc1) - [ ] [OpenAI Chat & Assistant hook functions (#38736)](https://github.com/apache/airflow/pull/38736): @nathadfield - [ ] [OpenAI Files & Vector Store Hooks (#39248)](https://github.com/apache/airflow/pull/39248): @nathadfield ## Provider [openlineage: 1.7.1rc1](https://pypi.org/project/apache-airflow-providers-openlineage/1.7.1rc1) - [ ] [openlineage, snowflake: do not run external queries for Snowflake when (#39113)](https://github.com/apache/airflow/pull/39113): @mobuchowski ## Provider [pagerduty: 3.7.0rc2](https://pypi.org/project/apache-airflow-providers-pagerduty/3.7.0rc2) - [ ] [Cleanup PagerdutyNotifier __init__ (#39145)](https://github.com/apache/airflow/pull/39145): @DavidTraina ## Provider [pinecone: 2.0.0rc1](https://pypi.org/project/apache-airflow-providers-pinecone/2.0.0rc1) - [ ] [Pinecone provider support for `pinecone-client`>=3 (#37307)](https://github.com/apache/airflow/pull/37307): @rawwar Linked issues: - [ ] [Linked Issue #37158](https://github.com/apache/airflow/issues/37158): @Taragolis - [ ] [Bump ruff to 0.3.3 (#38240)](https://github.com/apache/airflow/pull/38240): @hussein-awala - [ ] [Add comment about versions updated by release manager (#37488)](https://github.com/apache/airflow/pull/37488): @potiuk ## Provider [postgres: 5.11.0rc1](https://pypi.org/project/apache-airflow-providers-postgres/5.11.0rc1) - [ ] [Fix schema assigment in PostgresOperator (#39264)](https://github.com/apache/airflow/pull/39264): @eladkal - [ ] [Always use the executemany method when inserting rows in DbApiHook as it's way much faster (#38715)](https://github.com/apache/airflow/pull/38715): @dabla ## Provider [qdrant: 1.1.0rc1](https://pypi.org/project/apache-airflow-providers-qdrant/1.1.0rc1) - [ ] [Test connection in Qdrant provider (#39273)](https://github.com/apache/airflow/pull/39273): @Anush008 ## Provider [salesforce: 5.7.0rc2](https://pypi.org/project/apache-airflow-providers-salesforce/5.7.0rc2) - [ ] [update simple-salesforce type hints to support 1.12.6 (#39047)](https://github.com/apache/airflow/pull/39047): @hussein-awala ## Provider [slack: 8.7.0rc1](https://pypi.org/project/apache-airflow-providers-slack/8.7.0rc1) - [ ] [Use `upload_files_v2` Slack SDK method by default in Slack Operators (#39340)](https://github.com/apache/airflow/pull/39340): @Taragolis ## Provider [snowflake: 5.5.0rc1](https://pypi.org/project/apache-airflow-providers-snowflake/5.5.0rc1) - [ ] [openlineage, snowflake: do not run external queries for Snowflake when (#39113)](https://github.com/apache/airflow/pull/39113): @mobuchowski ## Provider [teradata: 2.1.0rc1](https://pypi.org/project/apache-airflow-providers-teradata/2.1.0rc1) - [ ] [Always use the executemany method when inserting rows in DbApiHook as it's way much faster (#38715)](https://github.com/apache/airflow/pull/38715): @dabla <!-- NOTE TO RELEASE MANAGER: You can move here the providers that have doc-only changes or for which changes are trivial, and you could assess that they are OK. --> All users involved in the PRs: @mobuchowski @josix @mateuslatrova @kacpermuda @dabla @Anush008 @jedcunningham @boraberke @pankajastro @Taragolis @gopidesupavan @hussein-awala @wlinamchurch @nathadfield @pankajkoti @potiuk @Kylea650 ### Committer - [X] I acknowledge that I am a maintainer/committer of the Apache Airflow project. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
