This is an automated email from the ASF dual-hosted git repository.
mobuchowski pushed a change to branch listener-task-timeout
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 460fadf1b9 local task job: add timeout, to not kill
on_task_instance_success listener prematurely
add 9dd77520be Introduce Amazon Comprehend Service (#39592)
add 6afc75fcce Add `breeze generate-migration-file` command (#39632)
add a7960a1e16 Prevent generation of empty revision files by alembic
autogenerate (#39634)
add 610747d25a Add timeout when watching pod events in k8s executor
(#39551)
add e0dd075d1b AIP-21: yandexcloud: rename files, emit deprecation
warning (#39618)
add b0e3915316 Remove mysql/postgres from extras before determining the
installation_command_flags (#39610)
add 287c1887e9 feat: K8S 1.30 support & kind 0.23.0 (#39631)
add 9284dc5391 Amazon Bedrock - Retrieve and RetrieveAndGenerate (#39500)
add 81a82d8481 Run unit tests for Providers with airflow installed as
package. (#39513)
add 07c40bd78a Support failing tasks stuck in queued for hybrid executors
(#39624)
add 9ea78d9d72 Fix default value for aws batch operator retry strategy
(#39608)
add cba653ee23 Update contributing doc with breeze generate-migration-file
command (#39655)
add 435ba144d1 Doc fix: Remove misplaced backquotes in ``faq.rst`` (#39661)
add 8987b09fd8 Use `compose-spec.json` which do not complaining about
interpolations (#39662)
add 9aae5311dd Revert "Use `compose-spec.json` which do not complaining
about interpolations…" (#39669)
add ea8ed7c43f Improvising high availability field name in hive hook
(#39658)
add f3687b68a6 Sagemaker trigger: pass the job name as part of the event
(#39671)
add 4dce7459d9 Fix #35946: Visible DAG RUN doesn't point to the same dag
run id (#38365)
add 448707c1d8 Add task failed dependencies to details page. (#38449)
add cd0c6a7e77 Add Scarf based telemetry (#39510)
add 4d0c7242bc Small refactor for example_bedrock_knowledge_base.py
(#39672)
add d4a5f4e3a7 Rename `telemetry-collection` to `usage-data-collection`
(#39673)
add a07d799482 Change dataset URI validation to raise warning instead of
error in Airflow 2.9 (#39670)
add e8183a9e8d Add indexes on dag_id column in referencing tables to speed
up deletion of dag records (#39638)
add 74c3fb366e Remove parent_model version suffix if it is passed to
Vertex AI operators (#39640)
add 8d1bd345b2 fix: empty openlineage dataset name for AthenaExtractor
(#39677)
add 4de79a0f6b feat: Add custom provider runtime checks (#39609)
add 0b698a852b Add missing `dag_state_change_process_pool_size` in
`provider.yaml`. (#39674)
add 8b19b78ba5 Update plugins.rst examples to use pyproject.toml over
setup.py (#39665)
add 9ff245591e Fix the argument type of input_vectors in pinecone upsert
(#39688)
add 77a6b4f419 Add args to docker service ContainerSpec (#39464)
add a3ae17f62f Update providers metadata 2024-05-17 (#39691)
add db80e5e21c DbAPiHook: Don't log a warning message if placeholder is
None and make sure warning message is formatted correctly (#39690)
add 70f868e867 Print stderr from helm test failures (#39698)
add a31169bd63 Pin google-cloud-bigquery to < 3.21.0 (#39583)
add 27b3a22e34 Introduce anonymous credentials in GCP base hook (#39695)
add 655bb601fe docs: fix typo in iceberg dags (#39705)
add 79cf7e0609 weaviate remove deprecations (#39707)
add cc9308c710 Reraise exception from strict dataset URI checks (#39719)
add 4ee46b984d Remove `openlineage.common` dependencies in Google and
Snowflake providers. (#39614)
add 4d525aa32d Determine needs_expansion at time of serialization (#39604)
add f509b0a924 More typing in TimeSensor and TimeSensorAsync (#39696)
add a81504e316 chore: Update conf retrieval docstring and adjust pool_size
(#39721)
add b7671ef5ab Re-configure ORM in spawned OpenLineage process in
scheduler. (#39735)
add b41f429612 Remove deprecations eks (#39709)
add 1da7f1f433 Pin requests due to incompatibility with docker-py (#39740)
add ec2e245f0e Fetch served logs also when task attempt is up for retry
and no remote logs available (#39496)
add f18e6340d8 Better typing for BaseOperator `defer` (#39742)
add 9532cc7a6c fix: Prevent error when extractor can't be imported (#39736)
add 49b38719e2 Fix automatic termination issue in `EmrOperator` by
ensuring `waiter_max_attempts` is set for deferrable triggers (#38658)
add 0f717ea5e6 removed stale code (#39744)
add 791f3cfc5c Fix acknowledged functionality in deferrable mode for
PubSubPullSensor (#39711)
add a78ee74b6a bugfix: handle invalid cluster states in
NeptuneStopDbClusterOperator (#38287)
add 60f8ff65d3 remove deprecation jdbc (#39733)
add cee446467f remove deprecation sqlite (#39730)
add cb57a677be removing deprecation postgres (#39706)
add 7b588b4dd9 misc: add comment about remove unused code (#39748)
add 39269d6a71 Limit requests in botocore upgrade test (#39747)
add 61710518ab fix: OpenLineage in FileTransferOperator for Airflow 2.8
(#39755)
add 8d5b43443a Reroute AutoML operator links to Google Translation links
(#39668)
add 2250a1a228 UI changes for DAG Reparsing feature (#39636)
add 9139b22944 Add metrics about task CPU and memory usage (#39650)
add 9e2a230769 Skip `tests/providers/google/cloud/links/test_translate.py`
module in no-pydantic environment (#39763)
add 77b0ed7155 Fix system test example_cloud_memorystore_memcached (#39749)
add 9845e2282d Make elastic search index_pattern more configurable (#38423)
add 3b1ecbc068 Bump version of atlassian-python-api (#39714)
add abd85a993b Mark `TestStandardTaskRunner::test_read_task_utilization`
as db test (#39776)
add 5aad588ef9 Fix BigQueryCursor execute method if the location is
missing (#39659)
add 61e90704c6 Fix example_branch_operator failing in python 3.12 (#39783)
add 57dab1303c typo: wrong OpenLineage facet key in spec (#39782)
add e565cea65c Resolving EMR deprecated warnings (#39743)
add ae8e3d7f99 Fix typo in console message for static checks on
last-commit (#39774)
add 482c0be434 breeze k8s version upgrade (#39785)
add 1734b15e4e Add typing for SqlSensor (#39773)
add 7e2b7ccca0 Add Inoopa to the list of companies using Apache Airflow
(#39800)
add 1f7f1743ee Revert "breeze k8s version upgrade (#39785)" (#39805)
add d525ec5fea Update uv and build dependencies. (#39808)
add b995d9433f Fix `mypy` error about missing attribute in
`TestPodTemplateFile` (#39806)
add 5aee696e44 Limit concurrent downloads by UV (#39810)
add c9c352d0dd Load Provider Plugins in Development (#39723)
add 7163da9a71 Fix MLEngine system test (#39766)
add 4fe55e5992 Deprecate AutoML Tables operators (#39752)
add ed5bcbbf01 Fix mypy errors in "mypy-airflow" tests. (#39816)
add 7663dd5ac3 Fix common.sql mypy errors (#39820)
add ee1df37b83 Avoid setting test constants as pytest module attributes
(#39819)
add a95150362b Fix Google System tests to satisfy MyPy project_id checks
(#39817)
add bfdeb270c7 Fix microsoft conftest mypy errors (#39821)
add 6fb9fb7bf8 Fix MyPy checks in CI to cover also tests folders (#39815)
add bca2930d0e Make _run_raw_task AIP-44 compatible (#38992)
add c2f1739aa0 Fix TIPydantic serialization of MappedOperator (#39288)
add 2b6f8ffc69 Upgrade node version used to compile assets (#39824)
add cae96630e1 Remove 'legacy' definition for `CronDataIntervalTimetable`
(#39780)
add f4c4519f89 Add max_consecutive_failed_dag_runs in API spec (#39830)
add e81cefb8a6 Remove unused backward compatibility _read function in
WasbTaskHandler (#39827)
add 34500f3a2f Prepare docs 3rd wave May 2024 (#39738)
add cd25d1a458 Image used for k8s tests copies airflow sources with
airflow user (#39843)
add 54ed71f9b0 Fix failing main after merging serialization fix (#39845)
add 8a35a6abdc Resolving ECS fargate deprecated warnings (#39834)
add 42e93bfc8b helm version upgrade (#39828)
add 478ac9cb87 Update Iceberg provider name in provider.yaml (#39846)
add d4fe325f84 Adding params. to create_auto_ml_forecasting_training_job
in AutoMl hook (#39767)
add 8fbf466d50 Resolving google bigquery deprecated warnings (#39841)
add e3c31752f1 Resolving mysql deprecated operator warnings (#39725)
add cda4a8604f uv version bump to 0.2.3 (#39848)
add 1fc551b146 Fail image build in case dependencies are not regenerated
(#39847)
add 993053ad3e Also limit concurrent uv downloads when building (#39852)
add 2201154252 Fix long-standing side-effect of test replacing formatter
(#39853)
add df0790d2c5 Remove redundant code from Dockerfile (#39833)
add 9c034abdec Fix openlineage dbt test for openlineage 1.15 (#39857)
add 8615419b77 Quarantine
test_backfill_failed_dag_with_upstream_failed_task (#39859)
add 7ad6b98f58 remove deprecation drill (#39732)
add aec3f80278 Bump docker version to 7.1.0 (#39839)
add ad7cb99fcd Improve typing for allowed/failed_states in
TriggerDagRunOperator (#39855)
add 4dffec4d92 Change type definition for `provider_info_cache` decorator
(#39750)
add d5f81a4e2d Switch AzureDataLakeStorageV2Hook to use
DefaultAzureCredential for managed identity/workload auth (#38497)
add f0677275f9 Add `mssql` integration tests and relocate existing unit
tests (#39831)
add a12a4a5d93 gcs to gcs deprecation removal (#39726)
add 23e03db2d7 Add method to get metadata from GCS blob in GCSHook (#38398)
add 0fe1041cc6 remove deprecations snowflake (#39729)
add 8694b95a52 Remove merch link. URL is broken (#39869)
add 349e43c9d9 ruff version bump 0.4.5 (#39849)
add 7cefd0acac Simplify Integration test execution in CI (#39866)
add 9f0b0258e2 Allow FTPHook to change port number (#39465)
add 53081cd342 Implement amazon s3 to dynamodb transfer operator (#39654)
add 98c5a3a2c6 Add a warning message to KPO to warn of one second interval
logs duplication (#39861)
add 8b3ec451e8 Increase Health Check parameters for mssql integration
(#39877)
add a07256af4b update k8s supported version in docs (#39878)
add 4d6b8fd87b Further increase startup time for mssql (#39882)
add 54b4d7604b Remove DAG Run Add option from FAB view (#39881)
add 483d408041 Upgrade to FAB 4.5.0 (#39851)
add 36562feb8a Run tests for Providers also for Airflow 2.8 (#39606)
add 73e692c63b Disable health check for mssql integration (#39885)
add 7dc2b52436 Use SKIP LOCKED instead of NOWAIT in mini scheduler (#39745)
add e190cff272 fix wrong arguments in read_namespaced_pod_log call (#39874)
add 358830d66c Much simpler way to run provider compatibility tests
(#39883)
add 02ba205cf4 Refactor BashOperator and Bash decorator for consistency
and simplicity (#39871)
add b805a9dba0 Bump Snowflake client driver versions per support policy
(#39886)
add 130e9df10e Add missing test for json api client (#39870)
add 53970a8d1f Handling exception getting logs when pods finish success
(#39296)
add 729bb64827 Attempt to stabilize flaky celery integration test (#39892)
add 22305477bb Use UUIDv7 for OpenLineage runIds (#39889)
add 99d5157583 Filter out for issues vs other links while generating
provider issue content (#39894)
add dad3c59034 Add possibility to collect all provider tests for
compatibility testing (#39888)
add c531e387b2 Add running provider tests against Airflow 2.7 (#39862)
add 1c7fe71fb8 Replace non-functional TOC directive with explanation where
to look (#39901)
add 70121f82f7 Implement `CloudDataTransferServiceRunJobOperator` (#39154)
add c3a5b6edaf Add example DAGs for inlet_events (#39893)
add e0899047ac Add Wikimedia Foundation to list of users (#39900)
add fa47f7424f Use walrus to remove one line of code (#39906)
add d9e7c42cdc Fix typo in example inlet events DAG (#39909)
add 0eb019571c Improve code coverage for CgroupTaskRunner (#39896)
add 5f2ebb312b ECS Overrides for AWS Batch submit_job (#39903)
add aba8def5f3 Add error stacktrace to OpenLineage task event (#39813)
add 1b13cf50b6 Another attempt to stabilize (and limit) integration tests
(#39913)
add a2c09d203e Fetch intermediate log async GKEStartPod (#39348)
add 95e5be23f0 Revert "Improve code coverage for CgroupTaskRunner
(#39896)" (#39915)
add 0b9232e54d Updates to Teradata Provider (#39217)
add d3b278daec Resolve teradata deprecations in tests (#39926)
add b3c8990626 Update providers metadata 2024-05-30 (#39933)
add 960b554494 Resolve yandex deprecations in tests (#39929)
add eba6406053 Improve visibility of links / variables / other configs in
Configuration Reference (#39916)
add 2ecf7fa07d Add DatabricksWorkflowTaskGroup (#39771)
add f04dbec69e Resolve sftp deprecations in tests (#39930)
add f0ea079594 Lower log levels from INFO to DEBUG to reduce log verbosity
- Databricks provider auth (#39941)
add 32e46dde86 local task job: add timeout, to not kill
on_task_instance_success listener prematurely
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (460fadf1b9)
\
N -- N -- N refs/heads/listener-task-timeout (32e46dde86)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/workflows/check-providers.yml | 29 +-
.github/workflows/ci.yml | 9 +-
.github/workflows/integration-tests.yml | 87 +-
.pre-commit-config.yaml | 6 +-
COMMITTERS.rst | 2 +-
CONTRIBUTING.rst | 2 +-
Dockerfile | 45 +-
Dockerfile.ci | 37 +-
INTHEWILD.md | 2 +
PROVIDERS.rst | 2 +-
README.md | 22 +-
airflow/__init__.py | 23 +-
.../endpoints/task_instance_endpoint.py | 66 +-
airflow/api_connexion/openapi/v1.yaml | 89 +
.../api_connexion/schemas/task_instance_schema.py | 14 +
airflow/api_internal/endpoints/rpc_api_endpoint.py | 21 +-
airflow/cli/commands/scheduler_command.py | 3 +
airflow/cli/commands/task_command.py | 13 +-
airflow/config_templates/config.yml | 408 +--
airflow/datasets/__init__.py | 15 +-
airflow/datasets/manager.py | 36 +-
airflow/example_dags/example_bash_decorator.py | 2 +-
airflow/example_dags/example_bash_operator.py | 2 +-
airflow/example_dags/example_branch_operator.py | 4 +-
...event_extra.py => example_inlet_event_extra.py} | 47 +-
airflow/example_dags/example_outlet_event_extra.py | 2 +
airflow/jobs/local_task_job_runner.py | 4 +-
airflow/jobs/scheduler_job_runner.py | 30 +-
airflow/migrations/env.py | 9 +
...dexes_on_dag_id_column_in_referencing_tables.py | 105 +
...=> 0144_2_10_0_add_new_executor_field_to_db.py} | 4 +-
..._10_0_added_dagpriorityparsingrequest_table.py} | 0
airflow/models/abstractoperator.py | 15 +-
airflow/models/baseoperator.py | 4 +-
airflow/models/dag.py | 2 +
airflow/models/dagrun.py | 2 +-
airflow/models/dagwarning.py | 3 +-
airflow/models/dataset.py | 3 +
airflow/models/expandinput.py | 7 +-
airflow/models/mappedoperator.py | 8 +-
airflow/models/operator.py | 26 +-
airflow/models/taskinstance.py | 774 ++++--
airflow/models/xcom.py | 1 +
airflow/operators/trigger_dagrun.py | 35 +-
airflow/providers/MANAGING_PROVIDERS_LIFECYCLE.rst | 2 +-
airflow/providers/airbyte/CHANGELOG.rst | 13 +
airflow/providers/airbyte/__init__.py | 2 +-
airflow/providers/airbyte/provider.yaml | 3 +-
airflow/providers/alibaba/CHANGELOG.rst | 18 +
airflow/providers/alibaba/__init__.py | 2 +-
airflow/providers/alibaba/provider.yaml | 3 +-
airflow/providers/amazon/CHANGELOG.rst | 25 +
airflow/providers/amazon/__init__.py | 2 +-
.../amazon/aws/executors/ecs/ecs_executor.py | 1 +
airflow/providers/amazon/aws/hooks/batch_client.py | 3 +
airflow/providers/amazon/aws/hooks/bedrock.py | 20 +
.../aws/hooks/comprehend.py} | 20 +-
airflow/providers/amazon/aws/hooks/dynamodb.py | 35 +-
airflow/providers/amazon/aws/hooks/neptune.py | 37 +-
airflow/providers/amazon/aws/operators/athena.py | 2 +-
airflow/providers/amazon/aws/operators/batch.py | 12 +-
airflow/providers/amazon/aws/operators/bedrock.py | 220 +-
.../providers/amazon/aws/operators/comprehend.py | 192 ++
airflow/providers/amazon/aws/operators/ecs.py | 10 +-
airflow/providers/amazon/aws/operators/emr.py | 32 +-
airflow/providers/amazon/aws/operators/neptune.py | 149 +-
.../providers/amazon/aws/operators/sagemaker.py | 24 +-
airflow/providers/amazon/aws/sensors/comprehend.py | 147 +
airflow/providers/amazon/aws/sensors/emr.py | 8 +
.../amazon/aws/transfers/s3_to_dynamodb.py | 257 ++
.../providers/amazon/aws/triggers/comprehend.py | 61 +
airflow/providers/amazon/aws/triggers/neptune.py | 45 +
airflow/providers/amazon/aws/triggers/sagemaker.py | 2 +-
airflow/providers/amazon/aws/utils/__init__.py | 7 +
.../providers/amazon/aws/waiters/comprehend.json | 49 +
airflow/providers/amazon/aws/waiters/dynamodb.json | 37 +
airflow/providers/amazon/provider.yaml | 25 +-
airflow/providers/apache/beam/CHANGELOG.rst | 18 +
airflow/providers/apache/beam/__init__.py | 2 +-
airflow/providers/apache/beam/provider.yaml | 3 +-
airflow/providers/apache/cassandra/CHANGELOG.rst | 13 +
airflow/providers/apache/cassandra/__init__.py | 2 +-
airflow/providers/apache/cassandra/provider.yaml | 3 +-
airflow/providers/apache/drill/CHANGELOG.rst | 13 +
airflow/providers/apache/drill/__init__.py | 2 +-
airflow/providers/apache/drill/operators/drill.py | 4 -
airflow/providers/apache/drill/provider.yaml | 3 +-
airflow/providers/apache/druid/CHANGELOG.rst | 19 +
airflow/providers/apache/druid/__init__.py | 2 +-
airflow/providers/apache/druid/provider.yaml | 3 +-
airflow/providers/apache/flink/CHANGELOG.rst | 14 +
airflow/providers/apache/flink/__init__.py | 2 +-
airflow/providers/apache/flink/provider.yaml | 3 +-
airflow/providers/apache/hdfs/CHANGELOG.rst | 13 +
airflow/providers/apache/hdfs/__init__.py | 2 +-
airflow/providers/apache/hdfs/provider.yaml | 3 +-
airflow/providers/apache/hive/CHANGELOG.rst | 14 +
airflow/providers/apache/hive/__init__.py | 2 +-
airflow/providers/apache/hive/hooks/hive.py | 6 +-
airflow/providers/apache/hive/provider.yaml | 3 +-
airflow/providers/apache/iceberg/provider.yaml | 2 +-
airflow/providers/apache/impala/CHANGELOG.rst | 13 +
airflow/providers/apache/impala/__init__.py | 2 +-
airflow/providers/apache/impala/provider.yaml | 3 +-
airflow/providers/apache/kafka/CHANGELOG.rst | 18 +
airflow/providers/apache/kafka/__init__.py | 2 +-
airflow/providers/apache/kafka/provider.yaml | 3 +-
airflow/providers/apache/kylin/CHANGELOG.rst | 13 +
airflow/providers/apache/kylin/__init__.py | 2 +-
airflow/providers/apache/kylin/provider.yaml | 3 +-
airflow/providers/apache/livy/CHANGELOG.rst | 14 +
airflow/providers/apache/livy/__init__.py | 2 +-
airflow/providers/apache/livy/provider.yaml | 3 +-
airflow/providers/apache/pig/CHANGELOG.rst | 13 +
airflow/providers/apache/pig/__init__.py | 2 +-
airflow/providers/apache/pig/provider.yaml | 3 +-
airflow/providers/apache/pinot/CHANGELOG.rst | 13 +
airflow/providers/apache/pinot/__init__.py | 2 +-
airflow/providers/apache/pinot/provider.yaml | 3 +-
airflow/providers/apache/spark/CHANGELOG.rst | 13 +
airflow/providers/apache/spark/__init__.py | 2 +-
airflow/providers/apache/spark/provider.yaml | 3 +-
airflow/providers/apprise/CHANGELOG.rst | 18 +
airflow/providers/apprise/__init__.py | 2 +-
airflow/providers/apprise/provider.yaml | 3 +-
airflow/providers/arangodb/CHANGELOG.rst | 13 +
airflow/providers/arangodb/__init__.py | 2 +-
airflow/providers/arangodb/provider.yaml | 3 +-
airflow/providers/asana/CHANGELOG.rst | 13 +
airflow/providers/asana/__init__.py | 2 +-
airflow/providers/asana/provider.yaml | 3 +-
airflow/providers/atlassian/jira/CHANGELOG.rst | 15 +
airflow/providers/atlassian/jira/__init__.py | 2 +-
airflow/providers/atlassian/jira/provider.yaml | 9 +-
airflow/providers/celery/CHANGELOG.rst | 16 +
airflow/providers/celery/__init__.py | 2 +-
.../providers/celery/executors/celery_executor.py | 1 +
airflow/providers/celery/provider.yaml | 3 +-
airflow/providers/cloudant/CHANGELOG.rst | 13 +
airflow/providers/cloudant/__init__.py | 2 +-
airflow/providers/cloudant/provider.yaml | 3 +-
airflow/providers/cncf/kubernetes/CHANGELOG.rst | 27 +
airflow/providers/cncf/kubernetes/__init__.py | 2 +-
.../executors/kubernetes_executor_utils.py | 14 +-
airflow/providers/cncf/kubernetes/operators/pod.py | 23 +-
airflow/providers/cncf/kubernetes/provider.yaml | 3 +-
.../providers/cncf/kubernetes/utils/pod_manager.py | 2 +-
airflow/providers/cohere/CHANGELOG.rst | 13 +
airflow/providers/cohere/__init__.py | 2 +-
airflow/providers/cohere/provider.yaml | 3 +-
airflow/providers/common/io/CHANGELOG.rst | 18 +
airflow/providers/common/io/__init__.py | 2 +-
.../providers/common/io/operators/file_transfer.py | 14 +-
airflow/providers/common/io/provider.yaml | 3 +-
airflow/providers/common/sql/CHANGELOG.rst | 25 +
airflow/providers/common/sql/__init__.py | 2 +-
airflow/providers/common/sql/hooks/sql.py | 17 +-
airflow/providers/common/sql/provider.yaml | 3 +-
airflow/providers/common/sql/sensors/sql.py | 40 +-
airflow/providers/common/sql/sensors/sql.pyi | 17 +-
airflow/providers/databricks/CHANGELOG.rst | 24 +
airflow/providers/databricks/__init__.py | 2 +-
airflow/providers/databricks/hooks/databricks.py | 18 +
.../providers/databricks/hooks/databricks_base.py | 16 +-
.../providers/databricks/operators/databricks.py | 143 +-
.../databricks/operators/databricks_workflow.py | 312 +++
airflow/providers/databricks/provider.yaml | 13 +-
airflow/providers/datadog/CHANGELOG.rst | 13 +
airflow/providers/datadog/__init__.py | 2 +-
airflow/providers/datadog/provider.yaml | 3 +-
airflow/providers/dbt/cloud/CHANGELOG.rst | 27 +-
airflow/providers/dbt/cloud/__init__.py | 2 +-
airflow/providers/dbt/cloud/provider.yaml | 3 +-
airflow/providers/dingding/CHANGELOG.rst | 13 +
airflow/providers/dingding/__init__.py | 2 +-
airflow/providers/dingding/provider.yaml | 3 +-
airflow/providers/discord/CHANGELOG.rst | 13 +
airflow/providers/discord/__init__.py | 2 +-
airflow/providers/discord/provider.yaml | 3 +-
airflow/providers/docker/CHANGELOG.rst | 21 +
airflow/providers/docker/__init__.py | 2 +-
airflow/providers/docker/operators/docker_swarm.py | 19 +
airflow/providers/docker/provider.yaml | 5 +-
airflow/providers/elasticsearch/CHANGELOG.rst | 19 +
airflow/providers/elasticsearch/__init__.py | 2 +-
.../providers/elasticsearch/log/es_task_handler.py | 30 +-
airflow/providers/elasticsearch/provider.yaml | 12 +-
airflow/providers/exasol/CHANGELOG.rst | 13 +
airflow/providers/exasol/__init__.py | 2 +-
airflow/providers/exasol/provider.yaml | 3 +-
airflow/providers/fab/CHANGELOG.rst | 15 +
airflow/providers/fab/__init__.py | 2 +-
.../fab/auth_manager/security_manager/override.py | 20 +-
airflow/providers/fab/provider.yaml | 5 +-
airflow/providers/facebook/CHANGELOG.rst | 13 +
airflow/providers/facebook/__init__.py | 2 +-
airflow/providers/facebook/provider.yaml | 3 +-
airflow/providers/ftp/CHANGELOG.rst | 13 +
airflow/providers/ftp/__init__.py | 2 +-
airflow/providers/ftp/hooks/ftp.py | 13 +-
airflow/providers/ftp/provider.yaml | 3 +-
airflow/providers/github/CHANGELOG.rst | 13 +
airflow/providers/github/__init__.py | 2 +-
airflow/providers/github/provider.yaml | 3 +-
airflow/providers/google/CHANGELOG.rst | 39 +
airflow/providers/google/__init__.py | 2 +-
airflow/providers/google/cloud/hooks/bigquery.py | 21 +-
.../cloud/hooks/cloud_storage_transfer_service.py | 26 +
airflow/providers/google/cloud/hooks/gcs.py | 21 +
.../google/cloud/hooks/vertex_ai/auto_ml.py | 8 +
airflow/providers/google/cloud/links/automl.py | 38 +
airflow/providers/google/cloud/links/translate.py | 180 ++
.../google/cloud/log/stackdriver_task_handler.py | 3 +-
.../cloud/openlineage/BigQueryErrorRunFacet.json | 30 +
.../cloud/openlineage/BigQueryJobRunFacet.json | 37 +
.../google/cloud/openlineage}/__init__.py | 0
.../providers/google/cloud/openlineage/utils.py | 388 +++
airflow/providers/google/cloud/operators/automl.py | 138 +-
.../providers/google/cloud/operators/bigquery.py | 63 +-
.../operators/cloud_storage_transfer_service.py | 76 +
.../google/cloud/operators/kubernetes_engine.py | 15 +-
.../google/cloud/operators/vertex_ai/auto_ml.py | 11 +
.../google/cloud/operators/vertex_ai/custom_job.py | 6 +
.../google/cloud/transfers/bigquery_to_gcs.py | 2 +-
.../google/cloud/transfers/gcs_to_bigquery.py | 2 +-
.../google/cloud/triggers/kubernetes_engine.py | 2 +
airflow/providers/google/cloud/triggers/pubsub.py | 19 +-
.../google/cloud/utils/credentials_provider.py | 73 +-
.../providers/google/cloud/utils/openlineage.py | 81 -
.../providers/google/common/hooks/base_google.py | 16 +-
airflow/providers/google/provider.yaml | 14 +-
airflow/providers/grpc/CHANGELOG.rst | 13 +
airflow/providers/grpc/__init__.py | 2 +-
airflow/providers/grpc/provider.yaml | 3 +-
airflow/providers/hashicorp/CHANGELOG.rst | 18 +
airflow/providers/hashicorp/__init__.py | 2 +-
airflow/providers/hashicorp/provider.yaml | 3 +-
airflow/providers/http/CHANGELOG.rst | 13 +
airflow/providers/http/__init__.py | 2 +-
airflow/providers/http/provider.yaml | 3 +-
airflow/providers/imap/CHANGELOG.rst | 13 +
airflow/providers/imap/__init__.py | 2 +-
airflow/providers/imap/provider.yaml | 3 +-
airflow/providers/influxdb/CHANGELOG.rst | 13 +
airflow/providers/influxdb/__init__.py | 2 +-
airflow/providers/influxdb/provider.yaml | 3 +-
airflow/providers/jdbc/CHANGELOG.rst | 13 +
airflow/providers/jdbc/__init__.py | 2 +-
airflow/providers/jdbc/provider.yaml | 3 +-
airflow/providers/jenkins/CHANGELOG.rst | 13 +
airflow/providers/jenkins/__init__.py | 2 +-
airflow/providers/jenkins/provider.yaml | 3 +-
.../microsoft/azure/.latest-doc-only-change.txt | 2 +-
airflow/providers/microsoft/azure/CHANGELOG.rst | 9 +
airflow/providers/microsoft/azure/__init__.py | 2 +-
.../providers/microsoft/azure/hooks/data_lake.py | 7 +-
.../microsoft/azure/log/wasb_task_handler.py | 28 +-
airflow/providers/microsoft/azure/provider.yaml | 3 +-
airflow/providers/microsoft/mssql/CHANGELOG.rst | 19 +
airflow/providers/microsoft/mssql/__init__.py | 2 +-
airflow/providers/microsoft/mssql/hooks/mssql.py | 9 +-
airflow/providers/microsoft/mssql/provider.yaml | 3 +-
airflow/providers/microsoft/psrp/CHANGELOG.rst | 13 +
airflow/providers/microsoft/psrp/__init__.py | 2 +-
airflow/providers/microsoft/psrp/provider.yaml | 3 +-
airflow/providers/microsoft/winrm/CHANGELOG.rst | 13 +
airflow/providers/microsoft/winrm/__init__.py | 2 +-
airflow/providers/microsoft/winrm/provider.yaml | 3 +-
airflow/providers/mongo/CHANGELOG.rst | 13 +
airflow/providers/mongo/__init__.py | 2 +-
airflow/providers/mongo/provider.yaml | 3 +-
airflow/providers/mysql/CHANGELOG.rst | 13 +
airflow/providers/mysql/__init__.py | 2 +-
airflow/providers/mysql/operators/mysql.py | 2 +-
airflow/providers/mysql/provider.yaml | 3 +-
airflow/providers/neo4j/CHANGELOG.rst | 13 +
airflow/providers/neo4j/__init__.py | 2 +-
airflow/providers/neo4j/provider.yaml | 3 +-
airflow/providers/odbc/CHANGELOG.rst | 13 +
airflow/providers/odbc/__init__.py | 2 +-
airflow/providers/odbc/provider.yaml | 3 +-
airflow/providers/openai/CHANGELOG.rst | 13 +
airflow/providers/openai/__init__.py | 2 +-
airflow/providers/openai/provider.yaml | 3 +-
airflow/providers/openfaas/CHANGELOG.rst | 13 +
airflow/providers/openfaas/__init__.py | 2 +-
airflow/providers/openfaas/provider.yaml | 3 +-
airflow/providers/openlineage/CHANGELOG.rst | 37 +-
airflow/providers/openlineage/__init__.py | 2 +-
airflow/providers/openlineage/conf.py | 25 +-
.../providers/openlineage/extractors/manager.py | 21 +-
airflow/providers/openlineage/plugins/adapter.py | 144 +-
airflow/providers/openlineage/plugins/listener.py | 97 +-
airflow/providers/openlineage/plugins/macros.py | 2 +-
.../providers/openlineage/plugins/openlineage.py | 3 +
airflow/providers/openlineage/provider.yaml | 15 +-
airflow/providers/openlineage/utils/utils.py | 17 +-
airflow/providers/opensearch/CHANGELOG.rst | 13 +
airflow/providers/opensearch/__init__.py | 2 +-
airflow/providers/opensearch/provider.yaml | 3 +-
airflow/providers/opsgenie/CHANGELOG.rst | 13 +
airflow/providers/opsgenie/__init__.py | 2 +-
airflow/providers/opsgenie/provider.yaml | 3 +-
airflow/providers/oracle/CHANGELOG.rst | 13 +
airflow/providers/oracle/__init__.py | 2 +-
airflow/providers/oracle/provider.yaml | 3 +-
airflow/providers/pagerduty/CHANGELOG.rst | 13 +
airflow/providers/pagerduty/__init__.py | 2 +-
airflow/providers/pagerduty/provider.yaml | 3 +-
airflow/providers/papermill/CHANGELOG.rst | 13 +
airflow/providers/papermill/__init__.py | 2 +-
airflow/providers/papermill/provider.yaml | 3 +-
airflow/providers/pgvector/CHANGELOG.rst | 13 +
airflow/providers/pgvector/__init__.py | 2 +-
airflow/providers/pgvector/provider.yaml | 3 +-
airflow/providers/pinecone/CHANGELOG.rst | 2 +
airflow/providers/pinecone/hooks/pinecone.py | 3 +-
airflow/providers/pinecone/operators/pinecone.py | 8 +-
airflow/providers/postgres/CHANGELOG.rst | 18 +
airflow/providers/postgres/__init__.py | 2 +-
airflow/providers/postgres/provider.yaml | 3 +-
airflow/providers/presto/CHANGELOG.rst | 13 +
airflow/providers/presto/__init__.py | 2 +-
airflow/providers/presto/provider.yaml | 3 +-
airflow/providers/qdrant/CHANGELOG.rst | 13 +
airflow/providers/qdrant/__init__.py | 2 +-
airflow/providers/qdrant/provider.yaml | 3 +-
airflow/providers/redis/CHANGELOG.rst | 13 +
airflow/providers/redis/__init__.py | 2 +-
airflow/providers/redis/provider.yaml | 3 +-
airflow/providers/salesforce/CHANGELOG.rst | 13 +
airflow/providers/salesforce/__init__.py | 2 +-
airflow/providers/salesforce/provider.yaml | 3 +-
airflow/providers/samba/CHANGELOG.rst | 13 +
airflow/providers/samba/__init__.py | 2 +-
airflow/providers/samba/provider.yaml | 3 +-
airflow/providers/segment/CHANGELOG.rst | 13 +
airflow/providers/segment/__init__.py | 2 +-
airflow/providers/segment/provider.yaml | 3 +-
airflow/providers/sendgrid/CHANGELOG.rst | 13 +
airflow/providers/sendgrid/__init__.py | 2 +-
airflow/providers/sendgrid/provider.yaml | 3 +-
airflow/providers/sftp/CHANGELOG.rst | 18 +
airflow/providers/sftp/__init__.py | 2 +-
airflow/providers/sftp/provider.yaml | 3 +-
airflow/providers/singularity/CHANGELOG.rst | 13 +
airflow/providers/singularity/__init__.py | 2 +-
airflow/providers/singularity/provider.yaml | 3 +-
airflow/providers/slack/CHANGELOG.rst | 13 +
airflow/providers/slack/__init__.py | 2 +-
airflow/providers/slack/provider.yaml | 3 +-
airflow/providers/smtp/CHANGELOG.rst | 13 +
airflow/providers/smtp/__init__.py | 2 +-
airflow/providers/smtp/provider.yaml | 3 +-
airflow/providers/snowflake/CHANGELOG.rst | 15 +
airflow/providers/snowflake/__init__.py | 2 +-
airflow/providers/snowflake/hooks/snowflake.py | 5 +-
airflow/providers/snowflake/provider.yaml | 7 +-
airflow/providers/snowflake/utils/openlineage.py | 63 +
airflow/providers/sqlite/CHANGELOG.rst | 13 +
airflow/providers/sqlite/__init__.py | 2 +-
airflow/providers/sqlite/provider.yaml | 3 +-
airflow/providers/ssh/CHANGELOG.rst | 13 +
airflow/providers/ssh/__init__.py | 2 +-
airflow/providers/ssh/provider.yaml | 3 +-
airflow/providers/tableau/CHANGELOG.rst | 13 +
airflow/providers/tableau/__init__.py | 2 +-
airflow/providers/tableau/provider.yaml | 3 +-
airflow/providers/telegram/CHANGELOG.rst | 13 +
airflow/providers/telegram/__init__.py | 2 +-
airflow/providers/telegram/provider.yaml | 3 +-
airflow/providers/teradata/CHANGELOG.rst | 13 +
airflow/providers/teradata/__init__.py | 2 +-
airflow/providers/teradata/hooks/teradata.py | 66 +
airflow/providers/teradata/operators/teradata.py | 49 +-
airflow/providers/teradata/provider.yaml | 19 +-
.../teradata/transfers/azure_blob_to_teradata.py | 103 +
.../providers/teradata/transfers/s3_to_teradata.py | 109 +
airflow/providers/trino/CHANGELOG.rst | 13 +
airflow/providers/trino/__init__.py | 2 +-
airflow/providers/trino/provider.yaml | 3 +-
airflow/providers/vertica/CHANGELOG.rst | 13 +
airflow/providers/vertica/__init__.py | 2 +-
airflow/providers/vertica/provider.yaml | 3 +-
airflow/providers/weaviate/CHANGELOG.rst | 13 +
airflow/providers/weaviate/__init__.py | 2 +-
airflow/providers/weaviate/provider.yaml | 3 +-
airflow/providers/yandex/CHANGELOG.rst | 16 +
airflow/providers/yandex/__init__.py | 2 +-
.../hooks/{yandexcloud_dataproc.py => dataproc.py} | 0
.../providers/yandex/hooks/yandexcloud_dataproc.py | 25 +-
airflow/providers/yandex/hooks/yq.py | 15 +-
.../{yandexcloud_dataproc.py => dataproc.py} | 2 +-
.../yandex/operators/yandexcloud_dataproc.py | 521 +---
airflow/providers/yandex/provider.yaml | 10 +-
airflow/providers/zendesk/CHANGELOG.rst | 13 +
airflow/providers/zendesk/__init__.py | 2 +-
airflow/providers/zendesk/provider.yaml | 3 +-
airflow/providers_manager.py | 26 +-
airflow/sensors/date_time.py | 6 +-
airflow/sensors/time_delta.py | 6 +-
airflow/sensors/time_sensor.py | 12 +-
airflow/serialization/pydantic/taskinstance.py | 87 +-
airflow/serialization/serialized_objects.py | 5 +-
airflow/settings.py | 39 +-
airflow/task/task_runner/standard_task_runner.py | 25 +
airflow/ti_deps/deps/trigger_rule_dep.py | 5 +-
airflow/utils/db.py | 2 +-
airflow/utils/log/file_task_handler.py | 8 +-
airflow/utils/task_instance_session.py | 3 +
airflow/utils/usage_data_collection.py | 97 +
airflow/www/static/js/api/index.ts | 2 +
.../www/static/js/api/useTaskFailedDependency.ts | 63 +
airflow/www/static/js/dag/details/Header.tsx | 4 +-
.../details/taskInstance/TaskFailedDependency.tsx | 93 +
.../static/js/dag/details/taskInstance/index.tsx | 13 +
airflow/www/static/js/types/api-generated.ts | 124 +
airflow/www/static/js/utils/index.test.ts | 4 +-
airflow/www/static/js/utils/index.ts | 2 +-
airflow/www/templates/airflow/dag.html | 9 +
airflow/www/templates/airflow/dags.html | 7 +
airflow/www/views.py | 69 +-
contributing-docs/01_roles_in_airflow_project.rst | 2 +-
contributing-docs/02_how_to_communicate.rst | 2 +-
contributing-docs/03_contributors_quick_start.rst | 2 +-
contributing-docs/04_how_to_contribute.rst | 2 +-
contributing-docs/05_pull_requests.rst | 2 +-
contributing-docs/06_development_environments.rst | 2 +-
contributing-docs/07_local_virtualenv.rst | 2 +-
contributing-docs/08_static_code_checks.rst | 2 +-
contributing-docs/10_working_with_git.rst | 2 +-
contributing-docs/11_provider_packages.rst | 2 +-
.../12_airflow_dependencies_and_extras.rst | 2 +-
contributing-docs/13_metadata_database_updates.rst | 10 +-
contributing-docs/14_node_environment_setup.rst | 2 +-
contributing-docs/16_contribution_workflow.rst | 2 +-
contributing-docs/README.rst | 2 +-
.../contributors_quick_start_gitpod.rst | 2 +-
.../contributors_quick_start_pycharm.rst | 2 +-
.../contributors_quick_start_vscode.rst | 2 +-
contributing-docs/testing/docker_compose_tests.rst | 2 +-
contributing-docs/testing/integration_tests.rst | 4 +-
contributing-docs/testing/k8s_tests.rst | 2 +-
contributing-docs/testing/system_tests.rst | 2 +-
contributing-docs/testing/testing_packages.rst | 2 +-
contributing-docs/testing/unit_tests.rst | 684 +++--
dev/breeze/doc/01_installation.rst | 2 +-
dev/breeze/doc/02_customizing.rst | 2 +-
dev/breeze/doc/03_developer_tasks.rst | 20 +-
dev/breeze/doc/04_troubleshooting.rst | 2 +-
dev/breeze/doc/05_test_commands.rst | 2 +-
dev/breeze/doc/06_managing_docker_images.rst | 2 +-
dev/breeze/doc/07_breeze_maintenance_tasks.rst | 2 +-
dev/breeze/doc/08_ci_tasks.rst | 2 +-
dev/breeze/doc/09_release_management_tasks.rst | 2 +-
dev/breeze/doc/10_advanced_breeze_topics.rst | 2 +-
dev/breeze/doc/ci/04_selective_checks.md | 1 +
dev/breeze/doc/images/output-commands.svg | 108 +-
.../doc/images/output_generate-migration-file.svg | 112 +
.../doc/images/output_generate-migration-file.txt | 1 +
.../doc/images/output_k8s_configure-cluster.svg | 6 +-
.../doc/images/output_k8s_configure-cluster.txt | 2 +-
.../doc/images/output_k8s_create-cluster.svg | 6 +-
.../doc/images/output_k8s_create-cluster.txt | 2 +-
.../doc/images/output_k8s_delete-cluster.svg | 4 +-
.../doc/images/output_k8s_delete-cluster.txt | 2 +-
.../doc/images/output_k8s_deploy-airflow.svg | 6 +-
.../doc/images/output_k8s_deploy-airflow.txt | 2 +-
dev/breeze/doc/images/output_k8s_k9s.svg | 4 +-
dev/breeze/doc/images/output_k8s_k9s.txt | 2 +-
dev/breeze/doc/images/output_k8s_logs.svg | 4 +-
dev/breeze/doc/images/output_k8s_logs.txt | 2 +-
.../doc/images/output_k8s_run-complete-tests.svg | 6 +-
.../doc/images/output_k8s_run-complete-tests.txt | 2 +-
dev/breeze/doc/images/output_k8s_shell.svg | 4 +-
dev/breeze/doc/images/output_k8s_shell.txt | 2 +-
dev/breeze/doc/images/output_k8s_status.svg | 4 +-
dev/breeze/doc/images/output_k8s_status.txt | 2 +-
dev/breeze/doc/images/output_k8s_tests.svg | 6 +-
dev/breeze/doc/images/output_k8s_tests.txt | 2 +-
.../doc/images/output_k8s_upload-k8s-image.svg | 6 +-
.../doc/images/output_k8s_upload-k8s-image.txt | 2 +-
...elease-management_install-provider-packages.svg | 54 +-
...elease-management_install-provider-packages.txt | 2 +-
...release-management_verify-provider-packages.svg | 2 +-
...release-management_verify-provider-packages.txt | 2 +-
.../output_setup_check-all-params-in-groups.svg | 18 +-
.../output_setup_check-all-params-in-groups.txt | 2 +-
.../output_setup_regenerate-command-images.svg | 74 +-
.../output_setup_regenerate-command-images.txt | 2 +-
dev/breeze/doc/images/output_shell.svg | 6 +-
dev/breeze/doc/images/output_shell.txt | 2 +-
dev/breeze/doc/images/output_start-airflow.svg | 104 +-
dev/breeze/doc/images/output_start-airflow.txt | 2 +-
dev/breeze/doc/images/output_testing_db-tests.svg | 100 +-
dev/breeze/doc/images/output_testing_db-tests.txt | 2 +-
.../doc/images/output_testing_helm-tests.svg | 22 +-
.../doc/images/output_testing_helm-tests.txt | 2 +-
.../images/output_testing_integration-tests.svg | 38 +-
.../images/output_testing_integration-tests.txt | 2 +-
.../doc/images/output_testing_non-db-tests.svg | 94 +-
.../doc/images/output_testing_non-db-tests.txt | 2 +-
dev/breeze/doc/images/output_testing_tests.svg | 116 +-
dev/breeze/doc/images/output_testing_tests.txt | 2 +-
.../airflow_breeze/commands/ci_image_commands.py | 4 +-
.../airflow_breeze/commands/developer_commands.py | 37 +-
.../commands/developer_commands_config.py | 11 +
.../airflow_breeze/commands/kubernetes_commands.py | 8 +-
.../commands/release_management_commands.py | 7 +
.../src/airflow_breeze/commands/setup_commands.py | 11 +-
.../airflow_breeze/commands/testing_commands.py | 4 +-
dev/breeze/src/airflow_breeze/global_constants.py | 56 +-
.../src/airflow_breeze/params/shell_params.py | 22 +-
.../src/airflow_breeze/utils/selective_checks.py | 18 +-
dev/breeze/tests/test_packages.py | 6 +-
dev/breeze/tests/test_selective_checks.py | 72 +-
docs/apache-airflow-providers-airbyte/commits.rst | 16 +-
docs/apache-airflow-providers-airbyte/index.rst | 6 +-
docs/apache-airflow-providers-alibaba/commits.rst | 17 +-
docs/apache-airflow-providers-alibaba/index.rst | 2 +-
docs/apache-airflow-providers-amazon/commits.rst | 23 +-
docs/apache-airflow-providers-amazon/index.rst | 6 +-
.../operators/bedrock.rst | 80 +-
.../operators/comprehend.rst | 74 +
.../operators/opensearchserverless.rst | 2 +-
.../transfer/s3_to_dynamodb.rst | 71 +
.../commits.rst | 17 +-
.../apache-airflow-providers-apache-beam/index.rst | 6 +-
.../commits.rst | 16 +-
.../index.rst | 2 +-
.../commits.rst | 16 +-
.../index.rst | 6 +-
.../operators.rst | 17 +-
.../commits.rst | 17 +-
.../index.rst | 6 +-
.../commits.rst | 16 +-
.../index.rst | 6 +-
.../commits.rst | 16 +-
.../apache-airflow-providers-apache-hdfs/index.rst | 2 +-
.../commits.rst | 17 +-
.../apache-airflow-providers-apache-hive/index.rst | 6 +-
.../index.rst | 2 +-
.../commits.rst | 16 +-
.../index.rst | 6 +-
.../commits.rst | 17 +-
.../index.rst | 2 +-
.../commits.rst | 16 +-
.../index.rst | 2 +-
.../commits.rst | 16 +-
.../apache-airflow-providers-apache-livy/index.rst | 6 +-
.../commits.rst | 16 +-
docs/apache-airflow-providers-apache-pig/index.rst | 2 +-
.../commits.rst | 16 +-
.../index.rst | 6 +-
.../commits.rst | 16 +-
.../index.rst | 6 +-
docs/apache-airflow-providers-apprise/commits.rst | 17 +-
docs/apache-airflow-providers-apprise/index.rst | 4 +-
docs/apache-airflow-providers-arangodb/commits.rst | 16 +-
docs/apache-airflow-providers-arangodb/index.rst | 2 +-
docs/apache-airflow-providers-asana/commits.rst | 16 +-
docs/apache-airflow-providers-asana/index.rst | 2 +-
.../commits.rst | 18 +-
.../index.rst | 11 +-
docs/apache-airflow-providers-celery/commits.rst | 19 +-
docs/apache-airflow-providers-celery/index.rst | 6 +-
docs/apache-airflow-providers-cloudant/commits.rst | 16 +-
docs/apache-airflow-providers-cloudant/index.rst | 2 +-
.../commits.rst | 20 +
.../index.rst | 2 +-
docs/apache-airflow-providers-cohere/commits.rst | 16 +-
docs/apache-airflow-providers-cohere/index.rst | 2 +-
.../apache-airflow-providers-common-io/commits.rst | 17 +-
docs/apache-airflow-providers-common-io/index.rst | 6 +-
.../commits.rst | 20 +-
docs/apache-airflow-providers-common-sql/index.rst | 6 +-
.../commits.rst | 17 +
...icks_workflow_task_group_airflow_graph_view.png | Bin 0 -> 188072 bytes
.../img/workflow_run_databricks_graph_view.png | Bin 0 -> 405053 bytes
docs/apache-airflow-providers-databricks/index.rst | 6 +-
.../operators/workflow.rst | 71 +
docs/apache-airflow-providers-datadog/commits.rst | 16 +-
docs/apache-airflow-providers-datadog/index.rst | 2 +-
.../apache-airflow-providers-dbt-cloud/commits.rst | 19 +-
docs/apache-airflow-providers-dbt-cloud/index.rst | 6 +-
docs/apache-airflow-providers-dingding/commits.rst | 16 +-
docs/apache-airflow-providers-dingding/index.rst | 6 +-
docs/apache-airflow-providers-discord/commits.rst | 16 +-
docs/apache-airflow-providers-discord/index.rst | 6 +-
docs/apache-airflow-providers-docker/commits.rst | 20 +-
docs/apache-airflow-providers-docker/index.rst | 9 +-
.../commits.rst | 18 +-
.../index.rst | 6 +-
docs/apache-airflow-providers-exasol/commits.rst | 16 +-
docs/apache-airflow-providers-exasol/index.rst | 6 +-
docs/apache-airflow-providers-fab/commits.rst | 18 +-
docs/apache-airflow-providers-fab/index.rst | 2 +-
docs/apache-airflow-providers-facebook/commits.rst | 16 +-
docs/apache-airflow-providers-facebook/index.rst | 2 +-
docs/apache-airflow-providers-ftp/commits.rst | 16 +-
docs/apache-airflow-providers-ftp/index.rst | 6 +-
docs/apache-airflow-providers-github/commits.rst | 16 +-
docs/apache-airflow-providers-github/index.rst | 2 +-
docs/apache-airflow-providers-google/commits.rst | 22 +
docs/apache-airflow-providers-google/index.rst | 8 +-
.../cloud/cloud_storage_transfer_service.rst | 35 +
.../operators/cloud/vertex_ai.rst | 4 +-
.../operators/transfer/gcs_to_gcs.rst | 28 +-
docs/apache-airflow-providers-grpc/commits.rst | 16 +-
docs/apache-airflow-providers-grpc/index.rst | 2 +-
.../apache-airflow-providers-hashicorp/commits.rst | 17 +-
docs/apache-airflow-providers-hashicorp/index.rst | 6 +-
docs/apache-airflow-providers-http/commits.rst | 16 +-
docs/apache-airflow-providers-http/index.rst | 2 +-
docs/apache-airflow-providers-imap/commits.rst | 16 +-
docs/apache-airflow-providers-imap/index.rst | 2 +-
docs/apache-airflow-providers-influxdb/commits.rst | 16 +-
docs/apache-airflow-providers-influxdb/index.rst | 2 +-
docs/apache-airflow-providers-jdbc/commits.rst | 16 +-
docs/apache-airflow-providers-jdbc/index.rst | 6 +-
docs/apache-airflow-providers-jdbc/operators.rst | 8 +-
docs/apache-airflow-providers-jenkins/commits.rst | 16 +-
docs/apache-airflow-providers-jenkins/index.rst | 2 +-
.../commits.rst | 15 +-
.../index.rst | 6 +-
.../commits.rst | 18 +-
.../index.rst | 6 +-
.../commits.rst | 16 +-
.../index.rst | 2 +-
.../commits.rst | 16 +-
.../index.rst | 2 +-
docs/apache-airflow-providers-mongo/commits.rst | 16 +-
docs/apache-airflow-providers-mongo/index.rst | 2 +-
docs/apache-airflow-providers-mysql/commits.rst | 16 +-
docs/apache-airflow-providers-mysql/index.rst | 6 +-
docs/apache-airflow-providers-mysql/operators.rst | 14 +-
docs/apache-airflow-providers-neo4j/commits.rst | 16 +-
docs/apache-airflow-providers-neo4j/index.rst | 2 +-
docs/apache-airflow-providers-odbc/commits.rst | 16 +-
docs/apache-airflow-providers-odbc/index.rst | 6 +-
docs/apache-airflow-providers-openai/commits.rst | 14 +
docs/apache-airflow-providers-openai/index.rst | 2 +-
docs/apache-airflow-providers-openfaas/commits.rst | 16 +-
docs/apache-airflow-providers-openfaas/index.rst | 2 +-
.../commits.rst | 27 +-
.../apache-airflow-providers-openlineage/index.rst | 6 +-
.../commits.rst | 16 +-
docs/apache-airflow-providers-opensearch/index.rst | 2 +-
docs/apache-airflow-providers-opsgenie/commits.rst | 16 +-
docs/apache-airflow-providers-opsgenie/index.rst | 2 +-
docs/apache-airflow-providers-oracle/commits.rst | 16 +-
docs/apache-airflow-providers-oracle/index.rst | 6 +-
.../apache-airflow-providers-pagerduty/commits.rst | 16 +-
docs/apache-airflow-providers-pagerduty/index.rst | 2 +-
.../apache-airflow-providers-papermill/commits.rst | 16 +-
docs/apache-airflow-providers-papermill/index.rst | 2 +-
docs/apache-airflow-providers-pgvector/commits.rst | 16 +-
docs/apache-airflow-providers-pgvector/index.rst | 6 +-
docs/apache-airflow-providers-pinecone/commits.rst | 4 +-
docs/apache-airflow-providers-postgres/commits.rst | 17 +-
docs/apache-airflow-providers-postgres/index.rst | 6 +-
.../operators/postgres_operator_howto_guide.rst | 67 +-
docs/apache-airflow-providers-presto/commits.rst | 16 +-
docs/apache-airflow-providers-presto/index.rst | 6 +-
docs/apache-airflow-providers-qdrant/commits.rst | 16 +-
docs/apache-airflow-providers-qdrant/index.rst | 2 +-
docs/apache-airflow-providers-redis/commits.rst | 16 +-
docs/apache-airflow-providers-redis/index.rst | 2 +-
.../commits.rst | 16 +-
docs/apache-airflow-providers-salesforce/index.rst | 2 +-
docs/apache-airflow-providers-samba/commits.rst | 16 +-
docs/apache-airflow-providers-samba/index.rst | 6 +-
docs/apache-airflow-providers-segment/commits.rst | 16 +-
docs/apache-airflow-providers-segment/index.rst | 2 +-
docs/apache-airflow-providers-sendgrid/commits.rst | 16 +-
docs/apache-airflow-providers-sendgrid/index.rst | 2 +-
docs/apache-airflow-providers-sftp/commits.rst | 17 +-
docs/apache-airflow-providers-sftp/index.rst | 6 +-
.../commits.rst | 16 +-
.../apache-airflow-providers-singularity/index.rst | 2 +-
docs/apache-airflow-providers-slack/commits.rst | 14 +
docs/apache-airflow-providers-slack/index.rst | 6 +-
docs/apache-airflow-providers-smtp/commits.rst | 16 +-
docs/apache-airflow-providers-smtp/index.rst | 2 +-
.../apache-airflow-providers-snowflake/commits.rst | 18 +-
docs/apache-airflow-providers-snowflake/index.rst | 6 +-
.../operators/snowflake.rst | 13 +-
docs/apache-airflow-providers-sqlite/commits.rst | 16 +-
docs/apache-airflow-providers-sqlite/index.rst | 6 +-
docs/apache-airflow-providers-sqlite/operators.rst | 14 +-
docs/apache-airflow-providers-ssh/commits.rst | 16 +-
docs/apache-airflow-providers-ssh/index.rst | 2 +-
docs/apache-airflow-providers-tableau/commits.rst | 16 +-
docs/apache-airflow-providers-tableau/index.rst | 2 +-
docs/apache-airflow-providers-telegram/commits.rst | 16 +-
docs/apache-airflow-providers-telegram/index.rst | 2 +-
docs/apache-airflow-providers-teradata/commits.rst | 16 +-
docs/apache-airflow-providers-teradata/index.rst | 6 +-
.../operators/azure_blob_to_teradata.rst | 73 +
.../operators/s3_to_teradata.rst | 77 +
.../operators/teradata.rst | 114 +
docs/apache-airflow-providers-trino/commits.rst | 16 +-
docs/apache-airflow-providers-trino/index.rst | 6 +-
docs/apache-airflow-providers-vertica/commits.rst | 16 +-
docs/apache-airflow-providers-vertica/index.rst | 6 +-
docs/apache-airflow-providers-weaviate/commits.rst | 16 +-
docs/apache-airflow-providers-weaviate/index.rst | 2 +-
docs/apache-airflow-providers-yandex/commits.rst | 19 +-
docs/apache-airflow-providers-yandex/index.rst | 6 +-
docs/apache-airflow-providers-zendesk/commits.rst | 16 +-
docs/apache-airflow-providers-zendesk/index.rst | 2 +-
.../logging-monitoring/metrics.rst | 2 +
.../authoring-and-scheduling/plugins.rst | 11 +-
docs/apache-airflow/faq.rst | 26 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/airflow_erd.svg | 2865 +++++++++++---------
.../installation/installing-from-pypi.rst | 5 +
docs/apache-airflow/installation/prerequisites.rst | 2 +-
docs/apache-airflow/migrations-ref.rst | 4 +-
.../aws/[email protected] | Bin 0 -> 7254 bytes
docs/spelling_wordlist.txt | 6 +
generated/PYPI_README.md | 16 +-
generated/provider_dependencies.json | 128 +-
generated/provider_metadata.json | 364 +++
hatch_build.py | 14 +-
helm_tests/airflow_aux/test_pod_template_file.py | 27 +-
pyproject.toml | 9 +-
...ests-sources-only.yml => integration-mssql.yml} | 33 +-
...es-only.yml => providers-and-tests-sources.yml} | 5 +
.../{tests-sources-only.yml => tests-sources.yml} | 3 +
scripts/ci/pre_commit/mypy_folder.py | 26 +-
.../ci/pre_commit/update_providers_dependencies.py | 6 +-
.../ci/testing/run_integration_tests_with_retry.sh | 53 +
scripts/docker/common.sh | 2 +
scripts/docker/entrypoint_ci.sh | 11 +-
scripts/docker/install_airflow.sh | 20 +-
scripts/in_container/_in_container_utils.sh | 2 +
scripts/in_container/check_environment.sh | 4 +
.../in_container/install_airflow_and_providers.py | 19 +-
scripts/in_container/run_ci_tests.sh | 9 +
scripts/in_container/run_generate_constraints.py | 1 -
.../in_container/run_generate_migration.sh | 15 +-
.../in_container/run_provider_yaml_files_check.py | 4 +
tests/always/test_example_dags.py | 13 -
tests/always/test_project_structure.py | 11 +-
.../endpoints/test_extra_link_endpoint.py | 2 +-
.../endpoints/test_import_error_endpoint.py | 2 +-
.../endpoints/test_plugin_endpoint.py | 2 +-
.../endpoints/test_task_instance_endpoint.py | 116 +
tests/api_connexion/schemas/test_error_schema.py | 2 +-
tests/api_connexion/schemas/test_plugin_schema.py | 2 +-
tests/api_experimental/client/test_json_client.py | 86 +
tests/api_experimental/common/test_delete_dag.py | 2 +-
tests/charts/helm_template_generator.py | 10 +-
tests/cli/commands/test_config_command.py | 12 +-
tests/conftest.py | 40 +-
tests/core/test_settings.py | 25 +-
tests/dag_processing/test_job_runner.py | 2 +-
tests/dag_processing/test_processor.py | 2 +-
tests/dags/test_imports.py | 2 +-
tests/datasets/test_dataset.py | 29 +-
tests/deprecations_ignore.yml | 20 -
.../cli/commands/test_celery_command.py | 2 +-
.../integration/executors/test_celery_executor.py | 17 +-
.../apache/hive/transfers/test_mssql_to_hive.py | 121 +-
.../cloud/transfers/test_bigquery_to_mssql.py | 75 +-
.../google/cloud/transfers/test_mssql_to_gcs.py | 180 +-
.../integration/providers/microsoft}/__init__.py | 0
.../providers/microsoft/mssql}/__init__.py | 0
.../providers/microsoft/mssql/hooks}/__init__.py | 0
.../providers/microsoft/mssql/hooks/test_mssql.py} | 47 +-
tests/jobs/test_backfill_job.py | 2 +
tests/jobs/test_local_task_job.py | 38 +-
tests/jobs/test_scheduler_job.py | 37 +-
tests/listeners/class_listener.py | 92 +-
.../amazon/aws/auth_manager/avp/test_facade.py | 23 +-
.../aws/auth_manager/cli/test_avp_commands.py | 3 +
.../aws/auth_manager/cli/test_idc_commands.py | 3 +
.../test_aws_security_manager_override.py | 9 +-
.../aws/auth_manager/test_aws_auth_manager.py | 31 +-
.../amazon/aws/auth_manager/views/test_auth.py | 4 +
.../aws/executors/batch/test_batch_executor.py | 6 +
.../amazon/aws/executors/ecs/test_ecs_executor.py | 15 +-
tests/providers/amazon/aws/hooks/test_bedrock.py | 8 +-
.../aws/hooks/test_comprehend.py} | 16 +-
tests/providers/amazon/aws/hooks/test_dynamodb.py | 90 +-
.../amazon/aws/hooks/test_hooks_signature.py | 4 +-
tests/providers/amazon/aws/hooks/test_neptune.py | 8 +
tests/providers/amazon/aws/links/test_base_aws.py | 5 +-
.../providers/amazon/aws/operators/test_athena.py | 2 +-
tests/providers/amazon/aws/operators/test_batch.py | 105 +-
.../providers/amazon/aws/operators/test_bedrock.py | 174 ++
.../amazon/aws/operators/test_comprehend.py | 163 ++
.../amazon/aws/operators/test_emr_containers.py | 16 +
.../amazon/aws/operators/test_emr_serverless.py | 5 +-
.../providers/amazon/aws/operators/test_neptune.py | 222 +-
.../aws/operators/test_sagemaker_transform.py | 16 +-
.../amazon/aws/sensors/test_comprehend.py | 94 +
.../amazon/aws/sensors/test_emr_containers.py | 14 +
.../amazon/aws/transfers/test_s3_to_dynamodb.py | 234 ++
.../amazon/aws/triggers/test_comprehend.py | 67 +
.../providers/amazon/aws/triggers/test_neptune.py | 47 +
.../amazon/aws/triggers/test_sagemaker.py | 4 +-
.../amazon/aws/utils/test_eks_get_token.py | 6 +-
.../amazon/aws/waiters/test_comprehend.py | 71 +
tests/providers/amazon/aws/waiters/test_dynamo.py | 63 +-
.../apache/hive/transfers/test_mssql_to_hive.py | 11 +-
.../providers/apache/iceberg/hooks/test_iceberg.py | 34 +-
tests/providers/atlassian/jira/hooks/test_jira.py | 41 +-
.../atlassian/jira/operators/test_jira.py | 17 +-
.../providers/atlassian/jira/sensors/test_jira.py | 21 +-
.../cncf/kubernetes/operators/test_pod.py | 39 +-
.../kubernetes/operators/test_spark_kubernetes.py | 12 +-
.../cncf/kubernetes/test_template_rendering.py | 4 +-
.../common/io/operators/test_file_transfer.py | 33 +-
tests/providers/common/io/xcom/test_backend.py | 15 +-
tests/providers/common/sql/hooks/test_dbapi.py | 88 +-
tests/providers/common/sql/hooks/test_sql.py | 5 +
tests/providers/common/sql/hooks/test_sqlparse.py | 5 +
tests/providers/common/sql/operators/test_sql.py | 6 +-
.../common/sql/operators/test_sql_execute.py | 5 +
tests/providers/common/sql/sensors/test_sql.py | 49 +-
tests/providers/common/sql/test_utils.py | 10 +-
.../databricks/operators/test_databricks.py | 119 +
.../operators/test_databricks_workflow.py | 233 ++
.../providers/dbt/cloud/utils/test_openlineage.py | 23 +-
tests/providers/docker/hooks/test_docker.py | 2 +
.../docker/operators/test_docker_swarm.py | 77 +
.../elasticsearch/log/test_es_task_handler.py | 14 +-
.../api/auth/backend/test_basic_auth.py | 5 +
.../api/auth/backend/test_kerberos_auth.py | 5 +-
.../test_role_and_permission_endpoint.py | 11 +-
.../api_endpoints/test_user_endpoint.py | 7 +-
.../auth_manager/api_endpoints/test_user_schema.py | 11 +-
.../auth_manager/cli_commands/test_definition.py | 13 +-
.../auth_manager/cli_commands/test_role_command.py | 9 +-
.../cli_commands/test_sync_perm_command.py | 5 +-
.../auth_manager/cli_commands/test_user_command.py | 7 +-
.../fab/auth_manager/cli_commands/test_utils.py | 9 +-
.../fab/auth_manager/decorators/test_auth.py | 12 +-
.../fab/auth_manager/models/test_anonymous_user.py | 5 +-
.../security_manager/test_constants.py | 5 +-
.../auth_manager/security_manager/test_override.py | 5 +-
.../fab/auth_manager/test_fab_auth_manager.py | 17 +-
tests/providers/fab/auth_manager/test_models.py | 11 +-
tests/providers/fab/auth_manager/test_security.py | 19 +-
.../fab/auth_manager/views/test_permissions.py | 5 +
.../fab/auth_manager/views/test_roles_list.py | 5 +
.../providers/fab/auth_manager/views/test_user.py | 5 +
.../fab/auth_manager/views/test_user_edit.py | 5 +
.../fab/auth_manager/views/test_user_stats.py | 5 +
tests/providers/ftp/hooks/test_ftp.py | 42 +
.../hooks/test_cloud_storage_transfer_service.py | 19 +
tests/providers/google/cloud/hooks/test_gcs.py | 26 +
.../providers/google/cloud/hooks/test_mlengine.py | 30 +-
.../providers/google/cloud/links}/__init__.py | 0
.../providers/google/cloud/links/test_translate.py | 154 ++
.../cloud/log/test_stackdriver_task_handler.py | 38 +-
.../google/cloud/openlineage}/__init__.py | 0
.../google/cloud/openlineage/test_utils.py | 369 +++
.../google/cloud/operators/test_automl.py | 222 +-
.../google/cloud/operators/test_bigquery.py | 38 +-
.../test_cloud_storage_transfer_service.py | 56 +
.../google/cloud/operators/test_dataproc.py | 75 +-
.../cloud/operators/test_kubernetes_engine.py | 54 +
.../google/cloud/operators/test_vertex_ai.py | 520 ++++
.../google/cloud/transfers/test_gcs_to_bigquery.py | 210 +-
.../google/cloud/transfers/test_mssql_to_gcs.py | 8 +-
.../google/cloud/transfers/test_trino_to_gcs.py | 327 +++
.../cloud/triggers/test_kubernetes_engine.py | 2 +
.../cloud/{operators => utils}/job_details.json | 0
.../google/cloud/utils/out_table_details.json | 30 +
.../google/cloud/utils/script_job_details.json | 36 +
.../google/cloud/utils/table_details.json | 53 +
.../cloud/utils/test_credentials_provider.py | 12 +-
.../google/cloud/utils/test_openlineage.py | 142 -
.../google/common/hooks/test_base_google.py | 32 +-
.../microsoft/azure/hooks/test_data_factory.py | 29 +-
tests/providers/microsoft/conftest.py | 19 +-
tests/providers/mongo/hooks/test_mongo.py | 3 +-
.../providers/openlineage/plugins/test_adapter.py | 110 +-
.../providers/openlineage/plugins/test_listener.py | 168 +-
tests/providers/openlineage/plugins/test_macros.py | 25 +-
.../openlineage/plugins/test_openlineage.py | 13 +-
tests/providers/openlineage/test_conf.py | 56 +
.../openlineage/utils/test_selective_enable.py | 6 +-
tests/providers/openlineage/utils/test_sql.py | 11 -
tests/providers/sftp/operators/test_sftp.py | 44 +-
tests/providers/smtp/notifications/test_smtp.py | 12 +-
.../providers/snowflake/utils/test_openlineage.py | 62 +
tests/providers/teradata/hooks/test_teradata.py | 35 +-
.../providers/teradata/operators/test_teradata.py | 36 +-
.../transfers/test_azure_blob_to_teradata.py | 60 +
.../teradata/transfers/test_s3_to_teradata.py | 79 +
...st_yandexcloud_dataproc.py => test_dataproc.py} | 2 +-
tests/providers/yandex/hooks/test_yandex.py | 27 +-
tests/providers/yandex/hooks/test_yq.py | 21 +-
...st_yandexcloud_dataproc.py => test_dataproc.py} | 10 +-
tests/providers/yandex/secrets/test_lockbox.py | 16 +-
tests/serialization/test_dag_serialization.py | 20 +-
tests/serialization/test_pydantic_models.py | 67 +
...py => example_bedrock_retrieve_and_generate.py} | 85 +-
.../providers/amazon/aws/example_comprehend.py | 137 +
.../amazon/aws/example_eks_with_nodegroups.py | 1 +
.../providers/amazon/aws/example_s3_to_dynamodb.py | 192 ++
.../providers/apache/drill/example_drill_dag.py | 4 +-
.../databricks/example_databricks_workflow.py | 118 +
tests/system/providers/google/__init__.py | 3 +
tests/system/providers/google/ads/example_ads.py | 3 +-
.../google/cloud/automl/example_automl_dataset.py | 103 +-
.../google/cloud/automl/example_automl_model.py | 51 +-
.../cloud/automl/example_automl_translation.py | 13 +-
.../cloud/azure/example_azure_fileshare_to_gcs.py | 3 +-
.../google/cloud/bigquery/example_bigquery_dts.py | 3 +-
.../cloud/bigquery/example_bigquery_queries.py | 3 +-
.../cloud/bigquery/example_bigquery_sensors.py | 6 +-
.../cloud/bigquery/example_bigquery_tables.py | 3 +-
.../cloud/bigquery/example_bigquery_to_gcs.py | 3 +-
.../bigquery/example_bigquery_to_gcs_async.py | 3 +-
.../cloud/bigquery/example_bigquery_transfer.py | 3 +-
.../cloud/bigquery/example_bigquery_value_check.py | 148 +
.../google/cloud/bigtable/example_bigtable.py | 3 +-
.../cloud/cloud_build/example_cloud_build.py | 3 +-
.../cloud_build/example_cloud_build_trigger.py | 3 +-
.../cloud/cloud_functions/example_functions.py | 3 +-
.../example_cloud_memorystore_memcached.py | 2 +-
.../example_cloud_memorystore_redis.py | 3 +-
.../google/cloud/cloud_sql/example_cloud_sql.py | 3 +-
.../cloud/cloud_sql/example_cloud_sql_query.py | 3 +-
.../google/cloud/compute/example_compute.py | 3 +-
.../google/cloud/compute/example_compute_igm.py | 3 +-
.../google/cloud/compute/example_compute_ssh.py | 3 +-
.../cloud/compute/example_compute_ssh_os_login.py | 3 +-
.../cloud/compute/example_compute_ssh_parallel.py | 3 +-
.../example_dlp_deidentify_content.py | 3 +-
.../data_loss_prevention/example_dlp_info_types.py | 3 +-
.../example_dlp_inspect_template.py | 3 +-
.../cloud/data_loss_prevention/example_dlp_job.py | 3 +-
.../example_dlp_job_trigger.py | 3 +-
.../google/cloud/dataflow/example_dataflow_sql.py | 3 +-
.../cloud/dataflow/example_dataflow_template.py | 3 +-
.../google/cloud/datafusion/example_datafusion.py | 3 +-
.../cloud/datapipelines/example_datapipeline.py | 3 +-
.../google/cloud/dataprep/example_dataprep.py | 3 +-
.../cloud/dataproc/example_dataproc_batch.py | 3 +-
.../dataproc/example_dataproc_batch_deferrable.py | 3 +-
...proc_cluster_create_existing_stopped_cluster.py | 2 +-
.../example_dataproc_cluster_deferrable.py | 3 +-
.../dataproc/example_dataproc_cluster_diagnose.py | 3 +-
.../example_dataproc_cluster_start_stop.py | 2 +-
.../dataproc/example_dataproc_cluster_update.py | 3 +-
.../cloud/dataproc/example_dataproc_hadoop.py | 3 +-
.../google/cloud/dataproc/example_dataproc_hive.py | 3 +-
.../google/cloud/dataproc/example_dataproc_pig.py | 3 +-
.../cloud/dataproc/example_dataproc_presto.py | 3 +-
.../cloud/dataproc/example_dataproc_pyspark.py | 3 +-
.../cloud/dataproc/example_dataproc_spark.py | 3 +-
.../cloud/dataproc/example_dataproc_spark_async.py | 3 +-
.../dataproc/example_dataproc_spark_deferrable.py | 3 +-
.../cloud/dataproc/example_dataproc_spark_sql.py | 3 +-
.../cloud/dataproc/example_dataproc_sparkr.py | 3 +-
.../cloud/dataproc/example_dataproc_trino.py | 3 +-
.../cloud/dataproc/example_dataproc_workflow.py | 3 +-
.../example_dataproc_workflow_deferrable.py | 3 +-
.../cloud/datastore/example_datastore_commit.py | 3 +-
.../cloud/datastore/example_datastore_query.py | 3 +-
.../cloud/datastore/example_datastore_rollback.py | 3 +-
.../providers/google/cloud/gcs/example_gcs_acl.py | 3 +-
.../google/cloud/gcs/example_gcs_copy_delete.py | 3 +-
.../google/cloud/gcs/example_gcs_sensor.py | 3 +-
.../google/cloud/gcs/example_gcs_to_bigquery.py | 3 +-
.../cloud/gcs/example_gcs_to_bigquery_async.py | 3 +-
.../google/cloud/gcs/example_gcs_to_gcs.py | 32 +-
.../google/cloud/gcs/example_gcs_to_gdrive.py | 3 +-
.../google/cloud/gcs/example_gcs_transform.py | 3 +-
.../cloud/gcs/example_gcs_transform_timespan.py | 3 +-
.../cloud/gcs/example_gcs_upload_download.py | 3 +-
.../google/cloud/gcs/example_gdrive_to_gcs.py | 3 +-
.../google/cloud/gcs/example_mssql_to_gcs.py | 3 +-
.../google/cloud/gcs/example_oracle_to_gcs.py | 3 +-
.../google/cloud/gcs/example_s3_to_gcs.py | 3 +-
.../google/cloud/gcs/example_sftp_to_gcs.py | 3 +-
.../cloud/life_sciences/example_life_sciences.py | 3 +-
.../google/cloud/ml_engine/example_mlengine.py | 6 +-
.../google/cloud/spanner/example_spanner.py | 3 +-
.../cloud/speech_to_text/example_speech_to_text.py | 3 +-
.../cloud/stackdriver/example_stackdriver.py | 3 +-
.../example_cloud_storage_transfer_service_aws.py | 3 +-
.../example_cloud_storage_transfer_service_gcp.py | 10 +
.../cloud/text_to_speech/example_text_to_speech.py | 3 +-
.../google/cloud/transfers/example_gcs_to_sftp.py | 3 +-
.../cloud/transfers/example_gdrive_to_local.py | 3 +-
.../vertex_ai/example_vertex_ai_custom_job.py | 43 +-
.../cloud/vision/example_vision_annotate_image.py | 3 +-
.../cloud/vision/example_vision_autogenerated.py | 3 +-
.../google/cloud/vision/example_vision_explicit.py | 3 +-
.../datacatalog/example_datacatalog_entries.py | 3 +-
.../example_datacatalog_search_catalog.py | 3 +-
.../example_datacatalog_tag_templates.py | 3 +-
.../google/datacatalog/example_datacatalog_tags.py | 3 +-
.../marketing_platform/example_campaign_manager.py | 3 +-
.../system/providers/jdbc/example_jdbc_queries.py | 10 +-
tests/system/providers/mysql/example_mysql.py | 8 +-
.../providers/pinecone/example_pinecone_cohere.py | 15 +-
.../system/providers/postgres/example_postgres.py | 30 +-
.../system/providers/samba/example_gcs_to_samba.py | 3 +-
.../providers/snowflake/example_snowflake.py | 17 +-
tests/system/providers/sqlite/example_sqlite.py | 8 +-
.../example_azure_blob_to_teradata_transfer.py | 143 +
.../teradata/example_s3_to_teradata_transfer.py | 147 +
.../providers/teradata/example_ssl_teradata.py | 19 +-
.../system/providers/teradata/example_teradata.py | 37 +-
.../providers/teradata/example_teradata_call_sp.py | 174 ++
.../example_teradata_to_teradata_transfer.py | 27 +-
.../providers/weaviate/example_weaviate_cohere.py | 2 +-
.../providers/weaviate/example_weaviate_openai.py | 2 +-
.../weaviate/example_weaviate_operator.py | 8 +-
.../yandex/example_yandexcloud_dataproc.py | 2 +-
.../example_yandexcloud_dataproc_lightweight.py | 2 +-
.../task/task_runner/test_standard_task_runner.py | 38 +-
tests/test_utils/api_connexion_utils.py | 5 +-
tests/test_utils/compat.py | 136 +
tests/test_utils/db.py | 25 +-
tests/test_utils/mock_operators.py | 2 +-
tests/test_utils/mock_security_manager.py | 5 +-
tests/utils/log/test_secrets_masker.py | 8 +-
tests/utils/test_log_handlers.py | 24 +-
tests/utils/test_usage_data_collection.py | 84 +
tests/www/views/test_views.py | 30 +
tests/www/views/test_views_extra_links.py | 5 +-
tests/www/views/test_views_home.py | 14 +
1031 files changed, 20433 insertions(+), 6303 deletions(-)
copy airflow/example_dags/{example_outlet_event_extra.py =>
example_inlet_event_extra.py} (54%)
create mode 100644
airflow/migrations/versions/0143_2_9_2_add_indexes_on_dag_id_column_in_referencing_tables.py
rename
airflow/migrations/versions/{0143_2_10_0_add_new_executor_field_to_db.py =>
0144_2_10_0_add_new_executor_field_to_db.py} (96%)
rename
airflow/migrations/versions/{0144_2_10_0_added_dagpriorityparsingrequest_table.py
=> 0145_2_10_0_added_dagpriorityparsingrequest_table.py} (100%)
copy airflow/providers/{yandex/hooks/yandexcloud_dataproc.py =>
amazon/aws/hooks/comprehend.py} (64%)
create mode 100644 airflow/providers/amazon/aws/operators/comprehend.py
create mode 100644 airflow/providers/amazon/aws/sensors/comprehend.py
create mode 100644 airflow/providers/amazon/aws/transfers/s3_to_dynamodb.py
create mode 100644 airflow/providers/amazon/aws/triggers/comprehend.py
create mode 100644 airflow/providers/amazon/aws/waiters/comprehend.json
create mode 100644
airflow/providers/databricks/operators/databricks_workflow.py
create mode 100644 airflow/providers/google/cloud/links/translate.py
create mode 100644
airflow/providers/google/cloud/openlineage/BigQueryErrorRunFacet.json
create mode 100644
airflow/providers/google/cloud/openlineage/BigQueryJobRunFacet.json
copy airflow/{api_connexion => providers/google/cloud/openlineage}/__init__.py
(100%)
create mode 100644 airflow/providers/google/cloud/openlineage/utils.py
delete mode 100644 airflow/providers/google/cloud/utils/openlineage.py
create mode 100644 airflow/providers/snowflake/utils/openlineage.py
create mode 100644
airflow/providers/teradata/transfers/azure_blob_to_teradata.py
create mode 100644 airflow/providers/teradata/transfers/s3_to_teradata.py
copy airflow/providers/yandex/hooks/{yandexcloud_dataproc.py => dataproc.py}
(100%)
copy airflow/providers/yandex/operators/{yandexcloud_dataproc.py =>
dataproc.py} (99%)
create mode 100644 airflow/utils/usage_data_collection.py
create mode 100644 airflow/www/static/js/api/useTaskFailedDependency.ts
create mode 100644
airflow/www/static/js/dag/details/taskInstance/TaskFailedDependency.tsx
create mode 100644 dev/breeze/doc/images/output_generate-migration-file.svg
create mode 100644 dev/breeze/doc/images/output_generate-migration-file.txt
create mode 100644
docs/apache-airflow-providers-amazon/operators/comprehend.rst
create mode 100644
docs/apache-airflow-providers-amazon/transfer/s3_to_dynamodb.rst
create mode 100644
docs/apache-airflow-providers-databricks/img/databricks_workflow_task_group_airflow_graph_view.png
create mode 100644
docs/apache-airflow-providers-databricks/img/workflow_run_databricks_graph_view.png
create mode 100644
docs/apache-airflow-providers-databricks/operators/workflow.rst
create mode 100644
docs/apache-airflow-providers-teradata/operators/azure_blob_to_teradata.rst
create mode 100644
docs/apache-airflow-providers-teradata/operators/s3_to_teradata.rst
create mode 100644 docs/integration-logos/aws/[email protected]
copy scripts/ci/docker-compose/{tests-sources-only.yml =>
integration-mssql.yml} (61%)
copy scripts/ci/docker-compose/{tests-sources-only.yml =>
providers-and-tests-sources.yml} (82%)
rename scripts/ci/docker-compose/{tests-sources-only.yml => tests-sources.yml}
(87%)
create mode 100755 scripts/ci/testing/run_integration_tests_with_retry.sh
copy tests/test_utils/mock_security_manager.py =>
scripts/in_container/run_generate_migration.sh (72%)
mode change 100644 => 100755
create mode 100644 tests/api_experimental/client/test_json_client.py
copy {airflow/api_connexion =>
tests/integration/providers/microsoft}/__init__.py (100%)
copy {airflow/api_connexion =>
tests/integration/providers/microsoft/mssql}/__init__.py (100%)
copy {airflow/api_connexion =>
tests/integration/providers/microsoft/mssql/hooks}/__init__.py (100%)
copy tests/{providers/apache/iceberg/hooks/test_iceberg.py =>
integration/providers/microsoft/mssql/hooks/test_mssql.py} (52%)
copy tests/providers/{fab/auth_manager/cli_commands/test_utils.py =>
amazon/aws/hooks/test_comprehend.py} (67%)
create mode 100644 tests/providers/amazon/aws/operators/test_comprehend.py
create mode 100644 tests/providers/amazon/aws/sensors/test_comprehend.py
create mode 100644 tests/providers/amazon/aws/transfers/test_s3_to_dynamodb.py
create mode 100644 tests/providers/amazon/aws/triggers/test_comprehend.py
create mode 100644 tests/providers/amazon/aws/waiters/test_comprehend.py
copy tests/{integration =>
}/providers/apache/hive/transfers/test_mssql_to_hive.py (93%)
create mode 100644
tests/providers/databricks/operators/test_databricks_workflow.py
copy {airflow/api_connexion => tests/providers/google/cloud/links}/__init__.py
(100%)
create mode 100644 tests/providers/google/cloud/links/test_translate.py
copy {airflow/api_connexion =>
tests/providers/google/cloud/openlineage}/__init__.py (100%)
create mode 100644 tests/providers/google/cloud/openlineage/test_utils.py
copy tests/{integration =>
}/providers/google/cloud/transfers/test_mssql_to_gcs.py (96%)
create mode 100644 tests/providers/google/cloud/transfers/test_trino_to_gcs.py
rename tests/providers/google/cloud/{operators => utils}/job_details.json
(100%)
create mode 100644 tests/providers/google/cloud/utils/out_table_details.json
create mode 100644 tests/providers/google/cloud/utils/script_job_details.json
create mode 100644 tests/providers/google/cloud/utils/table_details.json
delete mode 100644 tests/providers/google/cloud/utils/test_openlineage.py
create mode 100644 tests/providers/snowflake/utils/test_openlineage.py
create mode 100644
tests/providers/teradata/transfers/test_azure_blob_to_teradata.py
create mode 100644 tests/providers/teradata/transfers/test_s3_to_teradata.py
rename tests/providers/yandex/hooks/{test_yandexcloud_dataproc.py =>
test_dataproc.py} (99%)
rename tests/providers/yandex/operators/{test_yandexcloud_dataproc.py =>
test_dataproc.py} (98%)
rename tests/system/providers/amazon/aws/{example_bedrock_knowledge_base.py =>
example_bedrock_retrieve_and_generate.py} (86%)
create mode 100644 tests/system/providers/amazon/aws/example_comprehend.py
create mode 100644 tests/system/providers/amazon/aws/example_s3_to_dynamodb.py
create mode 100644
tests/system/providers/databricks/example_databricks_workflow.py
create mode 100644
tests/system/providers/google/cloud/bigquery/example_bigquery_value_check.py
create mode 100644
tests/system/providers/teradata/example_azure_blob_to_teradata_transfer.py
create mode 100644
tests/system/providers/teradata/example_s3_to_teradata_transfer.py
create mode 100644 tests/system/providers/teradata/example_teradata_call_sp.py
create mode 100644 tests/test_utils/compat.py
create mode 100644 tests/utils/test_usage_data_collection.py
