This is an automated email from the ASF dual-hosted git repository.
ash pushed a change to branch task-sdk-first-code
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 50081fada27 fixup! Move over more of BaseOperator and DAG, along with
their tests
discard 44efb0b7c1d Move over more of BaseOperator and DAG, along with their
tests
discard 762f0212c10 Start replacing DAG class with the task-sdk DAG class
instead
discard 3c11050ad87 Start porting over all the DAG defintion code to the Task
SDK
add 43f48baef82 trove classifier upgrade (#42979)
add d6a1a522e7f Change directory used by simple auth manager to store
generated passwords (#42860)
add 20f82901f44 Disable flaky mssql based integration tests (#42811)
add d9956cc239c Remove BackfillJobRunner class (#42943)
add 7ae3a551944 Add early job_id xcom_push for google provider Beam
Pipeline operators (#42982)
add 72da8f6b608 Add logic to mark backfills as complete (#42683)
add baa87b600b6 fix mypy check failure on main (#42976)
add c5e4a74cfc9 mark
test_setup_constraint_mapped_task_upstream_removed_and_success as flaky (#42997)
add 85c2b7d722d Update providers metadata 2024-10-14 (#42995)
add 5316e618e09 Update json schema pre-commit to have draft7 schema in
file (#43005)
add b979debb511 AIP-84 Patch Variable (#42929)
add ced319fe95a Commit the session between writing and deletion of RTIF
(#42928)
add 2e5607a44c7 Refactor FastApi Dag and DagRun endpoints tests (#42949)
add 41225973475 Minor updates in UI contributing docs (#43013)
add 4c5ad9c8462 AIP-84 post variable (#42948)
add 2b101e2377f Feature: Added event_handler parameter in
MSGraphAsyncOperator (#42539)
add f918f14854b uv version bump to 0.4.20 (#42905)
add 272cd9ac6c1 Removed deprecated Chainable type from BaseOperator
(#42776)
add c9414840660 pin in providers (#43001)
add cd1e9f5a048 Move tests_common from "dev" to top-level. (#42985)
add cc0aad0bab2 Add AssetActive model (#42612)
add 6c9c32710e1 Migrate the public endpoint Delete DAG to FastAPI (#42914)
add c471c311119 Clarifying PLUGINS_FOLDER permissions by DAG authors
(#43022)
add 2eaa90fd369 kubernetes package version bump to 31.0.0 (#42907)
add 732e0564d47 Make google provider pyarrow dependency explicit (#42996)
add f38d56dbf4d Add search by dag_display_name_pattern on dag list page
with rebase (#42896)
add 9bd1edca92e Deprecate session auth backend (#42909)
add 520c4436568 fix: Change CustomSecurityManager method name (#43034)
add 12f0b9f4463 Update to 0.4.21 of UV (#43032)
add 477d74718ad Add retry on error 502 and 504 (#42994)
add 1ca620896bf AIP-84 Migrate get connections to FastAPI API #42571
(#42782)
add fe0cc6e8302 Make `RedshiftDataOperator` handle multiple queries
(#42900)
add d634904028b Bump `uv` to `0.4.21` in other places (#43033)
add 59cf3efb8b2 require 1.2.1 common.compat for openlineage provider
(#43039)
add 330cdf0fb01 Detect system color theme and add active state to nav
button (#43041)
add f708fc92697 Migrate health info to fastapi (#42938)
add 7b151d32c80 Add upperbound to microsoft-kiota-abstractions (#43021)
add c9c4ca545bf fix path to providers dir for mount sources breeze flag
(#43042)
add f1f9201a219 Create Operators for Google Cloud Vertex AI Context
Caching (#43008)
add 2899e5f0e27 add lowerbount to requests-toolbelt and replace
requests_toolbelt with requests-toolbelt (#43020)
add e939c42d4f6 ✨ Allow node_selector templating in KPO (#43051)
add 3c3973eb81a Rename views to routes for FastAPI apps (#43057)
add 6c3bc91f42e Follow-up SLA purge (#42808)
add 5101b7ea785 Bump `uv` to `0.4.22` (#43056)
add 1af5b93157a Add 'name' and 'group' to public Asset class (#42812)
add 6549b1798f6 Remove zombie ti from its executor (#42932)
add e20146d44b3 Drop unneeded unique() call on SQL (#43064)
add 80f8a0e05be Allow building docs for not-ready providers (#43071)
add aaa5f356666 Typo correction in 07_local_virtualenv.rst - "way" (#43072)
add 251fc5818c5 Enable explicit namespaces (#42951)
new ce77b025755 Start porting over all the DAG defintion code to the Task
SDK
new b0098baa7be Start replacing DAG class with the task-sdk DAG class
instead
new 32a6f3f6bf2 Move over more of BaseOperator and DAG, along with their
tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (50081fada27)
\
N -- N -- N refs/heads/task-sdk-first-code (32a6f3f6bf2)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 3 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.dockerignore | 1 +
.pre-commit-config.yaml | 12 +-
Dockerfile | 2 +-
Dockerfile.ci | 8 +-
airflow/__init__.py | 11 +-
airflow/api/auth/backend/session.py | 31 +-
.../api_connexion/endpoints/connection_endpoint.py | 1 +
airflow/api_connexion/endpoints/dag_endpoint.py | 2 +
airflow/api_connexion/endpoints/health_endpoint.py | 2 +
.../endpoints/task_instance_endpoint.py | 96 +-
.../api_connexion/endpoints/variable_endpoint.py | 2 +
.../api_connexion/schemas/task_instance_schema.py | 21 +-
airflow/api_fastapi/app.py | 4 +-
airflow/api_fastapi/openapi/v1-generated.yaml | 366 +++-
airflow/api_fastapi/parameters.py | 48 +-
airflow/api_fastapi/{views => routes}/__init__.py | 0
.../{views => routes}/public/__init__.py | 12 +-
.../{views => routes}/public/connections.py | 44 +-
.../{views => routes}/public/dag_run.py | 2 +-
.../api_fastapi/{views => routes}/public/dags.py | 26 +-
.../routes/public/monitor.py} | 15 +-
.../{views => routes}/public/variables.py | 43 +-
airflow/api_fastapi/{views => routes}/router.py | 0
.../api_fastapi/{views => routes}/ui/__init__.py | 6 +-
airflow/api_fastapi/{views => routes}/ui/assets.py | 4 +-
.../api_fastapi/{views => routes}/ui/dashboard.py | 2 +-
airflow/api_fastapi/serializers/connections.py | 9 +-
airflow/api_fastapi/serializers/dags.py | 2 +-
airflow/api_fastapi/serializers/dashboard.py | 4 +-
.../serializers/monitor.py} | 42 +-
airflow/api_fastapi/serializers/variables.py | 17 +-
airflow/api_internal/internal_api_call.py | 33 +-
airflow/assets/__init__.py | 73 +-
.../auth/managers/simple/simple_auth_manager.py | 20 +-
airflow/cli/cli_config.py | 2 +-
airflow/dag_processing/collection.py | 20 +-
airflow/example_dags/tutorial.py | 1 -
airflow/exceptions.py | 25 -
airflow/jobs/backfill_job_runner.py | 1106 -----------
airflow/jobs/job.py | 4 +-
airflow/jobs/scheduler_job_runner.py | 168 +-
.../0035_3_0_0_add_name_field_to_dataset_model.py | 4 +-
.../versions/0037_3_0_0_add_asset_active.py | 88 +
airflow/models/__init__.py | 3 -
airflow/models/asset.py | 79 +-
airflow/models/dag.py | 4 +-
airflow/models/dagpickle.py | 2 +-
airflow/models/dagrun.py | 2 +-
airflow/models/slamiss.py | 46 -
airflow/models/taskinstance.py | 12 +-
airflow/notifications/basenotifier.py | 8 +-
airflow/serialization/pydantic/asset.py | 1 -
airflow/ti_deps/dependencies_deps.py | 8 -
airflow/ti_deps/dependencies_states.py | 9 -
airflow/ui/openapi-gen/queries/common.ts | 46 +
airflow/ui/openapi-gen/queries/prefetch.ts | 41 +
airflow/ui/openapi-gen/queries/queries.ts | 180 +-
airflow/ui/openapi-gen/queries/suspense.ts | 55 +
airflow/ui/openapi-gen/requests/schemas.gen.ts | 207 +-
airflow/ui/openapi-gen/requests/services.gen.ts | 138 ++
airflow/ui/openapi-gen/requests/types.gen.ts | 216 +-
airflow/ui/package.json | 3 +-
airflow/ui/pnpm-lock.yaml | 13 +
.../ui/src/components/DataTable/searchParams.ts | 13 +-
airflow/ui/src/components/SearchBar.tsx | 62 +-
.../NavButton.tsx => constants/searchParams.ts} | 31 +-
airflow/ui/src/layouts/Nav/Nav.tsx | 13 +-
airflow/ui/src/layouts/Nav/NavButton.tsx | 18 +-
airflow/ui/src/pages/DagsList/DagsFilters.tsx | 16 +-
airflow/ui/src/pages/DagsList/DagsList.tsx | 78 +-
airflow/ui/src/theme.ts | 1 +
airflow/utils/api_migration.py | 2 +-
airflow/utils/context.py | 3 +
airflow/utils/db.py | 2 +-
airflow/www/auth.py | 4 +-
airflow/www/extensions/init_views.py | 3 -
airflow/www/views.py | 119 +-
contributing-docs/07_local_virtualenv.rst | 2 +-
contributing-docs/14_node_environment_setup.rst | 4 +-
contributing-docs/testing/system_tests.rst | 2 +-
contributing-docs/testing/unit_tests.rst | 6 +-
dev/README_AIRFLOW3_DEV.md | 2 +-
dev/breeze/doc/ci/02_images.md | 58 +-
.../commands/release_management_commands.py | 2 +-
dev/breeze/src/airflow_breeze/global_constants.py | 3 +
.../airflow_breeze/utils/docker_command_utils.py | 1 +
.../src/airflow_breeze/utils/selective_checks.py | 9 +-
dev/breeze/tests/test_selective_checks.py | 4 +-
dev/perf/scheduler_dag_execution_timing.py | 2 +-
.../auth-manager/webserver-authentication.rst | 2 +-
.../operators/cloud/vertex_ai.rst | 20 +
docs/apache-airflow-providers-http/index.rst | 2 +-
.../administration-and-deployment/listeners.rst | 2 +-
docs/apache-airflow/core-concepts/tasks.rst | 4 +-
docs/apache-airflow/howto/listener-plugin.rst | 4 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/airflow_erd.svg | 1253 ++++++------
docs/apache-airflow/migrations-ref.rst | 4 +-
docs/apache-airflow/security/security_model.rst | 5 +-
docs/conf.py | 1 +
docs/exts/provider_yaml_utils.py | 2 +-
docs/spelling_wordlist.txt | 1 -
generated/provider_dependencies.json | 22 +-
generated/provider_metadata.json | 358 ++--
helm_tests/airflow_aux/test_pod_template_file.py | 1 +
newsfragments/42776.significant.rst | 1 +
.../src/airflow/providers}/__init__.py | 10 +-
.../amazon/aws/operators/redshift_data.py | 63 +-
.../providers/apache/beam/operators/beam.py | 19 +-
.../airflow/providers/apache/livy/provider.yaml | 2 +-
.../providers/cncf/kubernetes/operators/pod.py | 3 +-
.../providers/cncf/kubernetes/provider.yaml | 4 +-
.../src/airflow/providers/dbt/cloud/provider.yaml | 2 +-
.../cloud/hooks/vertex_ai/generative_model.py | 79 +
.../cloud/operators/vertex_ai/generative_model.py | 156 +-
.../src/airflow/providers/google/provider.yaml | 4 +-
providers/src/airflow/providers/http/provider.yaml | 4 +-
.../providers/microsoft/azure/hooks/msgraph.py | 7 +
.../providers/microsoft/azure/operators/msgraph.py | 18 +-
.../providers/microsoft/azure/provider.yaml | 5 +
.../airflow/providers/openlineage/provider.yaml | 2 +-
.../alibaba/cloud/log/test_oss_task_handler.py | 5 +-
.../amazon/aws/auth_manager/avp/test_facade.py | 3 +-
.../aws/auth_manager/cli/test_avp_commands.py | 5 +-
.../test_aws_security_manager_override.py | 3 +-
.../aws/auth_manager/test_aws_auth_manager.py | 7 +-
.../amazon/aws/auth_manager/views/test_auth.py | 5 +-
.../aws/executors/batch/test_batch_executor.py | 5 +-
.../amazon/aws/executors/ecs/test_ecs_executor.py | 7 +-
providers/tests/amazon/aws/hooks/test_base_aws.py | 3 +-
providers/tests/amazon/aws/hooks/test_s3.py | 3 +-
providers/tests/amazon/aws/links/test_base_aws.py | 5 +-
.../amazon/aws/log/test_cloudwatch_task_handler.py | 3 +-
.../tests/amazon/aws/log/test_s3_task_handler.py | 3 +-
.../amazon/aws/operators/test_redshift_data.py | 95 +-
.../amazon/aws/secrets/test_systems_manager.py | 3 +-
.../amazon/aws/transfers/test_redshift_to_s3.py | 3 +-
.../amazon/aws/transfers/test_s3_to_redshift.py | 3 +-
.../tests/amazon/aws/transfers/test_s3_to_sftp.py | 3 +-
.../tests/amazon/aws/transfers/test_sftp_to_s3.py | 3 +-
providers/tests/apache/beam/operators/test_beam.py | 21 +
providers/tests/apache/hive/hooks/test_hive.py | 2 +-
providers/tests/apache/livy/hooks/test_livy.py | 3 +-
.../tests/apache/spark/hooks/test_spark_sql.py | 3 +-
providers/tests/atlassian/jira/hooks/test_jira.py | 3 +-
.../tests/atlassian/jira/operators/test_jira.py | 3 +-
.../tests/atlassian/jira/sensors/test_jira.py | 3 +-
providers/tests/celery/cli/test_celery_command.py | 5 +-
.../tests/celery/executors/test_celery_executor.py | 7 +-
.../tests/celery/log_handlers/test_log_handlers.py | 5 +-
.../cncf/kubernetes/cli/test_kubernetes_command.py | 3 +-
.../executors/test_kubernetes_executor.py | 5 +-
.../tests/cncf/kubernetes/hooks/test_kubernetes.py | 5 +-
.../kubernetes/log_handlers/test_log_handlers.py | 5 +-
.../tests/cncf/kubernetes/operators/test_pod.py | 3 +-
.../kubernetes/operators/test_spark_kubernetes.py | 3 +-
providers/tests/cncf/kubernetes/test_client.py | 3 +-
.../cncf/kubernetes/test_template_rendering.py | 3 +-
.../common/io/operators/test_file_transfer.py | 4 +-
providers/tests/common/io/xcom/test_backend.py | 11 +-
providers/tests/common/sql/hooks/test_dbapi.py | 3 +-
providers/tests/common/sql/hooks/test_sql.py | 2 +-
providers/tests/common/sql/hooks/test_sqlparse.py | 3 +-
providers/tests/common/sql/operators/test_sql.py | 3 +-
.../tests/common/sql/operators/test_sql_execute.py | 3 +-
providers/tests/common/sql/sensors/test_sql.py | 3 +-
providers/tests/common/sql/test_utils.py | 3 +-
providers/tests/conftest.py | 2 +-
.../databricks/plugins/test_databricks_workflow.py | 3 +-
.../edge/api_endpoints/test_rpc_api_endpoint.py | 5 +-
providers/tests/edge/cli/test_edge_command.py | 3 +-
.../edge/plugins/test_edge_executor_plugin.py | 3 +-
.../elasticsearch/log/test_es_task_handler.py | 4 +-
.../api/auth/backend/test_basic_auth.py | 3 +-
.../api/auth/backend/test_kerberos_auth.py | 2 +-
.../auth_manager/api/auth/backend/test_session.py | 3 +-
.../api_endpoints/api_connexion_utils.py | 2 +-
.../api_endpoints/test_asset_endpoint.py | 6 +-
.../fab/auth_manager/api_endpoints/test_auth.py | 13 +-
.../api_endpoints/test_backfill_endpoint.py | 14 +-
.../fab/auth_manager/api_endpoints/test_cors.py | 7 +-
.../api_endpoints/test_dag_endpoint.py | 6 +-
.../api_endpoints/test_dag_run_endpoint.py | 4 +-
.../api_endpoints/test_dag_source_endpoint.py | 4 +-
.../api_endpoints/test_dag_warning_endpoint.py | 4 +-
.../api_endpoints/test_event_log_endpoint.py | 4 +-
.../api_endpoints/test_import_error_endpoint.py | 6 +-
.../test_role_and_permission_endpoint.py | 4 +-
.../api_endpoints/test_task_instance_endpoint.py | 5 +-
.../api_endpoints/test_user_endpoint.py | 6 +-
.../api_endpoints/test_variable_endpoint.py | 4 +-
.../api_endpoints/test_xcom_endpoint.py | 4 +-
.../auth_manager/cli_commands/test_definition.py | 2 +-
.../auth_manager/cli_commands/test_role_command.py | 5 +-
.../cli_commands/test_sync_perm_command.py | 3 +-
.../auth_manager/cli_commands/test_user_command.py | 3 +-
.../fab/auth_manager/cli_commands/test_utils.py | 3 +-
providers/tests/fab/auth_manager/conftest.py | 7 +-
.../tests/fab/auth_manager/decorators/test_auth.py | 3 +-
.../fab/auth_manager/models/test_anonymous_user.py | 2 +-
.../fab/auth_manager/schemas/test_user_schema.py | 2 +-
.../security_manager/test_constants.py | 2 +-
.../auth_manager/security_manager/test_override.py | 2 +-
.../fab/auth_manager/test_fab_auth_manager.py | 2 +-
providers/tests/fab/auth_manager/test_models.py | 3 +-
providers/tests/fab/auth_manager/test_security.py | 12 +-
.../fab/auth_manager/views/test_permissions.py | 4 +-
.../fab/auth_manager/views/test_roles_list.py | 4 +-
.../tests/fab/auth_manager/views/test_user.py | 4 +-
.../tests/fab/auth_manager/views/test_user_edit.py | 4 +-
.../fab/auth_manager/views/test_user_stats.py | 4 +-
.../google/cloud/hooks/test_bigquery_system.py | 2 +-
.../tests/google/cloud/hooks/test_kms_system.py | 2 +-
.../cloud/hooks/test_secret_manager_system.py | 2 +-
.../cloud/hooks/vertex_ai/test_generative_model.py | 60 +-
.../google/cloud/log/test_gcs_task_handler.py | 5 +-
.../cloud/log/test_gcs_task_handler_system.py | 14 +-
.../cloud/log/test_stackdriver_task_handler.py | 7 +-
.../log/test_stackdriver_task_handler_system.py | 14 +-
.../tests/google/cloud/operators/test_bigquery.py | 13 +-
.../google/cloud/operators/test_bigquery_dts.py | 3 +-
.../google/cloud/operators/test_dataprep_system.py | 7 +-
.../tests/google/cloud/operators/test_dataproc.py | 5 +-
.../cloud/operators/test_datastore_system.py | 4 +-
.../tests/google/cloud/operators/test_looker.py | 3 +-
.../operators/vertex_ai/test_generative_model.py | 84 +-
.../cloud/secrets/test_secret_manager_system.py | 2 +-
.../transfers/test_facebook_ads_to_gcs_system.py | 10 +-
.../transfers/test_salesforce_to_gcs_system.py | 12 +-
.../tests/google/cloud/utils/gcp_authenticator.py | 6 +-
.../common/auth_backend/test_google_openid.py | 9 +-
.../operators/test_display_video_system.py | 12 +-
providers/tests/hashicorp/hooks/test_vault.py | 3 +-
providers/tests/imap/hooks/test_imap.py | 3 +-
.../tests/microsoft/azure/hooks/test_msgraph.py | 45 +-
.../microsoft/azure/log/test_wasb_task_handler.py | 5 +-
.../microsoft/azure/operators/test_msgraph.py | 30 +
providers/tests/mongo/hooks/test_mongo.py | 3 +-
providers/tests/mysql/hooks/test_mysql.py | 4 +-
.../tests/openlineage/extractors/test_bash.py | 3 +-
.../tests/openlineage/extractors/test_manager.py | 3 +-
.../tests/openlineage/extractors/test_python.py | 3 +-
.../tests/openlineage/plugins/test_adapter.py | 5 +-
.../tests/openlineage/plugins/test_execution.py | 5 +-
.../tests/openlineage/plugins/test_listener.py | 5 +-
.../tests/openlineage/plugins/test_openlineage.py | 5 +-
providers/tests/openlineage/plugins/test_utils.py | 3 +-
providers/tests/openlineage/test_conf.py | 3 +-
providers/tests/openlineage/utils/test_utils.py | 8 +-
.../tests/opensearch/log/test_os_task_handler.py | 4 +-
.../tests/redis/log/test_redis_task_handler.py | 3 +-
providers/tests/sftp/operators/test_sftp.py | 3 +-
providers/tests/smtp/hooks/test_smtp.py | 3 +-
providers/tests/smtp/notifications/test_smtp.py | 38 +-
providers/tests/ssh/operators/test_ssh.py | 3 +-
providers/tests/standard/operators/test_bash.py | 3 +-
.../tests/standard/operators/test_datetime.py | 3 +-
providers/tests/standard/operators/test_weekday.py | 3 +-
providers/tests/standard/sensors/test_weekday.py | 3 +-
.../system/airbyte/example_airbyte_trigger_job.py | 2 +-
.../system/alibaba/example_adb_spark_batch.py | 4 +-
.../tests/system/alibaba/example_adb_spark_sql.py | 4 +-
.../tests/system/alibaba/example_oss_bucket.py | 4 +-
.../tests/system/alibaba/example_oss_object.py | 4 +-
providers/tests/system/amazon/CONTRIBUTING.md | 2 +-
.../tests/system/amazon/aws/example_appflow.py | 4 +-
.../tests/system/amazon/aws/example_appflow_run.py | 4 +-
.../tests/system/amazon/aws/example_athena.py | 4 +-
.../system/amazon/aws/example_azure_blob_to_s3.py | 4 +-
providers/tests/system/amazon/aws/example_batch.py | 4 +-
.../tests/system/amazon/aws/example_bedrock.py | 4 +-
.../aws/example_bedrock_retrieve_and_generate.py | 4 +-
.../system/amazon/aws/example_cloudformation.py | 4 +-
.../tests/system/amazon/aws/example_comprehend.py | 4 +-
.../aws/example_comprehend_document_classifier.py | 4 +-
.../tests/system/amazon/aws/example_datasync.py | 4 +-
providers/tests/system/amazon/aws/example_dms.py | 4 +-
.../tests/system/amazon/aws/example_dynamodb.py | 4 +-
.../system/amazon/aws/example_dynamodb_to_s3.py | 4 +-
providers/tests/system/amazon/aws/example_ec2.py | 4 +-
providers/tests/system/amazon/aws/example_ecs.py | 4 +-
.../tests/system/amazon/aws/example_ecs_fargate.py | 4 +-
.../system/amazon/aws/example_eks_templated.py | 4 +-
.../aws/example_eks_with_fargate_in_one_step.py | 4 +-
.../amazon/aws/example_eks_with_fargate_profile.py | 4 +-
.../aws/example_eks_with_nodegroup_in_one_step.py | 4 +-
.../amazon/aws/example_eks_with_nodegroups.py | 4 +-
providers/tests/system/amazon/aws/example_emr.py | 4 +-
.../tests/system/amazon/aws/example_emr_eks.py | 4 +-
.../amazon/aws/example_emr_notebook_execution.py | 4 +-
.../system/amazon/aws/example_emr_serverless.py | 4 +-
.../tests/system/amazon/aws/example_eventbridge.py | 2 +-
.../tests/system/amazon/aws/example_ftp_to_s3.py | 4 +-
.../tests/system/amazon/aws/example_gcs_to_s3.py | 4 +-
.../system/amazon/aws/example_glacier_to_gcs.py | 4 +-
providers/tests/system/amazon/aws/example_glue.py | 4 +-
.../system/amazon/aws/example_glue_data_quality.py | 4 +-
...xample_glue_data_quality_with_recommendation.py | 4 +-
.../system/amazon/aws/example_glue_databrew.py | 4 +-
.../amazon/aws/example_google_api_sheets_to_s3.py | 4 +-
.../amazon/aws/example_google_api_youtube_to_s3.py | 4 +-
.../system/amazon/aws/example_hive_to_dynamodb.py | 4 +-
.../tests/system/amazon/aws/example_http_to_s3.py | 4 +-
.../amazon/aws/example_imap_attachment_to_s3.py | 4 +-
.../system/amazon/aws/example_kinesis_analytics.py | 4 +-
.../tests/system/amazon/aws/example_lambda.py | 4 +-
.../tests/system/amazon/aws/example_local_to_s3.py | 4 +-
.../tests/system/amazon/aws/example_mongo_to_s3.py | 4 +-
.../tests/system/amazon/aws/example_neptune.py | 4 +-
.../tests/system/amazon/aws/example_quicksight.py | 4 +-
.../tests/system/amazon/aws/example_rds_event.py | 4 +-
.../tests/system/amazon/aws/example_rds_export.py | 4 +-
.../system/amazon/aws/example_rds_instance.py | 4 +-
.../system/amazon/aws/example_rds_snapshot.py | 4 +-
.../tests/system/amazon/aws/example_redshift.py | 42 +-
.../amazon/aws/example_redshift_s3_transfers.py | 4 +-
providers/tests/system/amazon/aws/example_s3.py | 4 +-
.../system/amazon/aws/example_s3_to_dynamodb.py | 4 +-
.../tests/system/amazon/aws/example_s3_to_ftp.py | 4 +-
.../tests/system/amazon/aws/example_s3_to_sftp.py | 4 +-
.../tests/system/amazon/aws/example_s3_to_sql.py | 5 +-
.../tests/system/amazon/aws/example_sagemaker.py | 4 +-
.../amazon/aws/example_sagemaker_endpoint.py | 4 +-
.../amazon/aws/example_sagemaker_notebook.py | 4 +-
.../amazon/aws/example_sagemaker_pipeline.py | 4 +-
.../system/amazon/aws/example_salesforce_to_s3.py | 4 +-
.../tests/system/amazon/aws/example_sftp_to_s3.py | 4 +-
providers/tests/system/amazon/aws/example_sns.py | 4 +-
.../tests/system/amazon/aws/example_sql_to_s3.py | 4 +-
providers/tests/system/amazon/aws/example_sqs.py | 4 +-
.../system/amazon/aws/example_step_functions.py | 4 +-
.../amazon/aws/tests/test_aws_auth_manager.py | 4 +-
providers/tests/system/apache/beam/example_beam.py | 2 +-
.../system/apache/beam/example_beam_java_flink.py | 2 +-
.../system/apache/beam/example_beam_java_spark.py | 2 +-
providers/tests/system/apache/beam/example_go.py | 2 +-
.../system/apache/beam/example_go_dataflow.py | 2 +-
.../system/apache/beam/example_java_dataflow.py | 2 +-
.../tests/system/apache/beam/example_python.py | 2 +-
.../system/apache/beam/example_python_async.py | 2 +-
.../system/apache/beam/example_python_dataflow.py | 2 +-
.../apache/cassandra/example_cassandra_dag.py | 2 +-
.../tests/system/apache/drill/example_drill_dag.py | 2 +-
.../tests/system/apache/druid/example_druid_dag.py | 2 +-
.../system/apache/hive/example_twitter_dag.py | 4 +-
.../tests/system/apache/iceberg/example_iceberg.py | 2 +-
.../apache/kafka/example_dag_event_listener.py | 2 +-
.../system/apache/kafka/example_dag_hello_kafka.py | 2 +-
.../tests/system/apache/kylin/example_kylin_dag.py | 4 +-
providers/tests/system/apache/livy/example_livy.py | 4 +-
providers/tests/system/apache/pig/example_pig.py | 2 +-
.../tests/system/apache/pinot/example_pinot_dag.py | 2 +-
.../tests/system/apache/spark/example_pyspark.py | 2 +-
.../tests/system/apache/spark/example_spark_dag.py | 2 +-
providers/tests/system/asana/example_asana.py | 4 +-
.../system/cncf/kubernetes/example_kubernetes.py | 4 +-
.../cncf/kubernetes/example_kubernetes_async.py | 4 +-
.../kubernetes/example_kubernetes_decorator.py | 2 +-
.../cncf/kubernetes/example_kubernetes_job.py | 4 +-
.../cncf/kubernetes/example_kubernetes_resource.py | 4 +-
.../cncf/kubernetes/example_spark_kubernetes.py | 4 +-
.../cohere/example_cohere_embedding_operator.py | 2 +-
.../common/io/example_file_transfer_local_to_s3.py | 4 +-
.../common/sql/example_sql_column_table_check.py | 2 +-
.../system/common/sql/example_sql_execute_query.py | 2 +-
.../tests/system/databricks/example_databricks.py | 4 +-
.../system/databricks/example_databricks_repos.py | 4 +-
.../databricks/example_databricks_sensors.py | 4 +-
.../system/databricks/example_databricks_sql.py | 4 +-
.../databricks/example_databricks_workflow.py | 4 +-
.../tests/system/dbt/cloud/example_dbt_cloud.py | 8 +-
.../tests/system/dingding/example_dingding.py | 4 +-
providers/tests/system/docker/example_docker.py | 2 +-
.../system/docker/example_docker_copy_data.py | 2 +-
.../tests/system/docker/example_docker_swarm.py | 2 +-
.../example_taskflow_api_docker_virtualenv.py | 8 +-
.../elasticsearch/example_elasticsearch_query.py | 2 +-
providers/tests/system/ftp/example_ftp.py | 4 +-
providers/tests/system/github/example_github.py | 2 +-
providers/tests/system/google/ads/example_ads.py | 4 +-
.../google/cloud/automl/example_automl_dataset.py | 4 +-
.../cloud/automl/example_automl_translation.py | 4 +-
.../automl/example_automl_video_classification.py | 4 +-
.../cloud/automl/example_automl_video_tracking.py | 4 +-
.../automl/example_automl_vision_classification.py | 4 +-
.../example_automl_vision_object_detection.py | 4 +-
.../cloud/azure/example_azure_blob_to_gcs.py | 4 +-
.../cloud/azure/example_azure_fileshare_to_gcs.py | 4 +-
.../cloud/bigquery/example_bigquery_dataset.py | 4 +-
.../google/cloud/bigquery/example_bigquery_dts.py | 4 +-
.../cloud/bigquery/example_bigquery_operations.py | 4 +-
.../example_bigquery_operations_location.py | 4 +-
.../cloud/bigquery/example_bigquery_queries.py | 4 +-
.../bigquery/example_bigquery_queries_async.py | 4 +-
.../cloud/bigquery/example_bigquery_sensors.py | 4 +-
.../cloud/bigquery/example_bigquery_tables.py | 4 +-
.../cloud/bigquery/example_bigquery_to_bigquery.py | 4 +-
.../cloud/bigquery/example_bigquery_to_gcs.py | 4 +-
.../bigquery/example_bigquery_to_gcs_async.py | 4 +-
.../cloud/bigquery/example_bigquery_to_mssql.py | 4 +-
.../cloud/bigquery/example_bigquery_to_mysql.py | 4 +-
.../cloud/bigquery/example_bigquery_to_postgres.py | 4 +-
.../cloud/bigquery/example_bigquery_transfer.py | 4 +-
.../cloud/bigquery/example_bigquery_value_check.py | 6 +-
.../google/cloud/bigtable/example_bigtable.py | 4 +-
.../cloud/cloud_batch/example_cloud_batch.py | 4 +-
.../cloud/cloud_build/example_cloud_build.py | 4 +-
.../cloud_build/example_cloud_build_trigger.py | 4 +-
.../cloud/cloud_functions/example_functions.py | 4 +-
.../example_cloud_memorystore_memcached.py | 4 +-
.../example_cloud_memorystore_redis.py | 4 +-
.../google/cloud/cloud_run/example_cloud_run.py | 4 +-
.../cloud/cloud_run/example_cloud_run_service.py | 4 +-
.../google/cloud/cloud_sql/example_cloud_sql.py | 4 +-
.../cloud/cloud_sql/example_cloud_sql_query.py | 4 +-
.../cloud/cloud_sql/example_cloud_sql_query_ssl.py | 4 +-
.../cloud/composer/example_cloud_composer.py | 4 +-
.../system/google/cloud/compute/example_compute.py | 4 +-
.../google/cloud/compute/example_compute_igm.py | 4 +-
.../google/cloud/compute/example_compute_ssh.py | 4 +-
.../cloud/compute/example_compute_ssh_os_login.py | 4 +-
.../cloud/compute/example_compute_ssh_parallel.py | 4 +-
.../example_dlp_deidentify_content.py | 4 +-
.../data_loss_prevention/example_dlp_info_types.py | 4 +-
.../example_dlp_inspect_template.py | 4 +-
.../cloud/data_loss_prevention/example_dlp_job.py | 4 +-
.../example_dlp_job_trigger.py | 4 +-
.../google/cloud/dataflow/example_dataflow_go.py | 2 +-
.../cloud/dataflow/example_dataflow_native_java.py | 4 +-
.../dataflow/example_dataflow_native_python.py | 4 +-
.../example_dataflow_native_python_async.py | 4 +-
.../cloud/dataflow/example_dataflow_pipeline.py | 4 +-
.../example_dataflow_sensors_deferrable.py | 4 +-
.../google/cloud/dataflow/example_dataflow_sql.py | 4 +-
.../dataflow/example_dataflow_streaming_python.py | 4 +-
.../cloud/dataflow/example_dataflow_template.py | 4 +-
.../google/cloud/dataflow/example_dataflow_yaml.py | 4 +-
.../google/cloud/dataform/example_dataform.py | 4 +-
.../google/cloud/datafusion/example_datafusion.py | 4 +-
.../cloud/datapipelines/example_datapipeline.py | 4 +-
.../google/cloud/dataplex/example_dataplex.py | 4 +-
.../google/cloud/dataplex/example_dataplex_dp.py | 4 +-
.../google/cloud/dataplex/example_dataplex_dq.py | 4 +-
.../google/cloud/dataprep/example_dataprep.py | 4 +-
.../cloud/dataproc/example_dataproc_batch.py | 4 +-
.../dataproc/example_dataproc_batch_deferrable.py | 4 +-
.../dataproc/example_dataproc_batch_persistent.py | 4 +-
...proc_cluster_create_existing_stopped_cluster.py | 4 +-
.../example_dataproc_cluster_deferrable.py | 4 +-
.../dataproc/example_dataproc_cluster_diagnose.py | 4 +-
.../dataproc/example_dataproc_cluster_generator.py | 4 +-
.../example_dataproc_cluster_start_stop.py | 4 +-
.../dataproc/example_dataproc_cluster_update.py | 4 +-
.../cloud/dataproc/example_dataproc_flink.py | 4 +-
.../google/cloud/dataproc/example_dataproc_gke.py | 4 +-
.../cloud/dataproc/example_dataproc_hadoop.py | 4 +-
.../google/cloud/dataproc/example_dataproc_hive.py | 4 +-
.../google/cloud/dataproc/example_dataproc_pig.py | 4 +-
.../cloud/dataproc/example_dataproc_presto.py | 4 +-
.../cloud/dataproc/example_dataproc_pyspark.py | 4 +-
.../cloud/dataproc/example_dataproc_spark.py | 4 +-
.../cloud/dataproc/example_dataproc_spark_async.py | 4 +-
.../dataproc/example_dataproc_spark_deferrable.py | 4 +-
.../cloud/dataproc/example_dataproc_spark_sql.py | 4 +-
.../cloud/dataproc/example_dataproc_sparkr.py | 4 +-
.../cloud/dataproc/example_dataproc_trino.py | 4 +-
.../cloud/dataproc/example_dataproc_workflow.py | 4 +-
.../example_dataproc_workflow_deferrable.py | 4 +-
.../example_dataproc_metastore.py | 4 +-
.../example_dataproc_metastore_backup.py | 4 +-
...ple_dataproc_metastore_hive_partition_sensor.py | 4 +-
.../cloud/datastore/example_datastore_commit.py | 4 +-
.../cloud/datastore/example_datastore_query.py | 4 +-
.../cloud/datastore/example_datastore_rollback.py | 4 +-
.../google/cloud/gcs/example_calendar_to_gcs.py | 4 +-
.../system/google/cloud/gcs/example_firestore.py | 4 +-
.../system/google/cloud/gcs/example_gcs_acl.py | 4 +-
.../google/cloud/gcs/example_gcs_copy_delete.py | 4 +-
.../system/google/cloud/gcs/example_gcs_sensor.py | 4 +-
.../google/cloud/gcs/example_gcs_to_bigquery.py | 4 +-
.../cloud/gcs/example_gcs_to_bigquery_async.py | 4 +-
.../system/google/cloud/gcs/example_gcs_to_gcs.py | 4 +-
.../google/cloud/gcs/example_gcs_to_gdrive.py | 4 +-
.../google/cloud/gcs/example_gcs_to_sheets.py | 4 +-
.../google/cloud/gcs/example_gcs_transform.py | 4 +-
.../cloud/gcs/example_gcs_transform_timespan.py | 4 +-
.../cloud/gcs/example_gcs_upload_download.py | 4 +-
.../google/cloud/gcs/example_gdrive_to_gcs.py | 4 +-
.../google/cloud/gcs/example_mssql_to_gcs.py | 4 +-
.../google/cloud/gcs/example_mysql_to_gcs.py | 4 +-
.../google/cloud/gcs/example_oracle_to_gcs.py | 4 +-
.../system/google/cloud/gcs/example_s3_to_gcs.py | 4 +-
.../system/google/cloud/gcs/example_sftp_to_gcs.py | 4 +-
.../system/google/cloud/gcs/example_sheets.py | 4 +-
.../google/cloud/gcs/example_sheets_to_gcs.py | 4 +-
.../google/cloud/gcs/example_trino_to_gcs.py | 4 +-
.../kubernetes_engine/example_kubernetes_engine.py | 4 +-
.../example_kubernetes_engine_async.py | 4 +-
.../example_kubernetes_engine_job.py | 4 +-
.../example_kubernetes_engine_kueue.py | 4 +-
.../example_kubernetes_engine_resource.py | 4 +-
.../cloud/life_sciences/example_life_sciences.py | 4 +-
.../google/cloud/ml_engine/example_mlengine.py | 4 +-
.../natural_language/example_natural_language.py | 4 +-
.../system/google/cloud/pubsub/example_pubsub.py | 4 +-
.../cloud/pubsub/example_pubsub_deferrable.py | 4 +-
.../system/google/cloud/spanner/example_spanner.py | 4 +-
.../cloud/speech_to_text/example_speech_to_text.py | 4 +-
.../cloud/sql_to_sheets/example_sql_to_sheets.py | 4 +-
.../cloud/stackdriver/example_stackdriver.py | 4 +-
.../example_cloud_storage_transfer_service_aws.py | 4 +-
.../example_cloud_storage_transfer_service_gcp.py | 4 +-
...le_cloud_storage_transfer_service_gcs_to_gcs.py | 4 +-
.../system/google/cloud/tasks/example_queue.py | 4 +-
.../system/google/cloud/tasks/example_tasks.py | 4 +-
.../cloud/text_to_speech/example_text_to_speech.py | 4 +-
.../google/cloud/transfers/example_gcs_to_sftp.py | 4 +-
.../cloud/transfers/example_gdrive_to_local.py | 4 +-
.../cloud/transfers/example_postgres_to_gcs.py | 4 +-
.../google/cloud/translate/example_translate.py | 4 +-
.../translate_speech/example_translate_speech.py | 4 +-
...ample_vertex_ai_auto_ml_forecasting_training.py | 4 +-
.../example_vertex_ai_auto_ml_image_training.py | 4 +-
.../example_vertex_ai_auto_ml_list_training.py | 4 +-
.../example_vertex_ai_auto_ml_tabular_training.py | 4 +-
.../example_vertex_ai_auto_ml_video_training.py | 4 +-
.../example_vertex_ai_batch_prediction_job.py | 4 +-
.../example_vertex_ai_custom_container.py | 4 +-
.../vertex_ai/example_vertex_ai_custom_job.py | 4 +-
.../example_vertex_ai_custom_job_python_package.py | 4 +-
.../cloud/vertex_ai/example_vertex_ai_dataset.py | 4 +-
.../cloud/vertex_ai/example_vertex_ai_endpoint.py | 4 +-
.../example_vertex_ai_generative_model.py | 52 +-
.../example_vertex_ai_generative_model_tuning.py | 4 +-
.../example_vertex_ai_hyperparameter_tuning_job.py | 4 +-
.../example_vertex_ai_list_custom_jobs.py | 4 +-
.../vertex_ai/example_vertex_ai_model_service.py | 4 +-
.../vertex_ai/example_vertex_ai_pipeline_job.py | 4 +-
.../example_video_intelligence.py | 4 +-
.../cloud/vision/example_vision_annotate_image.py | 4 +-
.../cloud/vision/example_vision_autogenerated.py | 4 +-
.../google/cloud/vision/example_vision_explicit.py | 4 +-
.../google/cloud/workflows/example_workflows.py | 4 +-
.../datacatalog/example_datacatalog_entries.py | 4 +-
.../example_datacatalog_search_catalog.py | 4 +-
.../example_datacatalog_tag_templates.py | 4 +-
.../google/datacatalog/example_datacatalog_tags.py | 4 +-
.../tests/system/google/leveldb/example_leveldb.py | 4 +-
.../marketing_platform/example_analytics_admin.py | 4 +-
.../marketing_platform/example_campaign_manager.py | 4 +-
.../marketing_platform/example_search_ads.py | 2 +-
.../system/google/suite/example_local_to_drive.py | 4 +-
providers/tests/system/http/example_http.py | 2 +-
.../tests/system/influxdb/example_influxdb.py | 4 +-
.../system/influxdb/example_influxdb_query.py | 2 +-
.../tests/system/jdbc/example_jdbc_queries.py | 4 +-
.../system/jenkins/example_jenkins_job_trigger.py | 2 +-
.../microsoft/azure/example_adf_run_pipeline.py | 4 +-
.../system/microsoft/azure/example_adls_create.py | 4 +-
.../system/microsoft/azure/example_adls_delete.py | 4 +-
.../system/microsoft/azure/example_adls_list.py | 4 +-
.../azure/example_azure_batch_operator.py | 2 +-
.../azure/example_azure_container_instances.py | 2 +-
.../microsoft/azure/example_azure_cosmosdb.py | 4 +-
.../microsoft/azure/example_azure_service_bus.py | 4 +-
.../microsoft/azure/example_azure_synapse.py | 2 +-
.../system/microsoft/azure/example_fileshare.py | 4 +-
.../microsoft/azure/example_local_to_adls.py | 4 +-
.../microsoft/azure/example_local_to_wasb.py | 4 +-
.../system/microsoft/azure/example_msfabric.py | 4 +-
.../system/microsoft/azure/example_msgraph.py | 4 +-
.../system/microsoft/azure/example_powerbi.py | 4 +-
.../azure/example_powerbi_dataset_refresh.py | 4 +-
.../system/microsoft/azure/example_s3_to_wasb.py | 4 +-
.../system/microsoft/azure/example_sftp_to_wasb.py | 4 +-
.../azure/example_synapse_run_pipeline.py | 4 +-
.../system/microsoft/azure/example_wasb_sensors.py | 2 +-
.../tests/system/microsoft/mssql/example_mssql.py | 4 +-
.../tests/system/microsoft/winrm/example_winrm.py | 4 +-
providers/tests/system/mysql/example_mysql.py | 2 +-
providers/tests/system/neo4j/example_neo4j.py | 2 +-
providers/tests/system/openai/example_openai.py | 2 +-
.../openai/example_trigger_batch_operator.py | 2 +-
.../tests/system/opensearch/example_opensearch.py | 4 +-
.../system/opsgenie/example_opsgenie_alert.py | 2 +-
.../system/opsgenie/example_opsgenie_notifier.py | 2 +-
.../tests/system/papermill/example_papermill.py | 2 +-
.../papermill/example_papermill_remote_verify.py | 2 +-
.../system/papermill/example_papermill_verify.py | 2 +-
.../tests/system/pgvector/example_pgvector.py | 2 +-
.../system/pgvector/example_pgvector_openai.py | 2 +-
.../system/pinecone/example_create_pod_index.py | 2 +-
.../pinecone/example_create_serverless_index.py | 2 +-
.../tests/system/pinecone/example_dag_pinecone.py | 2 +-
.../system/pinecone/example_pinecone_cohere.py | 2 +-
.../system/pinecone/example_pinecone_openai.py | 2 +-
.../tests/system/postgres/example_postgres.py | 4 +-
.../tests/system/presto/example_gcs_to_presto.py | 2 +-
.../tests/system/qdrant/example_dag_qdrant.py | 2 +-
.../tests/system/redis/example_redis_publish.py | 4 +-
providers/tests/system/salesforce/example_bulk.py | 2 +-
.../salesforce/example_salesforce_apex_rest.py | 2 +-
.../tests/system/samba/example_gcs_to_samba.py | 4 +-
providers/tests/system/sftp/example_sftp_sensor.py | 4 +-
.../system/singularity/example_singularity.py | 2 +-
providers/tests/system/slack/example_slack.py | 2 +-
.../tests/system/slack/example_slack_webhook.py | 2 +-
.../tests/system/slack/example_sql_to_slack.py | 2 +-
.../system/slack/example_sql_to_slack_webhook.py | 2 +-
.../snowflake/example_copy_into_snowflake.py | 2 +-
.../tests/system/snowflake/example_snowflake.py | 2 +-
.../system/snowflake/example_snowpark_decorator.py | 2 +-
.../system/snowflake/example_snowpark_operator.py | 2 +-
providers/tests/system/sqlite/example_sqlite.py | 4 +-
providers/tests/system/tableau/example_tableau.py | 2 +-
.../tests/system/telegram/example_telegram.py | 2 +-
.../example_azure_blob_to_teradata_transfer.py | 4 +-
.../teradata/example_s3_to_teradata_transfer.py | 4 +-
.../tests/system/teradata/example_ssl_teradata.py | 4 +-
.../tests/system/teradata/example_teradata.py | 4 +-
.../system/teradata/example_teradata_call_sp.py | 4 +-
.../teradata/example_teradata_compute_cluster.py | 4 +-
.../example_teradata_to_teradata_transfer.py | 4 +-
.../tests/system/trino/example_gcs_to_trino.py | 2 +-
providers/tests/system/trino/example_trino.py | 2 +-
.../system/weaviate/example_weaviate_cohere.py | 2 +-
.../example_weaviate_dynamic_mapping_dag.py | 2 +-
.../system/weaviate/example_weaviate_openai.py | 2 +-
.../system/weaviate/example_weaviate_operator.py | 2 +-
.../system/weaviate/example_weaviate_using_hook.py | 2 +-
.../weaviate/example_weaviate_vectorizer_dag.py | 2 +-
.../example_weaviate_without_vectorizer_dag.py | 2 +-
.../tests/system/yandex/example_yandexcloud.py | 7 +-
.../system/yandex/example_yandexcloud_dataproc.py | 8 +-
.../example_yandexcloud_dataproc_lightweight.py | 8 +-
.../tests/system/yandex/example_yandexcloud_yq.py | 8 +-
providers/tests/system/ydb/example_ydb.py | 4 +-
.../system/zendesk/example_zendesk_custom_get.py | 2 +-
providers/tests/yandex/hooks/test_yandex.py | 4 +-
providers/tests/yandex/links/test_yq.py | 5 +-
providers/tests/yandex/operators/test_yq.py | 3 +-
pyproject.toml | 8 +-
scripts/ci/docker-compose/local.yml | 3 +
.../docker-compose/providers-and-tests-sources.yml | 2 +-
scripts/ci/pre_commit/check_system_tests.py | 8 +-
scripts/ci/pre_commit/draft7_schema.json | 172 ++
scripts/docker/entrypoint_ci.sh | 3 +
.../in_container/install_airflow_and_providers.py | 2 +
scripts/in_container/install_devel_deps.py | 2 +-
task_sdk/tests/conftest.py | 4 +-
tests/always/test_connection.py | 3 +-
tests/always/test_example_dags.py | 3 +-
tests/always/test_secrets.py | 5 +-
tests/always/test_secrets_backends.py | 3 +-
tests/always/test_secrets_local_filesystem.py | 3 +-
tests/api_connexion/conftest.py | 7 +-
.../api_connexion/endpoints/test_asset_endpoint.py | 11 +-
.../endpoints/test_backfill_endpoint.py | 5 +-
.../endpoints/test_config_endpoint.py | 4 +-
.../endpoints/test_connection_endpoint.py | 9 +-
tests/api_connexion/endpoints/test_dag_endpoint.py | 9 +-
tests/api_connexion/endpoints/test_dag_parsing.py | 5 +-
.../endpoints/test_dag_run_endpoint.py | 11 +-
.../endpoints/test_dag_source_endpoint.py | 5 +-
.../endpoints/test_dag_stats_endpoint.py | 5 +-
.../endpoints/test_dag_warning_endpoint.py | 5 +-
.../endpoints/test_event_log_endpoint.py | 7 +-
.../endpoints/test_extra_link_endpoint.py | 11 +-
.../endpoints/test_import_error_endpoint.py | 9 +-
tests/api_connexion/endpoints/test_log_endpoint.py | 5 +-
.../test_mapped_task_instance_endpoint.py | 9 +-
.../endpoints/test_plugin_endpoint.py | 9 +-
.../api_connexion/endpoints/test_pool_endpoint.py | 9 +-
.../endpoints/test_provider_endpoint.py | 3 +-
.../api_connexion/endpoints/test_task_endpoint.py | 5 +-
.../endpoints/test_task_instance_endpoint.py | 34 +-
.../endpoints/test_variable_endpoint.py | 9 +-
.../api_connexion/endpoints/test_xcom_endpoint.py | 7 +-
.../schemas/test_connection_schema.py | 3 +-
tests/api_connexion/schemas/test_dag_run_schema.py | 5 +-
tests/api_connexion/schemas/test_dataset_schema.py | 3 +-
tests/api_connexion/schemas/test_error_schema.py | 5 +-
tests/api_connexion/schemas/test_plugin_schema.py | 3 +-
tests/api_connexion/schemas/test_pool_schemas.py | 3 +-
.../schemas/test_task_instance_schema.py | 64 +-
tests/api_connexion/schemas/test_xcom_schema.py | 3 +-
tests/api_connexion/test_auth.py | 11 +-
tests/api_connexion/test_parameters.py | 3 +-
tests/api_connexion/test_security.py | 2 +-
.../api_fastapi/routes}/__init__.py | 0
.../api_fastapi/routes/public}/__init__.py | 0
.../{views => routes}/public/test_connections.py | 77 +-
.../{views => routes}/public/test_dag_run.py | 55 +-
tests/api_fastapi/routes/public/test_dags.py | 443 +++++
tests/api_fastapi/routes/public/test_monitor.py | 104 +
tests/api_fastapi/routes/public/test_variables.py | 274 +++
.../api_fastapi/routes/ui}/__init__.py | 0
.../{views => routes}/ui/test_assets.py | 3 +-
.../{views => routes}/ui/test_dashboard.py | 3 +-
tests/api_fastapi/views/__init__.py | 16 -
tests/api_fastapi/views/public/__init__.py | 16 -
tests/api_fastapi/views/public/test_dags.py | 351 ----
tests/api_fastapi/views/public/test_variables.py | 137 --
tests/api_fastapi/views/ui/__init__.py | 16 -
.../endpoints/test_rpc_api_endpoint.py | 5 +-
tests/api_internal/test_internal_api_call.py | 26 +-
tests/assets/{tests_asset.py => test_asset.py} | 43 +-
tests/assets/test_manager.py | 3 +-
.../managers/simple/test_simple_auth_manager.py | 4 +-
tests/auth/managers/simple/views/test_auth.py | 5 +-
tests/cli/commands/test_backfill_command.py | 3 +-
tests/cli/commands/test_celery_command.py | 3 +-
tests/cli/commands/test_config_command.py | 3 +-
tests/cli/commands/test_connection_command.py | 3 +-
tests/cli/commands/test_dag_command.py | 6 +-
tests/cli/commands/test_dag_processor_command.py | 3 +-
tests/cli/commands/test_fastapi_api_command.py | 1 +
tests/cli/commands/test_info_command.py | 3 +-
tests/cli/commands/test_internal_api_command.py | 4 +-
tests/cli/commands/test_jobs_command.py | 3 +-
tests/cli/commands/test_kerberos_command.py | 3 +-
tests/cli/commands/test_kubernetes_command.py | 3 +-
tests/cli/commands/test_plugins_command.py | 16 +-
.../cli/commands/test_rotate_fernet_key_command.py | 5 +-
tests/cli/commands/test_scheduler_command.py | 3 +-
tests/cli/commands/test_task_command.py | 7 +-
tests/cli/commands/test_variable_command.py | 3 +-
tests/cli/commands/test_webserver_command.py | 4 +-
tests/cli/conftest.py | 3 +-
tests/cli/test_cli_parser.py | 3 +-
tests/conftest.py | 4 +-
tests/core/test_configuration.py | 6 +-
tests/core/test_core.py | 3 +-
tests/core/test_example_dags_system.py | 5 +-
tests/core/test_impersonation_tests.py | 27 +-
tests/core/test_logging_config.py | 3 +-
tests/core/test_otel_tracer.py | 3 +-
tests/core/test_sentry.py | 3 +-
tests/core/test_settings.py | 3 +-
tests/core/test_sqlalchemy_config.py | 3 +-
tests/core/test_stats.py | 3 +-
tests/dag_processing/test_job_runner.py | 12 +-
tests/dag_processing/test_processor.py | 11 +-
.../test_external_task_sensor_check_existense.py | 1 +
tests/dags/test_miscellaneous.py | 3 +-
tests/dags/test_sensor.py | 3 +-
tests/decorators/test_bash.py | 3 +-
tests/decorators/test_mapped.py | 1 +
tests/decorators/test_python.py | 4 +-
tests/executors/test_executor_loader.py | 3 +-
.../cli/commands/test_celery_command.py | 3 +-
.../integration/executors/test_celery_executor.py | 3 +-
tests/integration/security/test_kerberos.py | 3 +-
tests/jobs/test_backfill_job.py | 2094 --------------------
tests/jobs/test_base_job.py | 4 +-
tests/jobs/test_local_task_job.py | 11 +-
tests/jobs/test_scheduler_job.py | 172 +-
tests/jobs/test_triggerer_job.py | 4 +-
tests/jobs/test_triggerer_job_logging.py | 3 +-
tests/lineage/test_hook.py | 3 +-
tests/lineage/test_lineage.py | 3 +-
tests/listeners/class_listener.py | 3 +-
tests/listeners/test_asset_listener.py | 1 +
tests/listeners/test_dag_import_error_listener.py | 12 +-
tests/listeners/test_listeners.py | 1 +
tests/models/test_backfill.py | 3 +-
tests/models/test_base.py | 3 +-
tests/models/test_baseoperator.py | 4 +-
tests/models/test_cleartasks.py | 6 +-
tests/models/test_dag.py | 44 +-
tests/models/test_dagbag.py | 8 +-
tests/models/test_dagcode.py | 3 +-
tests/models/test_dagrun.py | 10 +-
tests/models/test_dagwarning.py | 3 +-
tests/models/test_mappedoperator.py | 8 +-
tests/models/test_param.py | 3 +-
tests/models/test_pool.py | 3 +-
tests/models/test_renderedtifields.py | 55 +-
tests/models/test_serialized_dag.py | 5 +-
tests/models/test_skipmixin.py | 3 +-
tests/models/test_taskinstance.py | 12 +-
tests/models/test_timestamp.py | 3 +-
tests/models/test_trigger.py | 3 +-
tests/models/test_variable.py | 5 +-
tests/models/test_xcom.py | 5 +-
tests/models/test_xcom_arg.py | 3 +-
tests/notifications/test_basenotifier.py | 1 -
tests/operators/test_branch_operator.py | 3 +-
tests/operators/test_email.py | 3 +-
tests/operators/test_latest_only_operator.py | 5 +-
tests/operators/test_python.py | 7 +-
tests/plugins/test_plugin.py | 8 +-
tests/plugins/test_plugins_manager.py | 5 +-
tests/secrets/test_cache.py | 3 +-
tests/security/test_kerberos.py | 3 +-
tests/sensors/test_base.py | 3 +-
tests/sensors/test_external_task_sensor.py | 8 +-
tests/sensors/test_python.py | 1 +
tests/serialization/test_dag_serialization.py | 37 +-
tests/serialization/test_pydantic_models.py | 4 +-
tests/serialization/test_serde.py | 3 +-
tests/serialization/test_serialized_objects.py | 6 +-
.../core/example_external_task_child_deferrable.py | 2 +-
.../example_external_task_parent_deferrable.py | 4 +-
tests/system/example_empty.py | 4 +-
.../task/task_runner/test_standard_task_runner.py | 6 +-
.../ti_deps/deps/test_pool_slots_available_dep.py | 3 +-
tests/ti_deps/deps/test_prev_dagrun_dep.py | 5 +-
tests/ti_deps/deps/test_ready_to_reschedule_dep.py | 3 +-
tests/ti_deps/deps/test_trigger_rule_dep.py | 1 +
tests/utils/log/test_log_reader.py | 5 +-
tests/utils/log/test_secrets_masker.py | 3 +-
tests/utils/log/test_task_context_logger.py | 5 +-
tests/utils/test_dag_cycle.py | 1 +
tests/utils/test_db.py | 3 +-
tests/utils/test_db_cleanup.py | 4 +-
tests/utils/test_db_manager.py | 3 +-
tests/utils/test_dot_renderer.py | 5 +-
tests/utils/test_email.py | 3 +-
tests/utils/test_file.py | 4 +-
tests/utils/test_helpers.py | 5 +-
tests/utils/test_log_handlers.py | 5 +-
tests/utils/test_net.py | 3 +-
tests/utils/test_serve_logs.py | 3 +-
tests/utils/test_sqlalchemy.py | 3 +-
tests/utils/test_state.py | 4 +-
tests/utils/test_task_group.py | 6 +-
.../test_task_handler_with_custom_formatter.py | 7 +-
tests/utils/test_types.py | 4 +-
tests/www/test_app.py | 5 +-
tests/www/test_utils.py | 3 +-
tests/www/views/conftest.py | 9 +-
tests/www/views/test_anonymous_as_admin_role.py | 3 +-
tests/www/views/test_session.py | 5 +-
tests/www/views/test_views.py | 15 +-
tests/www/views/test_views_acl.py | 10 +-
tests/www/views/test_views_base.py | 7 +-
tests/www/views/test_views_cluster_activity.py | 3 +-
tests/www/views/test_views_configuration.py | 5 +-
tests/www/views/test_views_connection.py | 3 +-
tests/www/views/test_views_custom_user_views.py | 4 +-
tests/www/views/test_views_dagrun.py | 8 +-
tests/www/views/test_views_dataset.py | 121 +-
tests/www/views/test_views_decorators.py | 7 +-
tests/www/views/test_views_extra_links.py | 15 +-
tests/www/views/test_views_grid.py | 7 +-
tests/www/views/test_views_home.py | 8 +-
tests/www/views/test_views_log.py | 11 +-
tests/www/views/test_views_mount.py | 3 +-
tests/www/views/test_views_paused.py | 3 +-
tests/www/views/test_views_pool.py | 3 +-
tests/www/views/test_views_rate_limit.py | 7 +-
tests/www/views/test_views_rendered.py | 7 +-
tests/www/views/test_views_robots.py | 2 +-
tests/www/views/test_views_task_norun.py | 2 +-
tests/www/views/test_views_tasks.py | 10 +-
tests/www/views/test_views_trigger_dag.py | 4 +-
tests/www/views/test_views_variable.py | 4 +-
{dev/tests_common => tests_common}/__init__.py | 0
.../_internals}/__init__.py | 0
.../_internals/capture_warnings.py | 4 +-
.../_internals/forbidden_warnings.py | 2 +-
{dev/tests_common => tests_common}/pyproject.toml | 2 +-
.../tests_common => tests_common}/pytest_plugin.py | 51 +-
.../test_utils/README.md | 0
.../test_utils/__init__.py | 2 +-
.../test_utils/api_connexion_utils.py | 12 +-
.../test_utils/asserts.py | 9 +-
.../test_utils/azure_system_helpers.py | 16 +-
.../test_utils/compat.py | 2 +-
.../test_utils/config.py | 0
.../tests_common => tests_common}/test_utils/db.py | 15 +-
.../test_utils/decorators.py | 0
.../test_utils/fake_datetime.py | 4 +-
.../test_utils/gcp_system_helpers.py | 17 +-
.../test_utils/get_all_tests.py | 8 +-
.../test_utils/hdfs_utils.py | 2 +
.../test_utils/logging_command_executor.py | 6 +-
.../test_utils/mapping.py | 0
.../test_utils/mock_executor.py | 11 +-
.../test_utils/mock_operators.py | 49 +-
.../test_utils/mock_plugins.py | 0
.../test_utils/mock_security_manager.py | 4 +-
.../test_utils/operators}/__init__.py | 0
.../operators/postgres_local_executor.cfg | 0
.../test_utils/perf}/__init__.py | 0
.../test_utils/perf/perf_kit/__init__.py | 5 +-
.../test_utils/perf/perf_kit/memory.py | 4 +-
.../test_utils/perf/perf_kit/python.py | 15 +-
.../test_utils/perf/perf_kit/repeat_and_time.py | 12 +-
.../test_utils/perf/perf_kit/sqlalchemy.py | 15 +-
.../test_utils/permissions.py | 2 +
.../test_utils/providers.py | 4 +-
.../test_utils/remote_user_api_auth_backend.py | 2 +-
.../test_utils/reset_warning_registry.py | 4 +-
.../test_utils/salesforce_system_helpers.py | 0
.../test_utils/sftp_system_helpers.py | 2 +-
.../test_utils/system_tests.py | 0
.../test_utils/system_tests_class.py | 13 +-
.../test_utils/terraform.py | 4 +-
.../test_utils/timetables.py | 2 +
.../test_utils/watcher.py | 8 +-
.../test_utils/www.py | 0
903 files changed, 6655 insertions(+), 7358 deletions(-)
rename airflow/api_fastapi/{views => routes}/__init__.py (100%)
rename airflow/api_fastapi/{views => routes}/public/__init__.py (69%)
rename airflow/api_fastapi/{views => routes}/public/connections.py (64%)
rename airflow/api_fastapi/{views => routes}/public/dag_run.py (97%)
rename airflow/api_fastapi/{views => routes}/public/dags.py (88%)
copy airflow/{api_connexion/endpoints/health_endpoint.py =>
api_fastapi/routes/public/monitor.py} (73%)
rename airflow/api_fastapi/{views => routes}/public/variables.py (55%)
rename airflow/api_fastapi/{views => routes}/router.py (100%)
rename airflow/api_fastapi/{views => routes}/ui/__init__.py (83%)
rename airflow/api_fastapi/{views => routes}/ui/assets.py (95%)
rename airflow/api_fastapi/{views => routes}/ui/dashboard.py (98%)
rename airflow/{api_connexion/schemas/sla_miss_schema.py =>
api_fastapi/serializers/monitor.py} (52%)
delete mode 100644 airflow/jobs/backfill_job_runner.py
create mode 100644 airflow/migrations/versions/0037_3_0_0_add_asset_active.py
delete mode 100644 airflow/models/slamiss.py
copy airflow/ui/src/{layouts/Nav/NavButton.tsx => constants/searchParams.ts}
(56%)
create mode 100644 newsfragments/42776.significant.rst
copy {dev/tests_common/test_utils =>
providers/src/airflow/providers}/__init__.py (69%)
create mode 100644 scripts/ci/pre_commit/draft7_schema.json
rename {dev/tests_common/_internals => tests/api_fastapi/routes}/__init__.py
(100%)
rename {dev/tests_common/test_utils/operators =>
tests/api_fastapi/routes/public}/__init__.py (100%)
rename tests/api_fastapi/{views => routes}/public/test_connections.py (58%)
rename tests/api_fastapi/{views => routes}/public/test_dag_run.py (72%)
create mode 100644 tests/api_fastapi/routes/public/test_dags.py
create mode 100644 tests/api_fastapi/routes/public/test_monitor.py
create mode 100644 tests/api_fastapi/routes/public/test_variables.py
rename {dev/tests_common/test_utils/perf =>
tests/api_fastapi/routes/ui}/__init__.py (100%)
rename tests/api_fastapi/{views => routes}/ui/test_assets.py (96%)
rename tests/api_fastapi/{views => routes}/ui/test_dashboard.py (98%)
delete mode 100644 tests/api_fastapi/views/__init__.py
delete mode 100644 tests/api_fastapi/views/public/__init__.py
delete mode 100644 tests/api_fastapi/views/public/test_dags.py
delete mode 100644 tests/api_fastapi/views/public/test_variables.py
delete mode 100644 tests/api_fastapi/views/ui/__init__.py
rename tests/assets/{tests_asset.py => test_asset.py} (95%)
delete mode 100644 tests/jobs/test_backfill_job.py
rename {dev/tests_common => tests_common}/__init__.py (100%)
copy {airflow/api_connexion => tests_common/_internals}/__init__.py (100%)
rename {dev/tests_common => tests_common}/_internals/capture_warnings.py (99%)
rename {dev/tests_common => tests_common}/_internals/forbidden_warnings.py
(99%)
rename {dev/tests_common => tests_common}/pyproject.toml (98%)
rename {dev/tests_common => tests_common}/pytest_plugin.py (97%)
rename {dev/tests_common => tests_common}/test_utils/README.md (100%)
rename {dev/tests_common => tests_common}/test_utils/__init__.py (94%)
rename {dev/tests_common => tests_common}/test_utils/api_connexion_utils.py
(91%)
rename {dev/tests_common => tests_common}/test_utils/asserts.py (95%)
rename {dev/tests_common => tests_common}/test_utils/azure_system_helpers.py
(93%)
rename {dev/tests_common => tests_common}/test_utils/compat.py (99%)
rename {dev/tests_common => tests_common}/test_utils/config.py (100%)
rename {dev/tests_common => tests_common}/test_utils/db.py (95%)
rename {dev/tests_common => tests_common}/test_utils/decorators.py (100%)
rename {dev/tests_common => tests_common}/test_utils/fake_datetime.py (92%)
rename {dev/tests_common => tests_common}/test_utils/gcp_system_helpers.py
(95%)
rename {dev/tests_common => tests_common}/test_utils/get_all_tests.py (92%)
rename {dev/tests_common => tests_common}/test_utils/hdfs_utils.py (95%)
rename {dev/tests_common =>
tests_common}/test_utils/logging_command_executor.py (97%)
rename {dev/tests_common => tests_common}/test_utils/mapping.py (100%)
rename {dev/tests_common => tests_common}/test_utils/mock_executor.py (94%)
rename {dev/tests_common => tests_common}/test_utils/mock_operators.py (84%)
rename {dev/tests_common => tests_common}/test_utils/mock_plugins.py (100%)
rename {dev/tests_common => tests_common}/test_utils/mock_security_manager.py
(89%)
copy {airflow/api_connexion => tests_common/test_utils/operators}/__init__.py
(100%)
rename {dev/tests_common =>
tests_common}/test_utils/operators/postgres_local_executor.cfg (100%)
copy {airflow/api_connexion => tests_common/test_utils/perf}/__init__.py (100%)
rename {dev/tests_common => tests_common}/test_utils/perf/perf_kit/__init__.py
(99%)
rename {dev/tests_common => tests_common}/test_utils/perf/perf_kit/memory.py
(94%)
rename {dev/tests_common => tests_common}/test_utils/perf/perf_kit/python.py
(90%)
rename {dev/tests_common =>
tests_common}/test_utils/perf/perf_kit/repeat_and_time.py (92%)
rename {dev/tests_common =>
tests_common}/test_utils/perf/perf_kit/sqlalchemy.py (97%)
rename {dev/tests_common => tests_common}/test_utils/permissions.py (93%)
rename {dev/tests_common => tests_common}/test_utils/providers.py (94%)
rename {dev/tests_common =>
tests_common}/test_utils/remote_user_api_auth_backend.py (97%)
rename {dev/tests_common => tests_common}/test_utils/reset_warning_registry.py
(96%)
rename {dev/tests_common =>
tests_common}/test_utils/salesforce_system_helpers.py (100%)
rename {dev/tests_common => tests_common}/test_utils/sftp_system_helpers.py
(99%)
rename {dev/tests_common => tests_common}/test_utils/system_tests.py (100%)
rename {dev/tests_common => tests_common}/test_utils/system_tests_class.py
(95%)
rename {dev/tests_common => tests_common}/test_utils/terraform.py (93%)
rename {dev/tests_common => tests_common}/test_utils/timetables.py (97%)
rename {dev/tests_common => tests_common}/test_utils/watcher.py (86%)
rename {dev/tests_common => tests_common}/test_utils/www.py (100%)
