This is an automated email from the ASF dual-hosted git repository.
dimberman pushed a change to branch handle-stuck-in-queue
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 1f8b6425cc4 Allow for retry when tasks are stuck in queued
discard 765bbd6dd07 Allow for retry when tasks are stuck in queued
add eda6a8fcf00 Mark all tasks as skipped when failing a dag_run manually
including tasks with None state (#43482)
add b344cc1474d Fix validating `parent_model` parameter in
`UploadModelOperator` (#43473)
add 74de7aa28ff Add docs for backfill (#43477)
add 85b095e8885 Update version of Google ADS (#43474)
add c3254a363ff Fix pytest plugin (#43280)
add e0fda354a71 Drop `task_fail` table (#43490)
add 4fc16f154d2 Rename --dag to --dag-id in backfill CLI command (#43478)
add 8911f709fb0 Revert "Fix pytest plugin (#43280)" (#43504)
add 0e52046de0a Make the warnings capture plugin know about the provider
workspace folder (#43505)
add 818165d60cd Add Briana to triage team (#43507)
add 6e547da7e05 Add type hints to DAG test utils (#43502)
add c0be4023a97 Run only ui tests for PR with new UI only changes. (#43503)
add 6641d78703b Add latest dag run info to dags list and dag details
(#43489)
add dc59212b852 update issue triage process to improve Dosu bot quality
(#43511)
add 64d8f4f6dbb Migrate the public endpoint Get DAG Source to FastAPI
(#43084)
add d8c7d28411b Start porting DAG definition code to the Task SDK (#43076)
add b94b1a19837 Skip integration tests for new UI only PR. Run docs build
only when needed. (#43512)
add da265d92cad Task SDK: Fix typing for `autoregistered_dags` (#43522)
add a2cbaecf625 Keycloak Breeze integration (#43240)
add 51f9e835bf4 Move `uncompress_file` function from `airflow.utils` to
Hive provider (#43526)
add 10c6464d86a Remove unused functions from `airflow.utils.dates` (#43527)
add 31ee7cbe5d6 Remove `task_fail` from `airflow db cleanup` (#43528)
add 15ae7ce9724 Remove deprecated functions from
`airflow/configuration.py` (#43530)
add a923a83338a Bump `uv` to `0.4.29` (#43531)
add d3e18121521 Remove `TaskInstanceKey.reduced` (#43532)
add c42b1d3acd5 Update providers metadata 2024-10-31 (#43539)
add 568c0352f11 Fix main: ignore spell check Uncompress (#43538)
add db4afd793a6 DOC fix documentation error in
`apache-airflow-providers-fab/access-control.rst` (#43495)
add 8308dad6ee0 Include doc-only commit in provider commit history (#43543)
add b3e902dd249 Fix testing tests command to allow passing test as extra
arg (#43529)
add b985b2579bd AIP-84 Get Task Instance (#43485)
add 5886016243d Prepare docs for Oct 2nd wave of providers RC2 (#43540)
add 06088a3abcb Standard provider python operator (#42081)
add 22d2aebe6a5 Add logging to the migration commands (#43516)
add 4fcbeb58016 Remove redundant functions in `airflow.utils.dates`
(#43533)
add 80f5c6f4cff [TRACES][OTEL] allow otel_debugging_on to emit traces to
otel host (#43500)
add 6ee5a610e42 Remove pluggable TaskRunners concept (#43551)
add be4d8943d3d Dedent help string in breeze configuration (#43555)
add 0c9e159dc4a Fix dependencies of standard provider (#43553)
add dd7f669e6c2 Mark standard provider as ready for release (#43550)
add 0531e78398c Bring back documentation publishing to Canary builds
(#43509)
add adb4d2adc2a AIP-84 Get Mapped Task Instance (#43548)
add 50dcb27c036 Fix reproducibility of provider builds (#43557)
add 533ad403ef3 Unify reattach_states parameter logic across BigQuery
operators (#43259)
add c0ae6b80ced add min versions to types- pytz, redis, requests,
setuptools (#43405)
add 013a6310de0 Add directions to build a provider package locally.
(#43524)
add 54d62762289 Add pavan kumar gopidesu into committer list (#43554)
add aaf801d138e Add missing extension in the Newsfragment file (#43564)
add c18b2a8499f Drop redundant code since Airflow 3 is Python `>=3.9`
(#43563)
add fe41e156084 Conditionally add OTEL events when processing executor
events (#43558)
add 3d36dc626bf Remove `astunparse` dependency (#43560)
add 4da7936a9b1 Remove `backports.zoneinfo` dependency (#43559)
add aa8ac32ed29 Remove deprecated Python Version identifiers (#43562)
add 7506beda5f5 Remove `importlib_resources` dependency (#43561)
add bc6e325c403 Fix TrySelector for Mapped Tasks in Logs and Details Grid
Panel (#43565)
add 199d9acd144 Add tirkarthi to committers list. (#43573)
add 2c67c9fb29f Add missing test for kubernetes compat converters (#43248)
add fe0dd9b8b35 Switch PythonVirtualenvOperator to venv from virtualenv
package (#43568)
add c4a0461dd36 Allow Utkarsh to run dockerhub image release job (#43588)
add 8eb60b114b6 Handle the scenario where a task is queued for longer than
`task_queued_timeout`.
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (1f8b6425cc4)
\
N -- N -- N refs/heads/handle-stuck-in-queue (8eb60b114b6)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.asf.yaml | 2 +-
.github/boring-cyborg.yml | 2 +-
.github/workflows/ci.yml | 1 +
.github/workflows/prod-image-build.yml | 5 +
.github/workflows/release_dockerhub_image.yml | 1 +
.github/workflows/static-checks-mypy-docs.yml | 21 +-
.pre-commit-config.yaml | 6 +-
Dockerfile | 37 +-
Dockerfile.ci | 4 +-
INSTALL | 3 +-
ISSUE_TRIAGE_PROCESS.rst | 7 +-
airflow/__init__.py | 9 -
airflow/api/common/mark_tasks.py | 11 +-
.../api_connexion/endpoints/dag_source_endpoint.py | 2 +
.../endpoints/task_instance_endpoint.py | 3 +
airflow/api_connexion/schemas/dag_schema.py | 1 -
airflow/api_fastapi/app.py | 18 +-
airflow/api_fastapi/core_api/app.py | 20 +
.../api_fastapi/core_api/openapi/v1-generated.yaml | 1338 +-
.../api_fastapi/core_api/routes/public/__init__.py | 13 +-
.../core_api/routes/public/dag_sources.py | 71 +
.../core_api/routes/public/task_instances.py | 85 +
.../core_api/serializers/dag_sources.py | 8 +-
airflow/api_fastapi/core_api/serializers/dags.py | 1 -
.../api_fastapi/core_api/serializers/job.py | 21 +-
.../core_api/serializers/task_instances.py | 71 +
.../api_fastapi/core_api/serializers/trigger.py | 17 +-
airflow/cli/cli_config.py | 2 +-
airflow/cli/commands/backfill_command.py | 6 +-
airflow/cli/commands/dag_command.py | 1 -
airflow/cli/commands/task_command.py | 3 +-
airflow/config_templates/config.yml | 9 -
airflow/configuration.py | 141 -
airflow/dag_processing/collection.py | 5 +-
airflow/decorators/base.py | 44 +-
airflow/decorators/bash.py | 4 +-
airflow/decorators/branch_external_python.py | 2 +-
airflow/decorators/branch_python.py | 2 +-
airflow/decorators/branch_virtualenv.py | 2 +-
airflow/decorators/external_python.py | 2 +-
airflow/decorators/python.py | 2 +-
airflow/decorators/python_virtualenv.py | 2 +-
airflow/decorators/sensor.py | 6 +-
airflow/decorators/short_circuit.py | 2 +-
airflow/decorators/task_group.py | 2 +-
.../example_asset_alias_with_no_taskflow.py | 2 +-
airflow/example_dags/example_branch_operator.py | 217 +-
.../example_branch_operator_decorator.py | 155 +-
.../example_python_context_decorator.py | 6 +-
.../example_python_context_operator.py | 12 +-
airflow/example_dags/example_python_decorator.py | 108 +-
airflow/example_dags/example_python_operator.py | 118 +-
airflow/example_dags/example_sensors.py | 2 +-
.../example_dags/example_short_circuit_operator.py | 2 +-
airflow/example_dags/tutorial_dag.py | 2 +-
.../tutorial_taskflow_api_virtualenv.py | 111 +-
.../example_dags/tutorial_taskflow_templates.py | 2 +-
airflow/exceptions.py | 6 +-
airflow/executors/base_executor.py | 9 -
airflow/jobs/local_task_job_runner.py | 4 +-
airflow/jobs/scheduler_job_runner.py | 53 +-
airflow/migrations/env.py | 4 +-
.../versions/0044_3_0_0__drop_task_fail_table.py | 75 +
airflow/models/__init__.py | 3 -
airflow/models/abstractoperator.py | 355 +-
airflow/models/backfill.py | 2 +-
airflow/models/baseoperator.py | 1190 +-
airflow/models/dag.py | 1053 +-
airflow/models/dagbag.py | 13 +-
airflow/models/mappedoperator.py | 7 +-
airflow/models/param.py | 2 +-
airflow/models/skipmixin.py | 2 +-
airflow/models/taskfail.py | 85 -
airflow/models/taskinstance.py | 17 +-
airflow/models/taskinstancekey.py | 8 -
airflow/models/taskmixin.py | 270 +-
airflow/models/xcom_arg.py | 21 +-
airflow/providers_manager.py | 6 +-
airflow/sensors/external_task.py | 4 +-
airflow/serialization/schema.json | 20 +-
airflow/serialization/serialized_objects.py | 61 +-
airflow/task/priority_strategy.py | 4 +-
.../task/{task_runner => }/standard_task_runner.py | 136 +-
airflow/task/task_runner/__init__.py | 66 -
airflow/task/task_runner/base_task_runner.py | 202 -
airflow/task/task_runner/cgroup_task_runner.py | 246 -
airflow/template/templater.py | 17 +-
airflow/traces/otel_tracer.py | 25 +-
airflow/typing_compat.py | 5 +-
airflow/ui/openapi-gen/queries/common.ts | 326 +-
airflow/ui/openapi-gen/queries/prefetch.ts | 345 +-
airflow/ui/openapi-gen/queries/queries.ts | 685 +-
airflow/ui/openapi-gen/queries/suspense.ts | 423 +-
airflow/ui/openapi-gen/requests/schemas.gen.ts | 577 +-
airflow/ui/openapi-gen/requests/services.gen.ts | 670 +-
airflow/ui/openapi-gen/requests/types.gen.ts | 670 +-
airflow/ui/src/layouts/Nav/Nav.tsx | 7 -
airflow/ui/src/pages/DagsList/Dag/Dag.tsx | 23 +-
airflow/ui/src/pages/DagsList/Dag/Header.tsx | 7 +-
airflow/ui/src/pages/DagsList/DagCard.test.tsx | 10 +-
airflow/ui/src/pages/DagsList/DagCard.tsx | 34 +-
airflow/ui/src/pages/DagsList/DagTags.tsx | 5 +-
airflow/ui/src/pages/DagsList/DagsFilters.tsx | 2 +-
airflow/ui/src/pages/DagsList/DagsList.tsx | 99 +-
airflow/ui/src/pages/DagsList/LatestRun.tsx | 41 +
airflow/ui/src/pages/DagsList/RecentRuns.tsx | 83 +
airflow/ui/src/pages/DagsList/Schedule.tsx | 36 +
airflow/ui/src/queries/useDags.tsx | 81 +
airflow/ui/src/utils/stateColor.ts | 36 +
airflow/utils/compression.py | 40 -
airflow/utils/dates.py | 136 -
airflow/utils/db.py | 35 +-
airflow/utils/db_cleanup.py | 1 -
airflow/utils/db_manager.py | 4 +
airflow/utils/decorators.py | 9 +-
airflow/utils/edgemodifier.py | 154 +-
airflow/utils/hashlib_wrapper.py | 5 +-
airflow/utils/log/logging_mixin.py | 1 +
airflow/utils/state.py | 1 -
airflow/utils/task_group.py | 624 +-
airflow/utils/types.py | 32 +-
airflow/www/static/js/api/useTIHistory.ts | 4 +-
contributing-docs/11_provider_packages.rst | 27 +
.../12_airflow_dependencies_and_extras.rst | 3 +-
contributing-docs/testing/integration_tests.rst | 68 +-
dev/PROVIDER_PACKAGE_DETAILS.md | 4 +
dev/breeze/doc/ci/02_images.md | 2 +-
dev/breeze/doc/images/output-commands.svg | 4 +-
dev/breeze/doc/images/output_build-docs.txt | 2 +-
dev/breeze/doc/images/output_prod-image.txt | 2 +-
dev/breeze/doc/images/output_prod-image_build.txt | 2 +-
...management_generate-issue-content-providers.svg | 2 +-
...management_generate-issue-content-providers.txt | 2 +-
...e-management_prepare-provider-documentation.svg | 2 +-
...e-management_prepare-provider-documentation.txt | 2 +-
...elease-management_prepare-provider-packages.svg | 2 +-
...elease-management_prepare-provider-packages.txt | 2 +-
dev/breeze/doc/images/output_setup.txt | 2 +-
.../doc/images/output_setup_autocomplete.txt | 2 +-
dev/breeze/doc/images/output_setup_config.txt | 2 +-
dev/breeze/doc/images/output_shell.svg | 4 +-
dev/breeze/doc/images/output_shell.txt | 2 +-
dev/breeze/doc/images/output_start-airflow.svg | 4 +-
dev/breeze/doc/images/output_start-airflow.txt | 2 +-
.../images/output_testing_integration-tests.svg | 4 +-
.../images/output_testing_integration-tests.txt | 2 +-
dev/breeze/doc/images/output_testing_tests.svg | 4 +-
dev/breeze/doc/images/output_testing_tests.txt | 2 +-
.../commands/release_management_commands.py | 2 +-
.../src/airflow_breeze/commands/setup_commands.py | 26 +-
.../airflow_breeze/commands/testing_commands.py | 12 +-
dev/breeze/src/airflow_breeze/global_constants.py | 21 +-
.../src/airflow_breeze/params/shell_params.py | 13 +
.../prepare_providers/provider_documentation.py | 1 +
.../templates/pyproject_TEMPLATE.toml.jinja2 | 2 +-
dev/breeze/src/airflow_breeze/utils/run_tests.py | 16 +-
.../src/airflow_breeze/utils/selective_checks.py | 48 +-
dev/breeze/tests/test_packages.py | 4 +-
.../tests/test_pytest_args_for_test_types.py | 10 +-
dev/breeze/tests/test_selective_checks.py | 74 +-
dev/mypy/plugin/outputs.py | 1 +
dev/perf/dags/sql_perf_dag.py | 2 +-
dev/perf/scheduler_dag_execution_timing.py | 2 -
docker_tests/test_prod_image.py | 1 -
docs/apache-airflow-providers-amazon/commits.rst | 7 +-
.../auth-manager/access-control.rst | 2 +-
docs/apache-airflow-providers-fab/commits.rst | 5 +-
.../guides/user.rst | 4 +-
.../dynamic-task-mapping.rst | 2 +-
docs/apache-airflow/best-practices.rst | 16 +-
docs/apache-airflow/core-concepts/backfill.rst | 69 +
docs/apache-airflow/core-concepts/dag-run.rst | 15 +-
docs/apache-airflow/core-concepts/dags.rst | 2 +-
docs/apache-airflow/core-concepts/debug.rst | 3 +-
.../core-concepts/executor/index.rst | 6 +-
docs/apache-airflow/core-concepts/index.rst | 1 +
docs/apache-airflow/core-concepts/operators.rst | 4 +-
docs/apache-airflow/extra-packages-ref.rst | 2 -
docs/apache-airflow/faq.rst | 5 +-
docs/apache-airflow/howto/operator/python.rst | 24 +-
docs/apache-airflow/img/airflow_erd.sha256 | 2 +-
docs/apache-airflow/img/airflow_erd.svg | 2571 ++--
docs/apache-airflow/migrations-ref.rst | 4 +-
docs/apache-airflow/operators-and-hooks-ref.rst | 4 +-
docs/apache-airflow/project.rst | 2 +
docs/apache-airflow/start.rst | 2 +-
docs/apache-airflow/tutorial/fundamentals.rst | 36 +-
docs/apache-airflow/tutorial/taskflow.rst | 2 +-
docs/docker-stack/build-arg-ref.rst | 1 -
docs/docker-stack/changelog.rst | 6 +
docs/exts/templates/openlineage.rst.jinja2 | 2 +-
docs/spelling_wordlist.txt | 1 +
generated/provider_dependencies.json | 21 +-
generated/provider_metadata.json | 84 +
hatch_build.py | 24 +-
newsfragments/42953.significant | 3 -
newsfragments/42953.significant.rst | 4 +
newsfragments/43490.significant.rst | 4 +
newsfragments/43530.significant.rst | 21 +
newsfragments/43533.significant.rst | 8 +
newsfragments/43562.significant.rst | 24 +
newsfragments/43568.significant.rst | 1 +
newsfragments/aip-72.significant.rst | 15 +
.../src/airflow/providers/amazon/CHANGELOG.rst | 10 +-
.../providers/amazon/aws/operators/appflow.py | 2 +-
.../providers/amazon/aws/operators/comprehend.py | 6 +-
.../airflow/providers/amazon/aws/operators/dms.py | 6 +-
.../amazon/aws/operators/kinesis_analytics.py | 6 +-
.../providers/amazon/aws/operators/sagemaker.py | 4 +-
.../airflow/providers/apache/beam/hooks/beam.py | 2 +-
.../providers/apache/drill/operators/drill.py | 4 +-
.../providers/apache/hive/transfers/s3_to_hive.py | 19 +-
.../providers/apache/spark/decorators/pyspark.py | 2 +-
.../providers/celery/executors/celery_executor.py | 1 -
.../celery/executors/celery_executor_utils.py | 5 +-
.../providers/cncf/kubernetes/operators/pod.py | 4 +-
.../cncf/kubernetes/operators/spark_kubernetes.py | 3 +-
.../providers/common/compat/standard}/__init__.py | 0
.../providers/common/compat/standard/operators.py | 54 +
.../providers/common/compat/standard/utils.py | 22 +-
.../airflow/providers/common/sql/operators/sql.py | 18 +-
.../airflow/providers/common/sql/operators/sql.pyi | 18 +-
.../databricks/operators/databricks_sql.py | 4 +-
.../airflow/providers/docker/decorators/docker.py | 4 +-
.../edge/example_dags/integration_test.py | 2 +-
.../airflow/providers/exasol/operators/exasol.py | 4 +-
providers/src/airflow/providers/fab/CHANGELOG.rst | 7 +-
.../fab/auth_manager/security_manager/override.py | 9 +-
.../src/airflow/providers/fab/migrations/env.py | 3 +-
.../src/airflow/providers/google/ads/hooks/ads.py | 12 +-
.../google/cloud/hooks/vertex_ai/model_service.py | 4 +-
.../cloud/operators/vertex_ai/model_service.py | 7 +-
.../google/cloud/transfers/bigquery_to_gcs.py | 16 +-
.../google/cloud/transfers/gcs_to_bigquery.py | 16 +-
.../google/cloud/utils/mlengine_operator_utils.py | 2 +-
.../src/airflow/providers/google/provider.yaml | 2 +-
.../src/airflow/providers/jdbc/operators/jdbc.py | 4 +-
.../providers/microsoft/mssql/operators/mssql.py | 4 +-
.../src/airflow/providers/mysql/operators/mysql.py | 4 +-
.../airflow/providers/openlineage/provider.yaml | 2 +-
.../airflow/providers/oracle/operators/oracle.py | 4 +-
.../providers/postgres/operators/postgres.py | 7 +-
.../providers/snowflake/operators/snowflake.py | 4 +-
.../providers/snowflake/operators/snowpark.py | 2 +-
.../airflow/providers/sqlite/operators/sqlite.py | 4 +-
.../src/airflow/providers/standard/CHANGELOG.rst | 2 +
.../providers/standard}/operators/python.py | 100 +-
.../src/airflow/providers/standard/provider.yaml | 9 +-
.../airflow/providers/standard}/sensors/python.py | 0
.../airflow/providers/standard/utils}/__init__.py | 0
.../providers/standard}/utils/python_virtualenv.py | 24 +-
.../utils/python_virtualenv_script.jinja2 | 4 +-
.../providers/teradata/operators/teradata.py | 4 +-
.../src/airflow/providers/trino/operators/trino.py | 4 +-
.../airflow/providers/vertica/operators/vertica.py | 4 +-
.../alibaba/cloud/log/test_oss_task_handler.py | 6 +-
providers/tests/amazon/aws/operators/test_batch.py | 2 +-
providers/tests/amazon/aws/sensors/test_ecs.py | 2 +-
.../tests/apache/hive/transfers/test_s3_to_hive.py | 20 +-
.../tests/cncf/kubernetes/backcompat}/__init__.py | 0
.../backcompat/test_backwards_compat_converters.py | 270 +
.../kubernetes/log_handlers/test_log_handlers.py | 3 +-
.../api_endpoints/test_xcom_endpoint.py | 4 +-
.../cloud/hooks/vertex_ai/test_model_service.py | 6 +-
.../tests/google/cloud/operators/test_bigquery.py | 4 +-
.../google/cloud/operators/test_cloud_build.py | 4 +-
.../tests/google/cloud/operators/test_compute.py | 10 +-
.../tests/google/cloud/operators/test_dataflow.py | 2 +-
.../tests/google/cloud/operators/test_dataproc.py | 6 +-
.../cloud/operators/test_kubernetes_engine.py | 16 +-
.../google/cloud/operators/test_speech_to_text.py | 10 +-
.../tests/google/cloud/sensors/test_dataproc.py | 2 +-
.../google/cloud/transfers/test_gcs_to_bigquery.py | 9 +-
.../cloud/utils/test_mlengine_operator_utils.py | 3 +-
.../tests/openlineage/extractors/test_base.py | 4 +-
.../tests/openlineage/extractors/test_manager.py | 3 +-
.../tests/openlineage/extractors/test_python.py | 3 +-
.../tests/openlineage/plugins/test_execution.py | 8 +-
.../tests/openlineage/plugins/test_listener.py | 3 +-
providers/tests/openlineage/plugins/test_utils.py | 4 +-
providers/tests/openlineage/utils/test_utils.py | 13 +-
providers/tests/salesforce/operators/test_bulk.py | 4 +-
.../tests/standard}/operators/test_python.py | 54 +-
providers/tests/standard/operators/test_weekday.py | 2 +-
.../tests/standard}/sensors/test_python.py | 4 +-
providers/tests/standard/sensors/test_weekday.py | 1 -
.../tests/standard/utils}/__init__.py | 0
.../standard}/utils/test_python_virtualenv.py | 23 +-
.../tests/system/amazon/aws/example_mongo_to_s3.py | 2 +-
providers/tests/system/amazon/aws/example_s3.py | 2 +-
.../tests/system/amazon/aws/example_sagemaker.py | 2 +-
.../apache/kafka/example_dag_event_listener.py | 6 +-
.../system/apache/kafka/example_dag_hello_kafka.py | 8 +-
.../common/sql/example_sql_column_table_check.py | 4 +-
.../system/common/sql/example_sql_execute_query.py | 2 +-
.../system/docker/example_docker_copy_data.py | 2 +-
.../elasticsearch/example_elasticsearch_query.py | 2 +-
.../cloud/cloud_batch/example_cloud_batch.py | 2 +-
.../google/cloud/cloud_run/example_cloud_run.py | 2 +-
.../system/google/cloud/gcs/example_gcs_to_gcs.py | 2 +-
.../vertex_ai/example_vertex_ai_model_service.py | 18 +-
pyproject.toml | 11 +-
scripts/ci/docker-compose/integration-keycloak.yml | 62 +
.../ci/docker-compose/keycloak/init-keycloak-db.sh | 12 +-
.../docker-compose/keycloak/keycloak-entrypoint.sh | 45 +
.../pre_commit/base_operator_partial_arguments.py | 80 +-
scripts/ci/pre_commit/check_deferrable_default.py | 8 +-
scripts/ci/pre_commit/sync_init_decorator.py | 201 +-
scripts/docker/entrypoint_ci.sh | 2 +-
.../docker/install_from_docker_context_files.sh | 33 +-
task_sdk/pyproject.toml | 35 +-
task_sdk/src/airflow/sdk/__init__.py | 39 +-
.../src/airflow/sdk/definitions}/__init__.py | 0
.../airflow/sdk/definitions/abstractoperator.py | 261 +
.../src/airflow/sdk/definitions/baseoperator.py | 1226 ++
.../src/airflow/sdk/definitions/contextmanager.py | 124 +
task_sdk/src/airflow/sdk/definitions/dag.py | 1119 ++
.../src/airflow/sdk/definitions/decorators.py | 33 +-
.../src/airflow/sdk/definitions/edges.py | 29 +-
task_sdk/src/airflow/sdk/definitions/mixins.py | 121 +
.../src/airflow/sdk/definitions/node.py | 174 +-
.../src/airflow/sdk/definitions/taskgroup.py | 290 +-
.../src/airflow/sdk/exceptions.py | 0
task_sdk/src/airflow/sdk/types.py | 75 +
task_sdk/tests/defintions/test_baseoperator.py | 343 +
task_sdk/tests/defintions/test_dag.py | 419 +
tests/always/test_project_structure.py | 1 -
tests/api_connexion/endpoints/test_dag_endpoint.py | 32 -
.../api_connexion/endpoints/test_xcom_endpoint.py | 26 +-
tests/api_connexion/schemas/test_dag_schema.py | 5 -
tests/api_connexion/schemas/test_xcom_schema.py | 10 +-
.../core_api/routes/public/test_dag_sources.py | 110 +
.../core_api/routes/public/test_dags.py | 1 -
.../core_api/routes/public/test_task_instances.py | 462 +
tests/cli/commands/test_backfill_command.py | 2 +-
tests/cli/commands/test_config_command.py | 24 +-
tests/cli/commands/test_task_command.py | 4 +-
tests/core/test_airflow_module.py | 25 -
tests/core/test_configuration.py | 29 +-
tests/core/test_core.py | 47 +-
tests/core/test_example_dags_system.py | 2 +-
tests/core/test_sentry.py | 2 +-
tests/dags/test_assets.py | 2 +-
tests/dags/test_cli_triggered_dags.py | 2 +-
tests/dags/test_dag_xcom_openlineage.py | 2 +-
tests/dags/test_dagrun_fast_follow.py | 2 +-
tests/dags/test_future_start_date.py | 2 +-
tests/dags/test_invalid_param.py | 2 +-
tests/dags/test_invalid_param2.py | 2 +-
tests/dags/test_invalid_param3.py | 2 +-
tests/dags/test_invalid_param4.py | 2 +-
tests/dags/test_logging_in_dag.py | 2 +-
tests/dags/test_mapped_classic.py | 2 +-
tests/dags/test_mark_state.py | 2 +-
tests/dags/test_on_failure_callback.py | 2 +-
tests/dags/test_task_view_type_check.py | 2 +-
tests/dags/test_valid_param.py | 2 +-
tests/dags/test_valid_param2.py | 2 +-
tests/dags_corrupted/test_impersonation_custom.py | 2 +-
tests/decorators/test_python.py | 2 +-
tests/decorators/test_python_virtualenv.py | 2 +-
tests/io/test_path.py | 2 -
tests/jobs/test_local_task_job.py | 14 +-
tests/jobs/test_scheduler_job.py | 64 +-
tests/jobs/test_triggerer_job.py | 2 +-
tests/models/test_backfill.py | 2 +-
tests/models/test_baseoperator.py | 301 +-
tests/models/test_baseoperatormeta.py | 2 +-
tests/models/test_cleartasks.py | 2 +-
tests/models/test_dag.py | 439 +-
tests/models/test_dagbag.py | 6 +-
tests/models/test_dagrun.py | 13 +-
tests/models/test_mappedoperator.py | 6 +-
tests/models/test_renderedtifields.py | 2 +-
tests/models/test_taskinstance.py | 29 +-
tests/models/test_xcom_arg.py | 2 +-
tests/sensors/test_external_task_sensor.py | 2 +-
.../serialization/serializers/test_serializers.py | 7 +-
tests/serialization/test_dag_serialization.py | 95 +-
tests/serialization/test_pydantic_models.py | 16 +-
tests/serialization/test_serialized_objects.py | 2 +-
tests/task/{task_runner => }/conftest.py | 7 +-
tests/task/task_runner/test_base_task_runner.py | 59 -
tests/task/task_runner/test_cgroup_task_runner.py | 45 -
tests/task/task_runner/test_task_runner.py | 66 -
.../{task_runner => }/test_standard_task_runner.py | 36 +-
.../deps/test_not_previously_skipped_dep.py | 2 +-
tests/utils/log/test_log_reader.py | 2 +-
tests/utils/test_compression.py | 87 -
tests/utils/test_dates.py | 89 -
tests/utils/test_db_cleanup.py | 2 +-
tests/utils/test_dot_renderer.py | 6 +-
tests/utils/test_edgemodifier.py | 2 +-
tests/utils/test_log_handlers.py | 2 +-
tests/utils/test_task_group.py | 20 +-
tests/www/views/test_views_dagrun.py | 76 +
tests/www/views/test_views_rendered.py | 3 +-
tests_common/_internals/capture_warnings.py | 6 +-
tests_common/pytest_plugin.py | 129 +-
tests_common/test_utils/compat.py | 4 +
tests_common/test_utils/db.py | 7 -
tests_common/test_utils/mock_operators.py | 15 +-
uv.lock | 13938 +++++++++++++++++++
403 files changed, 27045 insertions(+), 10480 deletions(-)
create mode 100644 airflow/api_fastapi/core_api/routes/public/dag_sources.py
create mode 100644 airflow/api_fastapi/core_api/routes/public/task_instances.py
copy task_sdk/tests/test_hello.py =>
airflow/api_fastapi/core_api/serializers/dag_sources.py (85%)
copy task_sdk/tests/test_hello.py =>
airflow/api_fastapi/core_api/serializers/job.py (63%)
create mode 100644 airflow/api_fastapi/core_api/serializers/task_instances.py
rename task_sdk/tests/test_hello.py =>
airflow/api_fastapi/core_api/serializers/trigger.py (67%)
create mode 100644
airflow/migrations/versions/0044_3_0_0__drop_task_fail_table.py
delete mode 100644 airflow/models/taskfail.py
rename airflow/task/{task_runner => }/standard_task_runner.py (62%)
delete mode 100644 airflow/task/task_runner/__init__.py
delete mode 100644 airflow/task/task_runner/base_task_runner.py
delete mode 100644 airflow/task/task_runner/cgroup_task_runner.py
create mode 100644 airflow/ui/src/pages/DagsList/LatestRun.tsx
create mode 100644 airflow/ui/src/pages/DagsList/RecentRuns.tsx
create mode 100644 airflow/ui/src/pages/DagsList/Schedule.tsx
create mode 100644 airflow/ui/src/queries/useDags.tsx
create mode 100644 airflow/ui/src/utils/stateColor.ts
delete mode 100644 airflow/utils/compression.py
create mode 100644 docs/apache-airflow/core-concepts/backfill.rst
delete mode 100644 newsfragments/42953.significant
create mode 100644 newsfragments/42953.significant.rst
create mode 100644 newsfragments/43490.significant.rst
create mode 100644 newsfragments/43530.significant.rst
create mode 100644 newsfragments/43533.significant.rst
create mode 100644 newsfragments/43562.significant.rst
create mode 100644 newsfragments/43568.significant.rst
create mode 100644 newsfragments/aip-72.significant.rst
copy {airflow/api_connexion =>
providers/src/airflow/providers/common/compat/standard}/__init__.py (100%)
create mode 100644
providers/src/airflow/providers/common/compat/standard/operators.py
copy airflow/utils/hashlib_wrapper.py =>
providers/src/airflow/providers/common/compat/standard/utils.py (63%)
rename {airflow =>
providers/src/airflow/providers/standard}/operators/python.py (94%)
rename {airflow => providers/src/airflow/providers/standard}/sensors/python.py
(100%)
copy {airflow/api_connexion =>
providers/src/airflow/providers/standard/utils}/__init__.py (100%)
rename {airflow =>
providers/src/airflow/providers/standard}/utils/python_virtualenv.py (86%)
rename {airflow =>
providers/src/airflow/providers/standard}/utils/python_virtualenv_script.jinja2
(95%)
copy {airflow/api_connexion =>
providers/tests/cncf/kubernetes/backcompat}/__init__.py (100%)
create mode 100644
providers/tests/cncf/kubernetes/backcompat/test_backwards_compat_converters.py
rename {tests => providers/tests/standard}/operators/test_python.py (98%)
rename {tests => providers/tests/standard}/sensors/test_python.py (95%)
copy {airflow/api_connexion => providers/tests/standard/utils}/__init__.py
(100%)
rename {tests => providers/tests/standard}/utils/test_python_virtualenv.py
(85%)
create mode 100644 scripts/ci/docker-compose/integration-keycloak.yml
rename tests/task/task_runner/__init__.py =>
scripts/ci/docker-compose/keycloak/init-keycloak-db.sh (70%)
mode change 100644 => 100755
create mode 100755 scripts/ci/docker-compose/keycloak/keycloak-entrypoint.sh
copy {airflow/api_connexion =>
task_sdk/src/airflow/sdk/definitions}/__init__.py (100%)
create mode 100644 task_sdk/src/airflow/sdk/definitions/abstractoperator.py
create mode 100644 task_sdk/src/airflow/sdk/definitions/baseoperator.py
create mode 100644 task_sdk/src/airflow/sdk/definitions/contextmanager.py
create mode 100644 task_sdk/src/airflow/sdk/definitions/dag.py
copy airflow/utils/hashlib_wrapper.py =>
task_sdk/src/airflow/sdk/definitions/decorators.py (52%)
copy airflow/utils/edgemodifier.py =>
task_sdk/src/airflow/sdk/definitions/edges.py (88%)
create mode 100644 task_sdk/src/airflow/sdk/definitions/mixins.py
copy airflow/models/taskmixin.py =>
task_sdk/src/airflow/sdk/definitions/node.py (57%)
copy airflow/utils/task_group.py =>
task_sdk/src/airflow/sdk/definitions/taskgroup.py (73%)
copy airflow/api_connexion/__init__.py =>
task_sdk/src/airflow/sdk/exceptions.py (100%)
create mode 100644 task_sdk/src/airflow/sdk/types.py
create mode 100644 task_sdk/tests/defintions/test_baseoperator.py
create mode 100644 task_sdk/tests/defintions/test_dag.py
create mode 100644 tests/api_fastapi/core_api/routes/public/test_dag_sources.py
create mode 100644
tests/api_fastapi/core_api/routes/public/test_task_instances.py
rename tests/task/{task_runner => }/conftest.py (89%)
delete mode 100644 tests/task/task_runner/test_base_task_runner.py
delete mode 100644 tests/task/task_runner/test_cgroup_task_runner.py
delete mode 100644 tests/task/task_runner/test_task_runner.py
rename tests/task/{task_runner => }/test_standard_task_runner.py (92%)
delete mode 100644 tests/utils/test_compression.py
delete mode 100644 tests/utils/test_dates.py
create mode 100644 uv.lock
