This is an automated email from the ASF dual-hosted git repository.
Lee-W pushed a change to branch ci-upgrade-main
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit a037bec9664 [main] CI: Upgrade important CI environment
add 79e9628fa1d Add example plugins and expand docs for asset partitions
(#68889)
add 7d96ee13a7f Java SDK docs: JUL setup(), pin java_executable,
config-reload note (#68938)
add 3e48e8d852a Surface databricks API errors as a separate exception
(#69019)
add 0619574dd21 Add `has_note` key to the API response for TI to render
saved note indicator in Airflow 3 Grid View UI (#68979)
add 85aa9b621f3 Defer Cadwyn import to keep FastAPI off the Task SDK
worker path (#69016)
add 3f600661aac Add Celery worker mp_start_method config to curb Python
3.14 memory (#69015)
add 3dc7a60a063 Fix GKE provider 401 on kubernetes client 36.x (#69032)
add b94ca10bb1e Deprecate implicit legacy SQL default in BigQuery
operators (#67113)
add 3733e62d6a4 Type asset_expression in the REST API so the UI does not
cast through unknown (#67725)
add 5226a1ba7f9 Reduce canary schedule frequency (AMD 4->2/day, ARM
8->2/day) (#68821)
add c217e9a577f Fix zimor unsound-ternary (#69026)
add 4e2f5d47ea2 Prepare ad-hoc provider documentation 2026-06-26 (#69022)
add ba8cfe47e57 Add proxy support to Databricks connections (#68527)
add 7d2c1a72e7f Extract configuration-specific logic from
_get_static_conn_params() (#68597)
add a85c7aad9f9 Import tableauserverclient lazily in TableauHook (#68971)
add b24194f48d9 Make test_cli_run_time tolerant of slow/cold CI runners
(#68968)
add 9b0fe283845 Stabilize testcontainers provider tests on slow (ARM) CI
runners (#68963)
add 419ac06e8ac Fix scheduled CI upgrade failing when its PR already
exists (#69018)
add e9ef38b9e00 Update local Otel Collector and Prometheus Versions to
support Exponential Histograms. (#69040)
add 9517b492391 Bump the uv-dependency-updates group across 1 directory
with 2 updates (#69044)
add 78263d0c538 Move known_airflow_exceptions.txt out of scripts/ into
generated/ (#69057)
add 64e9b865fe2 Bump tornado from 6.5.6 to 6.5.7 (#68609)
add c0404ba5e1a Bump cryptography from 48.0.0 to 48.0.1 (#68610)
add e6c1ae67ac8 Bump zeep from 4.3.2 to 4.3.3 (#68780)
add a3415620fe5 Remove dependabot updates for the v2-11-test branch
(#69061)
add 48127317d21 Bump starlette from 1.2.1 to 1.3.1 (#68626)
add 754c10633c2 Bump the registry-package-updates group across 1 directory
with 2 updates (#69043)
add 930d96e0da6 Add output_files_to_xcom parameter to
SageMakerProcessingOperator (#69002)
add 5a8409cc527 Add S3CopyPrefixOperator to copy all objects under a
prefix (#68946)
add 16144bd1b57 Bump boto3 in /dev/breeze in the uv-dependency-updates
group (#69067)
add 5e1d3e94188 [AIP-94] Add deprecation marker to assets list/details cli
commands (#68531)
add a2bf0ae7b3a Fix timezone-dependent symlink test in
TestFileProcessorHandler (#68910)
add d0450666eb3 Remove a Dag Run or Task Instance note when its content is
cleared (#69033)
add 427117ee911 Update OpenAI provider to OpenAI Python SDK 2.x (#69068)
add 438107f03ef Add Workload Identity authentication to OpenAI provider
(#69069)
add ff471253f18 Add OpenAI Responses and Conversations support (#69070)
add a02634a9572 [main] CI: Upgrade important CI environment
add 30a4c0b8f28 fixup! [main] CI: Upgrade important CI environment
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a037bec9664)
\
N -- N -- N refs/heads/ci-upgrade-main (30a4c0b8f28)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/dependabot.yml | 46 --
.github/workflows/ci-amd.yml | 9 +-
.github/workflows/ci-arm.yml | 6 +-
.github/workflows/publish-docs-to-s3.yml | 2 +-
.github/workflows/run-unit-tests.yml | 4 +-
.../logging-monitoring/metrics.rst | 36 +
.../docs/authoring-and-scheduling/assets.rst | 267 ++++++++
.../language-sdks/java.rst | 42 +-
.../api_fastapi/core_api/datamodels/common.py | 123 +++-
.../api_fastapi/core_api/datamodels/dags.py | 3 +-
.../api_fastapi/core_api/datamodels/ui/assets.py | 3 +-
.../api_fastapi/core_api/datamodels/ui/dags.py | 3 +-
.../api_fastapi/core_api/datamodels/ui/grid.py | 1 +
.../core_api/datamodels/ui/partitioned_dag_runs.py | 5 +-
.../api_fastapi/core_api/openapi/_private_ui.yaml | 153 ++++-
.../core_api/openapi/v2-rest-api-generated.yaml | 125 +++-
.../api_fastapi/core_api/routes/ui/assets.py | 27 +-
.../airflow/api_fastapi/core_api/routes/ui/grid.py | 11 +-
.../core_api/routes/ui/partitioned_dag_runs.py | 38 +-
.../core_api/services/public/dag_run.py | 6 +-
.../core_api/services/public/task_instances.py | 4 +-
.../api_fastapi/core_api/services/ui/grid.py | 5 +
.../src/airflow/cli/commands/asset_command.py | 2 +
.../example_dags/example_asset_partition.py | 10 +-
.../plugins/custom_partition_mapper.py | 96 +++
.../plugins/custom_partition_timetable.py | 55 ++
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 258 +++++++-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 81 ++-
.../components/AssetExpression/AssetExpression.tsx | 6 +-
.../src/components/AssetExpression/AssetNode.tsx | 9 +-
.../ui/src/components/AssetExpression/index.ts | 2 +-
.../ui/src/components/AssetExpression/types.ts | 27 +-
.../ui/src/components/AssetProgressCell.tsx | 4 +-
.../airflow/ui/src/layouts/Details/Grid/GridTI.tsx | 16 +-
.../layouts/Details/Grid/TaskInstancesColumn.tsx | 1 +
.../ui/src/pages/DagsList/AssetSchedule.tsx | 2 +-
.../api_fastapi/core_api/datamodels/test_common.py | 117 ++++
.../core_api/routes/public/test_dag_run.py | 16 +-
.../core_api/routes/public/test_dags.py | 35 +-
.../core_api/routes/public/test_plugins.py | 10 +-
.../core_api/routes/public/test_task_instances.py | 14 +
.../api_fastapi/core_api/routes/ui/test_grid.py | 35 +-
.../unit/cli/commands/test_command_deprecations.py | 2 +
airflow-core/tests/unit/cli/test_cli_parser.py | 10 +-
.../tests/unit/plugins/test_plugins_manager.py | 4 +-
.../unit/utils/log/test_file_processor_handler.py | 4 +-
.../src/airflowctl/api/datamodels/generated.py | 234 +++++--
.../src/airflow_breeze/commands/ci_commands.py | 21 +-
.../airflow_breeze/commands/testing_commands.py | 25 +
dev/breeze/src/airflow_breeze/global_constants.py | 11 +
dev/breeze/src/airflow_breeze/utils/run_tests.py | 36 +-
dev/breeze/tests/test_run_test_args.py | 79 ++-
dev/breeze/uv.lock | 16 +-
.../known_airflow_exceptions.txt | 2 +-
providers/.last_release_date.txt | 2 +-
providers/amazon/docs/operators/s3/s3.rst | 18 +
.../airflow/providers/amazon/aws/operators/s3.py | 155 +++++
.../providers/amazon/aws/operators/sagemaker.py | 100 ++-
.../amazon/tests/system/amazon/aws/example_s3.py | 12 +
.../tests/system/amazon/aws/example_sagemaker.py | 52 +-
.../tests/unit/amazon/aws/operators/test_s3.py | 245 +++++++
.../aws/operators/test_sagemaker_processing.py | 233 +++++++
providers/apache/spark/docs/changelog.rst | 1 +
providers/celery/README.rst | 6 +-
providers/celery/docs/changelog.rst | 16 +
providers/celery/docs/index.rst | 6 +-
providers/celery/provider.yaml | 32 +-
providers/celery/pyproject.toml | 6 +-
.../src/airflow/providers/celery/__init__.py | 2 +-
.../airflow/providers/celery/cli/celery_command.py | 20 +-
.../airflow/providers/celery/get_provider_info.py | 14 +
.../tests/unit/celery/cli/test_celery_command.py | 27 +-
providers/cncf/kubernetes/docs/changelog.rst | 3 +
.../databricks/docs/connections/databricks.rst | 30 +
.../src/airflow/providers/databricks/exceptions.py | 8 +
.../providers/databricks/hooks/databricks_base.py | 79 ++-
.../tests/unit/databricks/hooks/test_databricks.py | 104 ++-
.../unit/databricks/hooks/test_databricks_base.py | 55 ++
.../unit/databricks/hooks/test_databricks_sql.py | 24 +
providers/fab/README.rst | 6 +-
providers/fab/docs/changelog.rst | 17 +
providers/fab/docs/index.rst | 6 +-
providers/fab/provider.yaml | 3 +-
providers/fab/pyproject.toml | 6 +-
.../fab/src/airflow/providers/fab/__init__.py | 2 +-
providers/google/README.rst | 6 +-
providers/google/docs/changelog.rst | 17 +
providers/google/docs/connections/bigquery.rst | 4 +-
providers/google/docs/index.rst | 6 +-
providers/google/docs/operators/cloud/bigquery.rst | 10 +
providers/google/provider.yaml | 3 +-
providers/google/pyproject.toml | 6 +-
.../src/airflow/providers/google/__init__.py | 2 +-
.../providers/google/cloud/hooks/bigquery.py | 16 +-
.../google/cloud/hooks/kubernetes_engine.py | 13 +-
.../providers/google/cloud/operators/bigquery.py | 63 +-
.../google/cloud/transfers/gcs_to_bigquery.py | 3 +
providers/google/tests/deprecations_ignore.yml | 7 +
.../cloud/bigquery/example_bigquery_queries.py | 3 +
.../tests/unit/google/cloud/hooks/test_bigquery.py | 65 +-
.../google/cloud/hooks/test_kubernetes_engine.py | 27 +
.../unit/google/cloud/operators/test_bigquery.py | 51 ++
providers/keycloak/README.rst | 6 +-
providers/keycloak/docs/changelog.rst | 23 +
providers/keycloak/docs/index.rst | 6 +-
providers/keycloak/provider.yaml | 3 +-
providers/keycloak/pyproject.toml | 6 +-
.../src/airflow/providers/keycloak/__init__.py | 2 +-
providers/openai/README.rst | 2 +-
providers/openai/docs/connections.rst | 36 +
providers/openai/docs/index.rst | 2 +-
providers/openai/docs/operators/openai.rst | 41 ++
providers/openai/pyproject.toml | 2 +-
.../src/airflow/providers/openai/hooks/openai.py | 167 ++++-
.../airflow/providers/openai/operators/openai.py | 57 +-
.../openai/tests/system/openai/example_openai.py | 17 +-
.../openai/tests/unit/openai/hooks/test_openai.py | 216 +++++-
.../tests/unit/openai/operators/test_openai.py | 39 +-
.../openai/tests/unit/openai/test_exceptions.py | 3 +-
.../airflow/providers/snowflake/hooks/snowflake.py | 193 ++++--
.../src/airflow/providers/tableau/hooks/tableau.py | 15 +-
.../tests/unit/tableau/hooks/test_tableau.py | 54 +-
registry/package.json | 4 +-
registry/pnpm-lock.yaml | 74 +--
scripts/ci/docker-compose/integration-otel.yml | 7 +-
.../ci/docker-compose/otel-collector-config.yml | 7 +-
scripts/ci/prek/check_ci_workflows_in_sync.py | 15 +-
.../ci/prek/check_new_airflow_exception_usage.py | 10 +-
.../airflow/sdk/execution_time/schema/__init__.py | 13 +-
.../airflow/sdk/execution_time/schema/migrator.py | 9 +-
.../sdk/execution_time/schema/versions/__init__.py | 36 +-
.../execution_time/schema/test_migrator.py | 35 +
uv.lock | 729 +++++++++++----------
133 files changed, 4689 insertions(+), 891 deletions(-)
create mode 100644
airflow-core/src/airflow/example_dags/plugins/custom_partition_mapper.py
create mode 100644
airflow-core/src/airflow/example_dags/plugins/custom_partition_timetable.py
create mode 100644
airflow-core/tests/unit/api_fastapi/core_api/datamodels/test_common.py
rename {scripts/ci/prek => generated}/known_airflow_exceptions.txt (99%)
