This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch jason/lang-sdk-e2e/03-native-dag-parse
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard a70eb430b82 Find a native Dag's coordinator key through its importer
discard 917db98d4a1 Wait for the killed leftover in the Lang-SDK processor test
discard 778193ac00d Move the client-less Lang-SDK parse out of this change
discard e3bd6f0b0b3 Note in ADR-0010 that the manager routes claimed files for
now
discard 85c86080632 Clear import errors after the native-file Dag bag sync test
discard a68f51b07cd Apply the multi-team task checks to Lang-SDK Dags
discard bfcc16ccfca Spec the mocks of a hand-built Lang-SDK processor in tests
discard 4c5f7e2572f Name the setting in a Lang-SDK parse timeout error
discard 1697091e40a Kill what a Lang-SDK runtime leaves in its process group
discard 985271f2515 Match the schema error's JSON path in a Lang-SDK parse test
discard 2a253c147f6 Build each bundle's Dag importer registry before freezing
the heap
discard f13c43fa99b Fail a native Dag's task on a Python worker without retries
discard 77bd5b757d4 Find the claiming coordinator in the Task SDK and wrap it
in core
discard fe4b272b1ae Start each Dag file processor with explicit arguments
discard 76b203ed127 Drop callbacks for Lang-SDK Dag files before queuing the
file
discard 3dc7c020ac7 Report a native Dag's task that reaches a Python worker
discard 426d3bb8956 Report a Lang-SDK parse past its import timeout as an
import error
discard f72fb46b402 Parse coordinator-claimed Dag files with their runtime in
the Dag processor
discard 42c00d465ae Explain dag_bundle_name by the coordinator that parses a
Dag file
discard e610aa76488 Hand out a coordinator's Dag importer through
get_dag_importer alone
discard 6afe857c7e0 Say dag_bundle_name is only needed for two coordinators of
one class
discard 13d82cc57ec Drop serves_bundle so get_parsed_bundles is the only answer
discard 266131caf2a Check Dag file claims per bundle on the parse path
discard 2cb5c904df8 Import the coordinator manager at the top of the importer
base
discard fd5eb37bd70 Let a coordinator parse the Dag files of the bundles it
serves
discard 0afbeb1e11c Sync the Java SDK copy of the Dag schema
discard a63b9f1c091 Serialize the SDK Dag in tests that write a dag_maker Dag
discard 8cfd06019f4 Note in the parsing ADR that runtimes may omit
config-backed fields
discard fc73b8cc582 Check SDK Dags with the Dag processor's validation in
conformance
discard 2f770c94c9a Reject malformed task entries in a serialized Dag
discard bcba1cbf97f Fill a serialized Dag's unset settings from the Airflow
config
discard 235be2d6bda Check that a serialized Dag can be stored and loaded
discard ca0d7328288 Move the Dag file processor's shared plumbing into a base
class
add e14854221ad Group and filter Dags by folder on the Dags page (#68544)
add 18af65032e3 Render the Language SDK Spec diagrams as Mermaid (#74065)
add be53013349e Support azure-datalake-store 1.x in AzureDataLakeHook
(#73819)
add 66508dba3c2 Move the Dag file processor's shared plumbing into a base
class
add 9024fcd611a Keep the parse log handle required on the base Dag file
processor
add 87b4b9c9fa3 Check that a serialized Dag can be stored and loaded
add 7211bbe7d58 Fill a serialized Dag's unset settings from the Airflow
config
add e154d35e338 Reject malformed task entries in a serialized Dag
add 5c5ec770319 Check SDK Dags with the Dag processor's validation in
conformance
add ea4f7637ad9 Note in the parsing ADR that runtimes may omit
config-backed fields
add f1791c6a224 Serialize the SDK Dag in tests that write a dag_maker Dag
add 4f3cecbd67d Sync the Java SDK copy of the Dag schema
add c930dd6c297 Sync the Go SDK copy of the Dag schema
add aba8fbb4439 Let a coordinator parse the Dag files of the bundles it
serves
add 51a30974ca1 Import the coordinator manager at the top of the importer
base
add e2870bfa309 Check Dag file claims per bundle on the parse path
add 38d9f574563 Drop serves_bundle so get_parsed_bundles is the only answer
add 89df05be5ea Say dag_bundle_name is only needed for two coordinators of
one class
add 88b80ff7c67 Hand out a coordinator's Dag importer through
get_dag_importer alone
add 63a5cd3a808 Explain dag_bundle_name by the coordinator that parses a
Dag file
add 30cae48ebc8 Parse coordinator-claimed Dag files with their runtime in
the Dag processor
add 8eab3c08506 Report a Lang-SDK parse past its import timeout as an
import error
add 0736d24c011 Report a native Dag's task that reaches a Python worker
add 9acc7abea12 Drop callbacks for Lang-SDK Dag files before queuing the
file
add ff51e1d85a9 Start each Dag file processor with explicit arguments
add 2ec5a41e906 Find the claiming coordinator in the Task SDK and wrap it
in core
add c64fbd6eca3 Fail a native Dag's task on a Python worker without retries
add d7684f3c95b Build each bundle's Dag importer registry before freezing
the heap
add 2bdae62a32f Match the schema error's JSON path in a Lang-SDK parse test
add 5d8cc694c2e Kill what a Lang-SDK runtime leaves in its process group
add 9ef78a997b4 Name the setting in a Lang-SDK parse timeout error
add 403db14ca2b Spec the mocks of a hand-built Lang-SDK processor in tests
add 808f66c1371 Apply the multi-team task checks to Lang-SDK Dags
add 680f7db3cee Clear import errors after the native-file Dag bag sync test
add 6366a8b229b Note in ADR-0010 that the manager routes claimed files for
now
add 1f7f2ae6245 Move the client-less Lang-SDK parse out of this change
add f978d579bdf Wait for the killed leftover in the Lang-SDK processor test
add 91e12cdf26a Find a native Dag's coordinator key through its importer
add 22c2b156c13 Pass a parse log handle to the Lang-SDK processor in tests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a70eb430b82)
\
N -- N -- N refs/heads/jason/lang-sdk-e2e/03-native-dag-parse
(22c2b156c13)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../api_fastapi/common/parameters/__init__.py | 1 +
.../airflow/api_fastapi/common/parameters/dag.py | 41 +++
.../api_fastapi/core_api/datamodels/ui/dags.py | 14 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 90 ++++++
.../airflow/api_fastapi/core_api/routes/ui/dags.py | 51 ++++
.../src/airflow/dag_processing/processor.py | 6 +-
.../src/airflow/ui/openapi-gen/queries/common.ts | 9 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 25 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 37 +++
.../ui/openapi-gen/requests/services.gen.ts | 29 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 32 ++
.../airflow/ui/public/i18n/locales/en/dags.json | 7 +
.../src/airflow/ui/src/constants/localStorage.ts | 1 +
.../src/airflow/ui/src/constants/searchParams.ts | 2 +
.../DagsList/DagFolderTree/DagFolderTree.test.tsx | 323 +++++++++++++++++++++
.../pages/DagsList/DagFolderTree/DagFolderTree.tsx | 216 ++++++++++++++
.../DagsList/DagFolderTree/buildFolderTree.test.ts | 119 ++++++++
.../DagsList/DagFolderTree/buildFolderTree.ts | 152 ++++++++++
.../DagsList/DagFolderTree}/index.ts | 3 +-
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 188 ++++++++----
.../useColorMode.tsx => queries/useDagFolders.ts} | 18 +-
.../src/airflow/ui/src/queries/useDags.tsx | 6 +
.../api_fastapi/core_api/routes/ui/test_dags.py | 183 ++++++++++++
.../unit/dag_processing/test_lang_sdk_processor.py | 7 +-
go-sdk/schema/dag-schema.json | 13 +-
providers/microsoft/azure/README.rst | 4 +-
providers/microsoft/azure/docs/index.rst | 4 +-
providers/microsoft/azure/pyproject.toml | 6 +-
.../providers/microsoft/azure/hooks/data_lake.py | 13 +-
.../unit/microsoft/azure/hooks/test_data_lake.py | 61 ++--
pyproject.toml | 1 -
task-sdk/docs/conf.py | 1 +
task-sdk/docs/lang-sdk-spec.rst | 83 +++---
uv.lock | 11 +-
37 files changed, 1652 insertions(+), 180 deletions(-)
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.test.ts
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.ts
copy airflow-core/src/airflow/ui/src/{layouts/Details/Gantt =>
pages/DagsList/DagFolderTree}/index.ts (91%)
copy airflow-core/src/airflow/ui/src/{context/colorMode/useColorMode.tsx =>
queries/useDagFolders.ts} (62%)
