This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch jason/lang-sdk-e2e/07-java-sdk
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 0c530021239 Java SDK: Serialize native Dags to DagSerialization v3 on
parse requests
discard 2ccfade287f Java SDK: Declare a Dag's task graph with @Builder.Deps
discard 32c2b5c80a1 TS SDK: add triggerDagRun to the native Dag surface
discard 4542ce9778e TS SDK: add multi-way branching with dag.switch
discard 3c041acca23 TS SDK: add conditional branching with dag.if
discard 4c33164da25 Stub Dag file discovery with DagFileInfo in the Lang-SDK
manager tests
discard 1af68883a10 Handle Dags nested at any depth in Dag archives
discard 8eab80a0098 Migrating from filepath based Dag to DagDefinition
discovery
discard 44e11868ff9 Hand out the Node Dag importer through get_dag_importer
discard 4d6f5242f0f Pin the Code view to the entry module in Node bundle tests
discard 47a97f0f08d Let NodeDagImporter source read errors reach the caller
discard 8a8c43e4344 Import NodeDagImporter at the top of the Node coordinator
discard 9869a1e4f3a Warn against defining a Dag in both TypeScript and Python
discard 82d5a7ea320 Align the TypeScript build docs with the Code view
discard d73eaf8b39e Say when the Dag processor needs Node.js for TypeScript
bundles
discard 42d6dd45114 Run a native TypeScript Dag's task from its own bundle
discard 55d17f96ece Share the bundle section mutation helper between Node tests
discard 6b8fd989bf6 Document parsing native TypeScript Dags
discard 4f3c85f0dc1 Parse native TypeScript Dags with the Node coordinator
discard 51198c6125c Hand out the Java Dag importer through get_dag_importer
discard 9998e1ea102 Keep the Java parse_dag test from changing the pytest
worker
discard ae5ebe24c46 Stop promising that Java dependency JARs are not parsed
discard e02d9fb1f39 Reject a Java Dag JAR whose SDK cannot answer a parse
discard c44b01925f3 Run the JAR a native Java Dag was parsed from
discard cf32a1492cb Report why a Java Dag JAR cannot be read
discard 3ad83f79caf Pass the task's Dag file to the coordinator's command
builder
discard 0d10d21b2f1 Run a task on its run's version when dag_bundle_name names
its bundle
discard f08c7823a97 Document native Java Dag parsing
discard cbeab5bb190 Name the JAR manifest attributes the Java coordinator reads
discard 2bfbd658b47 Test the Java Dag source placeholder for a JAR without a
manifest
discard aab428b0bf7 Treat an unreadable JAR like one without a manifest
discard a7093859961 Parse native Java Dags with their coordinator
discard 4f8762224ec Embed the Dag source in Java bundle JARs
discard 00eaafa1752 Build the JVM command that parses a Dag JAR
discard 447eea4b249 Read JAR manifests per the JAR specification
discard 6c3ae5d7c33 Ignore the bagged Dag union in the external task sensor
test
discard 897a3ef4c63 List the tasks of a native Lang-SDK Dag
discard dca01a5a0c6 Bag native Lang-SDK Dags in core, only when asked
discard ff48b1f19aa Let a Dag file processor run without an API client
discard e8fbf760526 Parse coordinator-claimed Dag files with their runtime in
a Dag bag
discard a70eb430b82 Find a native Dag's coordinator key through its importer
discard 917db98d4a1 Wait for the killed leftover in the Lang-SDK processor test
discard 778193ac00d Move the client-less Lang-SDK parse out of this change
discard e3bd6f0b0b3 Note in ADR-0010 that the manager routes claimed files for
now
discard 85c86080632 Clear import errors after the native-file Dag bag sync test
discard a68f51b07cd Apply the multi-team task checks to Lang-SDK Dags
discard bfcc16ccfca Spec the mocks of a hand-built Lang-SDK processor in tests
discard 4c5f7e2572f Name the setting in a Lang-SDK parse timeout error
discard 1697091e40a Kill what a Lang-SDK runtime leaves in its process group
discard 985271f2515 Match the schema error's JSON path in a Lang-SDK parse test
discard 2a253c147f6 Build each bundle's Dag importer registry before freezing
the heap
discard f13c43fa99b Fail a native Dag's task on a Python worker without retries
discard 77bd5b757d4 Find the claiming coordinator in the Task SDK and wrap it
in core
discard fe4b272b1ae Start each Dag file processor with explicit arguments
discard 76b203ed127 Drop callbacks for Lang-SDK Dag files before queuing the
file
discard 3dc7c020ac7 Report a native Dag's task that reaches a Python worker
discard 426d3bb8956 Report a Lang-SDK parse past its import timeout as an
import error
discard f72fb46b402 Parse coordinator-claimed Dag files with their runtime in
the Dag processor
discard 42c00d465ae Explain dag_bundle_name by the coordinator that parses a
Dag file
discard e610aa76488 Hand out a coordinator's Dag importer through
get_dag_importer alone
discard 6afe857c7e0 Say dag_bundle_name is only needed for two coordinators of
one class
discard 13d82cc57ec Drop serves_bundle so get_parsed_bundles is the only answer
discard 266131caf2a Check Dag file claims per bundle on the parse path
discard 2cb5c904df8 Import the coordinator manager at the top of the importer
base
discard fd5eb37bd70 Let a coordinator parse the Dag files of the bundles it
serves
discard 0afbeb1e11c Sync the Java SDK copy of the Dag schema
discard a63b9f1c091 Serialize the SDK Dag in tests that write a dag_maker Dag
discard 8cfd06019f4 Note in the parsing ADR that runtimes may omit
config-backed fields
discard fc73b8cc582 Check SDK Dags with the Dag processor's validation in
conformance
discard 2f770c94c9a Reject malformed task entries in a serialized Dag
discard bcba1cbf97f Fill a serialized Dag's unset settings from the Airflow
config
discard 235be2d6bda Check that a serialized Dag can be stored and loaded
discard ca0d7328288 Move the Dag file processor's shared plumbing into a base
class
add e14854221ad Group and filter Dags by folder on the Dags page (#68544)
add 18af65032e3 Render the Language SDK Spec diagrams as Mermaid (#74065)
add be53013349e Support azure-datalake-store 1.x in AzureDataLakeHook
(#73819)
add 66508dba3c2 Move the Dag file processor's shared plumbing into a base
class
add 9024fcd611a Keep the parse log handle required on the base Dag file
processor
add 87b4b9c9fa3 Check that a serialized Dag can be stored and loaded
add 7211bbe7d58 Fill a serialized Dag's unset settings from the Airflow
config
add e154d35e338 Reject malformed task entries in a serialized Dag
add 5c5ec770319 Check SDK Dags with the Dag processor's validation in
conformance
add ea4f7637ad9 Note in the parsing ADR that runtimes may omit
config-backed fields
add f1791c6a224 Serialize the SDK Dag in tests that write a dag_maker Dag
add 4f3cecbd67d Sync the Java SDK copy of the Dag schema
add c930dd6c297 Sync the Go SDK copy of the Dag schema
add aba8fbb4439 Let a coordinator parse the Dag files of the bundles it
serves
add 51a30974ca1 Import the coordinator manager at the top of the importer
base
add e2870bfa309 Check Dag file claims per bundle on the parse path
add 38d9f574563 Drop serves_bundle so get_parsed_bundles is the only answer
add 89df05be5ea Say dag_bundle_name is only needed for two coordinators of
one class
add 88b80ff7c67 Hand out a coordinator's Dag importer through
get_dag_importer alone
add 63a5cd3a808 Explain dag_bundle_name by the coordinator that parses a
Dag file
add 30cae48ebc8 Parse coordinator-claimed Dag files with their runtime in
the Dag processor
add 8eab3c08506 Report a Lang-SDK parse past its import timeout as an
import error
add 0736d24c011 Report a native Dag's task that reaches a Python worker
add 9acc7abea12 Drop callbacks for Lang-SDK Dag files before queuing the
file
add ff51e1d85a9 Start each Dag file processor with explicit arguments
add 2ec5a41e906 Find the claiming coordinator in the Task SDK and wrap it
in core
add c64fbd6eca3 Fail a native Dag's task on a Python worker without retries
add d7684f3c95b Build each bundle's Dag importer registry before freezing
the heap
add 2bdae62a32f Match the schema error's JSON path in a Lang-SDK parse test
add 5d8cc694c2e Kill what a Lang-SDK runtime leaves in its process group
add 9ef78a997b4 Name the setting in a Lang-SDK parse timeout error
add 403db14ca2b Spec the mocks of a hand-built Lang-SDK processor in tests
add 808f66c1371 Apply the multi-team task checks to Lang-SDK Dags
add 680f7db3cee Clear import errors after the native-file Dag bag sync test
add 6366a8b229b Note in ADR-0010 that the manager routes claimed files for
now
add 1f7f2ae6245 Move the client-less Lang-SDK parse out of this change
add f978d579bdf Wait for the killed leftover in the Lang-SDK processor test
add 91e12cdf26a Find a native Dag's coordinator key through its importer
add 22c2b156c13 Pass a parse log handle to the Lang-SDK processor in tests
add 8b7d78fdba5 Parse coordinator-claimed Dag files with their runtime in
a Dag bag
add 5501436667c Let a Dag file processor run without an API client
add 69590246fdb Bag native Lang-SDK Dags in core, only when asked
add 0b449b3d908 List the tasks of a native Lang-SDK Dag
add 82f000bb947 Ignore the bagged Dag union in the external task sensor
test
add 9c3e5f0c0d5 Let a Dag file processor run without a parse log file
add b642ce42b7c Read JAR manifests per the JAR specification
add ba557380901 Build the JVM command that parses a Dag JAR
add bcd2defe70b Embed the Dag source in Java bundle JARs
add ee7cdd6926a Parse native Java Dags with their coordinator
add 2662263ad46 Treat an unreadable JAR like one without a manifest
add af129828a82 Test the Java Dag source placeholder for a JAR without a
manifest
add f6fafbced57 Name the JAR manifest attributes the Java coordinator reads
add 1bd815e6de5 Document native Java Dag parsing
add f4596dd2243 Run a task on its run's version when dag_bundle_name names
its bundle
add 620f89fcecd Pass the task's Dag file to the coordinator's command
builder
add 62c038ff917 Report why a Java Dag JAR cannot be read
add 5f546e305c4 Run the JAR a native Java Dag was parsed from
add bc7876e38f8 Reject a Java Dag JAR whose SDK cannot answer a parse
add 204ddea7bd8 Stop promising that Java dependency JARs are not parsed
add ecb55b91bcd Keep the Java parse_dag test from changing the pytest
worker
add b3a04e2a58e Hand out the Java Dag importer through get_dag_importer
add ed5bf214f23 Parse native TypeScript Dags with the Node coordinator
add 3d4f852aacf Document parsing native TypeScript Dags
add 2c730e212f4 Share the bundle section mutation helper between Node tests
add 6a997106e79 Run a native TypeScript Dag's task from its own bundle
add 71f9d62ddfe Say when the Dag processor needs Node.js for TypeScript
bundles
add 574c46a7ab8 Align the TypeScript build docs with the Code view
add fed3ceadbc4 Warn against defining a Dag in both TypeScript and Python
add 25b8a2d858c Import NodeDagImporter at the top of the Node coordinator
add 44bef1e3761 Let NodeDagImporter source read errors reach the caller
add 4e5d37bf8f3 Pin the Code view to the entry module in Node bundle tests
add 44f422dea17 Hand out the Node Dag importer through get_dag_importer
add 13f4d4c345b Migrating from filepath based Dag to DagDefinition
discovery
add 5ae9612bf91 Handle Dags nested at any depth in Dag archives
add 7df8d1fc6fc Stub Dag file discovery with DagFileInfo in the Lang-SDK
manager tests
add 4911badba47 TS SDK: add conditional branching with dag.if
add 5d99b09a78e TS SDK: add multi-way branching with dag.switch
add 23a6f677673 TS SDK: add triggerDagRun to the native Dag surface
add 9f54ff06de9 Java SDK: Declare a Dag's task graph with @Builder.Deps
add 51aab34160f Java SDK: Serialize native Dags to DagSerialization v3 on
parse requests
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0c530021239)
\
N -- N -- N refs/heads/jason/lang-sdk-e2e/07-java-sdk
(51aab34160f)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../api_fastapi/common/parameters/__init__.py | 1 +
.../airflow/api_fastapi/common/parameters/dag.py | 41 +++
.../api_fastapi/core_api/datamodels/ui/dags.py | 14 +
.../api_fastapi/core_api/openapi/_private_ui.yaml | 90 ++++++
.../airflow/api_fastapi/core_api/routes/ui/dags.py | 51 ++++
.../src/airflow/dag_processing/processor.py | 2 +
.../src/airflow/ui/openapi-gen/queries/common.ts | 9 +-
.../ui/openapi-gen/queries/ensureQueryData.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/prefetch.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/queries.ts | 25 +-
.../src/airflow/ui/openapi-gen/queries/suspense.ts | 25 +-
.../airflow/ui/openapi-gen/requests/schemas.gen.ts | 37 +++
.../ui/openapi-gen/requests/services.gen.ts | 29 +-
.../airflow/ui/openapi-gen/requests/types.gen.ts | 32 ++
.../airflow/ui/public/i18n/locales/en/dags.json | 7 +
.../src/airflow/ui/src/constants/localStorage.ts | 1 +
.../src/airflow/ui/src/constants/searchParams.ts | 2 +
.../DagsList/DagFolderTree/DagFolderTree.test.tsx | 323 +++++++++++++++++++++
.../pages/DagsList/DagFolderTree/DagFolderTree.tsx | 216 ++++++++++++++
.../DagsList/DagFolderTree/buildFolderTree.test.ts | 119 ++++++++
.../DagsList/DagFolderTree/buildFolderTree.ts | 152 ++++++++++
.../DagsList/DagFolderTree}/index.ts | 3 +-
.../src/airflow/ui/src/pages/DagsList/DagsList.tsx | 188 ++++++++----
.../useColorMode.tsx => queries/useDagFolders.ts} | 18 +-
.../src/airflow/ui/src/queries/useDags.tsx | 6 +
.../api_fastapi/core_api/routes/ui/test_dags.py | 183 ++++++++++++
.../unit/dag_processing/test_lang_sdk_processor.py | 7 +-
go-sdk/schema/dag-schema.json | 13 +-
providers/microsoft/azure/README.rst | 4 +-
providers/microsoft/azure/docs/index.rst | 4 +-
providers/microsoft/azure/pyproject.toml | 6 +-
.../providers/microsoft/azure/hooks/data_lake.py | 13 +-
.../unit/microsoft/azure/hooks/test_data_lake.py | 61 ++--
pyproject.toml | 1 -
task-sdk/docs/conf.py | 1 +
task-sdk/docs/lang-sdk-spec.rst | 83 +++---
uv.lock | 11 +-
37 files changed, 1653 insertions(+), 175 deletions(-)
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.test.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/DagFolderTree.tsx
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.test.ts
create mode 100644
airflow-core/src/airflow/ui/src/pages/DagsList/DagFolderTree/buildFolderTree.ts
copy airflow-core/src/airflow/ui/src/{layouts/Details/Gantt =>
pages/DagsList/DagFolderTree}/index.ts (91%)
copy airflow-core/src/airflow/ui/src/{context/colorMode/useColorMode.tsx =>
queries/useDagFolders.ts} (62%)
