This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch jason/lang-sdk-e2e/04-java-dag-importer
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 47500af0539 Keep the Java parse_dag test from changing the pytest
worker
discard 5586c0a9781 Stop promising that Java dependency JARs are not parsed
discard e206f8cfda6 Reject a Java Dag JAR whose SDK cannot answer a parse
discard 08cc9a263ff Run the JAR a native Java Dag was parsed from
discard d17430948f3 Report why a Java Dag JAR cannot be read
discard ecc6cc2ac64 Pass the task's Dag file to the coordinator's command
builder
discard 1a7c6d45a51 Run a task on its run's version when dag_bundle_name names
its bundle
discard 7ad9b106dcd Document native Java Dag parsing
discard 24e049de385 Name the JAR manifest attributes the Java coordinator reads
discard 490ea209115 Test the Java Dag source placeholder for a JAR without a
manifest
discard d5ad8e33dbf Treat an unreadable JAR like one without a manifest
discard be296473c5b Parse native Java Dags with their coordinator
discard 7e4ae847f12 Embed the Dag source in Java bundle JARs
discard 4673b378e83 Build the JVM command that parses a Dag JAR
discard a88ed9d8bbe Read JAR manifests per the JAR specification
discard bd89a180966 List the tasks of a native Lang-SDK Dag
discard 0d5cd08ff4c Bag native Lang-SDK Dags in core, only when asked
discard 243a2e3fbf0 Let a Dag file processor run without an API client
discard 3ff6be22d01 Parse coordinator-claimed Dag files with their runtime in
a Dag bag
discard b2ce95c7167 Move the client-less Lang-SDK parse out of this change
discard 5b9cd9c3664 Note in ADR-0010 that the manager routes claimed files for
now
discard 2a3e14ba73e Clear import errors after the native-file Dag bag sync test
discard 5f7f31b6776 Apply the multi-team task checks to Lang-SDK Dags
discard 6dceaa7f7b4 Spec the mocks of a hand-built Lang-SDK processor in tests
discard 9d2b068fc24 Name the setting in a Lang-SDK parse timeout error
discard 5e9b6c34007 Kill what a Lang-SDK runtime leaves in its process group
discard 6ec3b9d872a Match the schema error's JSON path in a Lang-SDK parse test
discard b70d91a5584 Build each bundle's Dag importer registry before freezing
the heap
discard f5134f7ca3a Fail a native Dag's task on a Python worker without retries
discard 6e7fbd6958e Find the claiming coordinator in the Task SDK and wrap it
in core
discard 273be37c009 Return a Dag importer from every SubprocessCoordinator call
discard 11346702645 Start each Dag file processor with explicit arguments
discard 0f933aa2b05 Drop callbacks for Lang-SDK Dag files before queuing the
file
discard c94553e7f90 Report a native Dag's task that reaches a Python worker
discard 97b61d1076d Report a Lang-SDK parse past its import timeout as an
import error
discard 8e0e58455bd Parse coordinator-claimed Dag files with their runtime in
the Dag processor
discard e3fc4402ad7 Drop serves_bundle so get_parsed_bundles is the only answer
discard 90e99e3cd7f Check Dag file claims per bundle on the parse path
discard 16e4118bb3a Import the coordinator manager at the top of the importer
base
discard 7bf33a97660 Let a coordinator parse the Dag files of the bundles it
serves
discard ad14011af01 Note in the parsing ADR that runtimes may omit
config-backed fields
discard 0f3006a6769 Check SDK Dags with the Dag processor's validation in
conformance
discard aefc2d92e62 Reject malformed task entries in a serialized Dag
discard d996d4f88ad Fill a serialized Dag's unset settings from the Airflow
config
discard 1153e7eecc9 Check that a serialized Dag can be stored and loaded
discard 3ff4e27860a Move the Dag file processor's shared plumbing into a base
class
discard e5ae872c9a1 Pin the empty task id case in the Dag cycle detection tests
discard 57e89c0a676 Share Dag cycle detection between the Task SDK and core
add 64f9f3ac784 Update issue templates now that Airflow 2 has reached end
of life (#74056)
add 6330b4de9c5 Go SDK: reject a second TaskSpec or Inputs when dag.Task
applies it (#74057)
add 737097a0dfe Share Dag cycle detection between the Task SDK and core
add 20603491841 Pin the empty task id case in the Dag cycle detection tests
add 0649a43f468 Move the Dag file processor's shared plumbing into a base
class
add adf872d0ac6 Check that a serialized Dag can be stored and loaded
add c1226408285 Fill a serialized Dag's unset settings from the Airflow
config
add d8cf5805ec7 Reject malformed task entries in a serialized Dag
add 76389a9ffa2 Check SDK Dags with the Dag processor's validation in
conformance
add e5d1a71ca9c Note in the parsing ADR that runtimes may omit
config-backed fields
add 464c9273c0c Serialize the SDK Dag in tests that write a dag_maker Dag
add f9fd8e01be2 Sync the Java SDK copy of the Dag schema
add 60d7af4cad9 Let a coordinator parse the Dag files of the bundles it
serves
add 9b98c04bd91 Import the coordinator manager at the top of the importer
base
add 75c77d508be Check Dag file claims per bundle on the parse path
add 52b88ba0c85 Drop serves_bundle so get_parsed_bundles is the only answer
add 30187220d54 Say dag_bundle_name is only needed for two coordinators of
one class
add 9ef4d490013 Parse coordinator-claimed Dag files with their runtime in
the Dag processor
add 50981761a38 Report a Lang-SDK parse past its import timeout as an
import error
add 7c38262692d Report a native Dag's task that reaches a Python worker
add 0cb6bf43a66 Drop callbacks for Lang-SDK Dag files before queuing the
file
add ed8cbad9fa0 Start each Dag file processor with explicit arguments
add 4f54283d0de Return a Dag importer from every SubprocessCoordinator call
add fd2a897053d Find the claiming coordinator in the Task SDK and wrap it
in core
add 001b6d90f29 Fail a native Dag's task on a Python worker without retries
add 3c0b4ba6181 Build each bundle's Dag importer registry before freezing
the heap
add 280fb07887a Match the schema error's JSON path in a Lang-SDK parse test
add f632602f806 Kill what a Lang-SDK runtime leaves in its process group
add 56c6820d7c3 Name the setting in a Lang-SDK parse timeout error
add 8b91affcf79 Spec the mocks of a hand-built Lang-SDK processor in tests
add 55311fe73c1 Apply the multi-team task checks to Lang-SDK Dags
add 04706bdc0b5 Clear import errors after the native-file Dag bag sync test
add 2ee5ead4c9a Note in ADR-0010 that the manager routes claimed files for
now
add 57a417b164d Move the client-less Lang-SDK parse out of this change
add e072db64742 Wait for the killed leftover in the Lang-SDK processor test
add c53c8c723e5 Parse coordinator-claimed Dag files with their runtime in
a Dag bag
add cab27a25c52 Let a Dag file processor run without an API client
add a8797750305 Bag native Lang-SDK Dags in core, only when asked
add fd0a742a2d7 List the tasks of a native Lang-SDK Dag
add 96f2076b96d Ignore the bagged Dag union in the external task sensor
test
add bd060ceb11b Read JAR manifests per the JAR specification
add 02bc2bc0794 Build the JVM command that parses a Dag JAR
add a7be34b19ff Embed the Dag source in Java bundle JARs
add 2eb08b4308c Parse native Java Dags with their coordinator
add 0b55a3dd2a6 Treat an unreadable JAR like one without a manifest
add 2ca56a5effe Test the Java Dag source placeholder for a JAR without a
manifest
add bac857dec40 Name the JAR manifest attributes the Java coordinator reads
add 97f60a3322e Document native Java Dag parsing
add c179527fedc Run a task on its run's version when dag_bundle_name names
its bundle
add ac38e820b06 Pass the task's Dag file to the coordinator's command
builder
add 1aac2d02868 Report why a Java Dag JAR cannot be read
add ee9521355b1 Run the JAR a native Java Dag was parsed from
add f3f6bea154b Reject a Java Dag JAR whose SDK cannot answer a parse
add 1a6e3ea86e1 Stop promising that Java dependency JARs are not parsed
add b045547d080 Keep the Java parse_dag test from changing the pytest
worker
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (47500af0539)
\
N -- N -- N refs/heads/jason/lang-sdk-e2e/04-java-dag-importer
(b045547d080)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/ISSUE_TEMPLATE/1-airflow_bug_report.yml | 7 ++-
.github/ISSUE_TEMPLATE/2-feature_request.yml | 3 +-
.../authoring-and-scheduling/language-sdks/go.rst | 18 +++----
.../language-sdks/java.rst | 21 +++++----
.../language-sdks/typescript.rst | 21 +++++----
.../src/airflow/config_templates/config.yml | 8 ++--
.../unit/dag_processing/test_lang_sdk_processor.py | 12 ++++-
airflow-core/tests/unit/models/test_dagrun.py | 6 ++-
.../tests/unit/models/test_serialized_dag.py | 4 +-
go-sdk/README.md | 4 +-
go-sdk/adr/0008-native-dag-interface.md | 2 +-
go-sdk/airflow/dag.go | 42 ++++++++++-------
go-sdk/airflow/dag_test.go | 17 ++++++-
go-sdk/airflow/inputs.go | 31 ++++++------
go-sdk/airflow/inputs_test.go | 55 ++++++++++++++++++++--
go-sdk/airflow/task_option.go | 25 +++++++---
go-sdk/airflow/trigger_dag_run_test.go | 4 ++
java-sdk/README.md | 3 ++
java-sdk/sdk/schema/dag-schema.json | 13 ++++-
.../standard/sensors/test_external_task_sensor.py | 2 +-
ts-sdk/README.md | 7 +--
ts-sdk/example/README.md | 3 ++
22 files changed, 213 insertions(+), 95 deletions(-)
