This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/08-java-sdk-task-handler-parse
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard 58e6a8ccbec Test the cache digest of classpath directories and non-zip
files
discard d85091ffee9 Tidy names and mocks in the Java probe test
discard a90f0d23eb4 Test the Java SDK schema sync hook's snapshot copy
discard f652f2dfd6c Say why the Java coordinator rejects a JAR it cannot probe
discard 73bbb3f87b9 Java SDK: check the task handler reply against the
supervisor schema
discard 1a0949a437e Java SDK: list a value schema's enum constants in a stable
order
discard 92969a41f55 Let the Java coordinator probe a JAR's task handlers
discard 9b19d627c2e Java SDK: register stub Dags' Java tasks as task handlers
discard 1958bc7f37c Java SDK: stamp a cache digest on the bundle JAR manifest
discard dfa2957b75a Java SDK: answer TaskHandlerParseRequest with registered
handlers
discard fb412512b3a Java SDK: record handler parameters in generated task
classes
discard 5dd490c1dc3 Java SDK: sync the vendored supervisor schema from the
monorepo
discard 5b770b6fcc0 Patch the TS real probe test with decorators
discard fa5a8f674d9 Skip the TS real probe when node cannot report a version
discard a877875d7d8 TS SDK: name the task handler lookup like its siblings
discard 5807835c8ec Let the Node coordinator probe a bundle's task handlers
discard 1ab24d29d70 TS SDK: answer TaskHandlerParseRequest from bundle.serve
discard f17015e9d3f Tidy names, mocks and cases in the Go probe and digest
tests
discard e3f963f2b1f Move Go SDK comments back to the code they describe
discard 1209ecc7229 Resolve the probed bundle path and tighten the digest docs
discard 2123669a919 Go SDK: take the bundle digests from the staged executable
discard 0aba605080f Document the task handler parse for Language SDK authors
discard 19e50af861d Read a Go bundle's stored cache digest without hashing it
discard baa914f5652 Let the Executable coordinator probe a bundle's task
handlers
discard b3a4ccdc876 Go SDK: record integrity and cache digests in bundle
metadata
discard b863f5e7578 Go SDK: regenerate genmodels for the named_open binding
discard 69573dcbd0f Go SDK: clarify parse timeout and nested-null docs, test
anyOf
discard abced658463 Go SDK: answer TaskHandlerParseRequest with registered
handlers
discard ff8a56316d5 Go SDK: describe task handler params with JSON Schema
discard 35da766903a Go SDK: regenerate genmodels from the supervisor schema
discard 11b1aa6a6ce Close the fake runtime's sockets when its reply is done
discard 2e1afe349eb Name the runtime transport test helpers with verbs
discard 86a2133932f Patch with decorators in the runtime transport tests
discard 62e2855bd7c Hang the fake runtimes without sleeping
discard 8c695ad30c0 Test that a start failing before the fork closes its
listeners
discard 31eaa27e543 Spec the mocks in the runtime transport and probe tests
discard 431975d5092 Import ctypes at the top of the subprocess coordinator
module
discard 829768b7f98 Pass the binding mode in the task-handler probe tests
discard 48ec2d68960 Tidy the runtime transport and the coordinator lookup
discard d222f346180 Clean up a Lang-SDK runtime parse whose start fails after
the fork
discard 68f8b6a9661 Do not exec a Lang-SDK runtime whose parent has already
exited
discard 574fdc0598b Kill what a Lang-SDK runtime leaves behind when it exits
discard e32271e6350 Read a Lang-SDK runtime's frames without blocking the
parse loop
discard d5781c3bfe6 Probe a Lang-SDK artifact for its task handlers
discard 8554e8f1ca5 Keep a nested supervised child's standard streams open
discard ad24f837beb Make the Dag file processor base generic over its result
discard b0c07d0bf37 Let a subprocess coordinator exec a task-handler parse
runtime
discard 6b23ddc4205 Add the named_open binding to the Lang-SDK ADRs
discard dc331d7eb3b Regenerate TS SDK supervisor types for the named_open
binding
discard d4d103f9217 Add a named_open binding for non-exhaustive task-handler
params
discard 1360aa4af35 Describe the binding-mode parse check in ADR-0011 and
ADR-0013
discard 847e4b5b456 Describe the task-handler binding mode in ADR-0012
discard 088360ba388 Regenerate TS SDK supervisor types for the binding mode
discard d4bdedae978 Add a binding mode to task-handler declarations
discard 6fd00cdd521 Document the version rules for a new supervisor schema body
discard 95285b73fb3 Regenerate TS SDK supervisor types for task-handler
messages
discard 0cb7a22b023 Add task-handler parse messages and their parse-channel
union
discard 72f869f68c7 Test closing a Dag file processor that has no parse log
file
discard 52a6e52b0aa Move the Dag file processor's shared plumbing into a base
class
add 190ebe44ac9 Cache an artifact's task handlers on its row, not on each
binding
add a25eb7bbae6 Move the handler declarations to the artifact in
ADR-0013's SQL
add ad99d4bb954 Let a task handler artifact store no cache digest
add 0cafad2a563 Allow a NULL cache digest in ADR-0013's artifact table SQL
add d301fee5d6c Move the Dag file processor's shared plumbing into a base
class
add e9b1ba3f249 Test closing a Dag file processor that has no parse log
file
add afcd4e6eb31 Add task-handler parse messages and their parse-channel
union
add 4e462352188 Regenerate TS SDK supervisor types for task-handler
messages
add c08b607a132 Document the version rules for a new supervisor schema body
add ffbd6892edb Add a binding mode to task-handler declarations
add b6137fe719e Regenerate TS SDK supervisor types for the binding mode
add e37df9a8e18 Describe the task-handler binding mode in ADR-0012
add 5b2d32caa7a Describe the binding-mode parse check in ADR-0011 and
ADR-0013
add 48c7217ec34 Add a named_open binding for non-exhaustive task-handler
params
add 267cc098be0 Regenerate TS SDK supervisor types for the named_open
binding
add b1b645435cd Add the named_open binding to the Lang-SDK ADRs
add 84460f71c1f Ask a task handler runtime for every handler it registers
add 68e0cdddd0d Regenerate TS SDK supervisor types for the probe without
dag_ids
add 020bdcc03ca Describe the all-handlers probe in ADR-0011 and ADR-0012
add 0e65496cbc6 Let a subprocess coordinator exec a task-handler parse
runtime
add b98c1882b65 Make the Dag file processor base generic over its result
add 6da772bd023 Keep a nested supervised child's standard streams open
add 6a77bfb1bd9 Probe a Lang-SDK artifact for its task handlers
add b4b98407ce5 Read a Lang-SDK runtime's frames without blocking the
parse loop
add 8c7e2888e59 Kill what a Lang-SDK runtime leaves behind when it exits
add de45ceba937 Do not exec a Lang-SDK runtime whose parent has already
exited
add d43b88f1a1a Clean up a Lang-SDK runtime parse whose start fails after
the fork
add 45580954214 Tidy the runtime transport and the coordinator lookup
add 8323e15e104 Pass the binding mode in the task-handler probe tests
add 173ac921a5d Import ctypes at the top of the subprocess coordinator
module
add cb8e7b92e57 Spec the mocks in the runtime transport and probe tests
add 86b6a606d85 Test that a start failing before the fork closes its
listeners
add 274b750192c Hang the fake runtimes without sleeping
add e04b994ae7c Patch with decorators in the runtime transport tests
add acba2e5ca6a Name the runtime transport test helpers with verbs
add 12301131f0a Close the fake runtime's sockets when its reply is done
add 0b77a65e354 Probe a task handler artifact without naming Dags
add 32941deefd2 Go SDK: regenerate genmodels from the supervisor schema
add 052cb8e3dd1 Go SDK: describe task handler params with JSON Schema
add e6307f893f6 Go SDK: answer TaskHandlerParseRequest with registered
handlers
add 46c30b41923 Go SDK: clarify parse timeout and nested-null docs, test
anyOf
add 1250abbcd98 Go SDK: regenerate genmodels for the named_open binding
add a859350b5d6 Go SDK: record integrity and cache digests in bundle
metadata
add aa05229820e Let the Executable coordinator probe a bundle's task
handlers
add f42aa98608c Read a Go bundle's stored cache digest without hashing it
add c437221182e Document the task handler parse for Language SDK authors
add d8613040957 Go SDK: take the bundle digests from the staged executable
add 5c994cda4a0 Resolve the probed bundle path and tighten the digest docs
add 583e9997819 Move Go SDK comments back to the code they describe
add 2367d83c173 Tidy names, mocks and cases in the Go probe and digest
tests
add 9ac136b7122 Go SDK: answer a task handler parse with every handler
add 421cb2314a9 State that a task handler parse answer depends only on the
artifact
add 5c0b389a228 TS SDK: answer TaskHandlerParseRequest from bundle.serve
add 989055e7a98 Let the Node coordinator probe a bundle's task handlers
add bce1697092f TS SDK: name the task handler lookup like its siblings
add bed59a8db61 Skip the TS real probe when node cannot report a version
add 895c5d9d250 Patch the TS real probe test with decorators
add 3d1332e644b TS SDK: answer a task handler parse with every handler
add fa3d7051ced Java SDK: sync the vendored supervisor schema from the
monorepo
add fc8d87a76aa Java SDK: record handler parameters in generated task
classes
add 6a335956646 Java SDK: answer TaskHandlerParseRequest with registered
handlers
add 6da54887166 Java SDK: stamp a cache digest on the bundle JAR manifest
add 90da59317fa Java SDK: register stub Dags' Java tasks as task handlers
add 62006b90089 Let the Java coordinator probe a JAR's task handlers
add d9b4faf50c9 Java SDK: list a value schema's enum constants in a stable
order
add c57160aca84 Java SDK: check the task handler reply against the
supervisor schema
add 1804b24ccf2 Say why the Java coordinator rejects a JAR it cannot probe
add c7fe9d0b1f9 Test the Java SDK schema sync hook's snapshot copy
add da11ffc9285 Tidy names and mocks in the Java probe test
add b55944b8182 Test the cache digest of classpath directories and non-zip
files
add 03405f9b3e3 Java SDK: answer a task handler parse with every handler
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (58e6a8ccbec)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/08-java-sdk-task-handler-parse
(03405f9b3e3)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../lang-sdk/0011-mixed-language-dag-processing.md | 20 +++----
.../adr/lang-sdk/0012-lang-sdk-parse-protocol.md | 12 ++---
.../0013-persisted-task-handler-bindings.md | 14 +++--
.../src/airflow/dag_processing/processor.py | 10 ++--
.../dag_processing/task_handler_processor.py | 10 ++--
.../0141_3_4_0_add_lang_sdk_task_handler_tables.py | 5 +-
.../src/airflow/models/lang_sdk_task_handler.py | 15 ++++--
.../unit/dag_processing/test_lang_sdk_runtime.py | 10 ++--
.../dag_processing/test_task_handler_processor.py | 10 ++--
.../test_task_handler_processor_go.py | 8 ++-
.../test_task_handler_processor_java.py | 9 +---
.../test_task_handler_processor_ts.py | 1 -
.../unit/models/test_lang_sdk_task_handler.py | 63 +++++++++++++++-------
contributing-docs/30_new_language_sdk.rst | 12 ++---
go-sdk/README.md | 4 +-
.../pkg/execution/genmodels/discriminators.gen.go | 4 ++
go-sdk/pkg/execution/genmodels/models.gen.go | 21 +++++---
go-sdk/pkg/execution/integration_test.go | 1 -
go-sdk/pkg/execution/messages_test.go | 4 +-
go-sdk/pkg/execution/server.go | 4 +-
go-sdk/pkg/execution/task_handler_parse.go | 12 +----
go-sdk/pkg/execution/task_handler_parse_test.go | 24 +++++----
java-sdk/README.md | 3 +-
java-sdk/sdk/schema/schema.json | 12 +----
.../main/kotlin/org/apache/airflow/sdk/Server.kt | 5 +-
.../airflow/sdk/execution/TaskHandlerParse.kt | 22 +++-----
.../kotlin/org/apache/airflow/sdk/ServerTest.kt | 9 ++--
.../airflow/sdk/execution/TaskHandlerParseTest.kt | 21 ++++----
task-sdk/docs/lang-sdk-spec.rst | 11 ++--
.../airflow/sdk/execution_time/schema/schema.json | 12 +----
ts-sdk/src/coordinator/runtime.ts | 5 +-
ts-sdk/src/coordinator/task-handler-parse.ts | 30 +++--------
ts-sdk/src/generated/supervisor.ts | 8 +--
ts-sdk/tests/coordinator/integration.test.ts | 1 -
ts-sdk/tests/coordinator/protocol.test.ts | 1 -
.../tests/coordinator/task-handler-parse.test.ts | 34 ++++--------
36 files changed, 202 insertions(+), 245 deletions(-)
