This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch
jason/core-taskhandler-refactor/10-task-handler-bindings-reconcile
in repository https://gitbox.apache.org/repos/asf/airflow.git
discard fead9511c86 Align the artifact sweep with the other cleanup checks
discard 41cbf34f7b2 Warn only about bindings of Dags missing from the parse
result
discard d160b8effa4 Give the task handler race tests their own sessions
discard 1a6b94f82af Describe the binding reconcile and artifact sweep in
ADR-0013
discard e828af9d1e0 Delete task handler artifacts no stub task references
discard 4a7fa62ad70 Index task handler artifacts by last_probed_at
discard d3fb42dc3e5 Record probed artifacts and reconcile task handler bindings
discard d841ada39dd Sync the Java SDK supervisor schema for task handler
bindings
discard 9e567d8f017 Regenerate TS SDK supervisor types for task handler
bindings
discard 19ec1cd3778 Carry task handler bindings and probed answers on the
parse result
discard 34aabf6b8ef Test that a Dag file gets only named bundles without a
fallback
discard 861140b3d65 Describe how known artifacts are scoped per Dag-parsing
child
discard 200108188ce Send the Dag-parsing child its known task-handler artifacts
discard 60a79123e8b Sync the Java SDK supervisor schema for known artifacts
discard 2d039448a3c Regenerate TS SDK supervisor types for known artifacts
discard 0980cd55475 Add known task-handler artifacts to the Dag parse request
discard 251d7b390bd Let the coordinator registry list its task-handler bundles
discard b6456cd2002 Java SDK: answer a task handler parse with every handler
discard fc8f15cf343 Test the cache digest of classpath directories and non-zip
files
discard a9aef326691 Tidy names and mocks in the Java probe test
discard 29628e38a1a Test the Java SDK schema sync hook's snapshot copy
discard 0cdc9b70e53 Say why the Java coordinator rejects a JAR it cannot probe
discard cb039b71030 Java SDK: check the task handler reply against the
supervisor schema
discard 20b9e55e91e Java SDK: list a value schema's enum constants in a stable
order
discard 7102e9a262b Let the Java coordinator probe a JAR's task handlers
discard fcac3a4d03b Java SDK: register stub Dags' Java tasks as task handlers
discard be00f52d2f3 Java SDK: stamp a cache digest on the bundle JAR manifest
discard 4594bce8dc5 Java SDK: answer TaskHandlerParseRequest with registered
handlers
discard 4af2bac88f1 Java SDK: record handler parameters in generated task
classes
discard 0b6da6a1d46 Java SDK: sync the vendored supervisor schema from the
monorepo
discard dc5a0827e4d TS SDK: answer a task handler parse with every handler
discard 4648117e0b5 Patch the TS real probe test with decorators
discard a0841101fe0 Skip the TS real probe when node cannot report a version
discard 0132669e25a TS SDK: name the task handler lookup like its siblings
discard ec70c20c020 Let the Node coordinator probe a bundle's task handlers
discard 6919d5453eb TS SDK: answer TaskHandlerParseRequest from bundle.serve
discard 78995cf2159 State that a task handler parse answer depends only on the
artifact
discard fb64343c9d8 Go SDK: answer a task handler parse with every handler
discard acf207ebde4 Tidy names, mocks and cases in the Go probe and digest
tests
discard d6f462f00fa Move Go SDK comments back to the code they describe
discard 91307f3fa75 Resolve the probed bundle path and tighten the digest docs
discard e939ac6fe39 Go SDK: take the bundle digests from the staged executable
discard b69ecca8f6d Document the task handler parse for Language SDK authors
discard 84d7d5116f1 Read a Go bundle's stored cache digest without hashing it
discard 7b809900183 Let the Executable coordinator probe a bundle's task
handlers
discard 0ab82445ec9 Go SDK: record integrity and cache digests in bundle
metadata
discard a5c07046389 Go SDK: regenerate genmodels for the named_open binding
discard 90eb57225a3 Go SDK: clarify parse timeout and nested-null docs, test
anyOf
discard 47741c863b2 Go SDK: answer TaskHandlerParseRequest with registered
handlers
discard 6da5481daec Go SDK: describe task handler params with JSON Schema
discard 0d0c1a66281 Go SDK: regenerate genmodels from the supervisor schema
discard f6918d8ecdc Probe a task handler artifact without naming Dags
discard 645a024a749 Close the fake runtime's sockets when its reply is done
discard ba5d137a275 Name the runtime transport test helpers with verbs
discard 59011fc543f Patch with decorators in the runtime transport tests
discard 54ff4e1f46b Hang the fake runtimes without sleeping
discard 59e16139df4 Test that a start failing before the fork closes its
listeners
discard 6360ff7f697 Spec the mocks in the runtime transport and probe tests
discard 70357892782 Import ctypes at the top of the subprocess coordinator
module
discard 34ef146bcf9 Pass the binding mode in the task-handler probe tests
discard f2100f22358 Tidy the runtime transport and the coordinator lookup
discard 775c0dd16f9 Clean up a Lang-SDK runtime parse whose start fails after
the fork
discard 9c95909cd18 Do not exec a Lang-SDK runtime whose parent has already
exited
discard e41925c8a03 Kill what a Lang-SDK runtime leaves behind when it exits
discard beeaafc8745 Read a Lang-SDK runtime's frames without blocking the
parse loop
discard a43ef8a4751 Probe a Lang-SDK artifact for its task handlers
discard 0f5792c18e4 Keep a nested supervised child's standard streams open
discard 3a114b15c0c Make the Dag file processor base generic over its result
discard f5e6690653e Let a subprocess coordinator exec a task-handler parse
runtime
add 69ac8da87fb Bind task handlers only by position or by name
add 6c3f0eda7c9 Regenerate TS SDK supervisor types for the two binding
modes
add 48f58db7caf Describe the two task handler binding modes in the ADRs
add 7e229e82f86 Let a subprocess coordinator exec a task-handler parse
runtime
add 400d08b04d4 Make the Dag file processor base generic over its result
add b3d5f26dcdd Keep a nested supervised child's standard streams open
add 0ab3e095f6d Probe a Lang-SDK artifact for its task handlers
add 6a37b6d6166 Read a Lang-SDK runtime's frames without blocking the
parse loop
add a1782e45de9 Kill what a Lang-SDK runtime leaves behind when it exits
add 9856d8b0cc1 Do not exec a Lang-SDK runtime whose parent has already
exited
add c32ed26aa98 Clean up a Lang-SDK runtime parse whose start fails after
the fork
add 7a7f588a6bc Tidy the runtime transport and the coordinator lookup
add 25a918a97cb Pass the binding mode in the task-handler probe tests
add 8e0f64d7ee3 Import ctypes at the top of the subprocess coordinator
module
add 4acdded53e1 Spec the mocks in the runtime transport and probe tests
add b12cf16c735 Test that a start failing before the fork closes its
listeners
add c1861502159 Hang the fake runtimes without sleeping
add fde3390849a Patch with decorators in the runtime transport tests
add 1206141d55b Name the runtime transport test helpers with verbs
add f1ac0ab61d4 Close the fake runtime's sockets when its reply is done
add eb406929ad3 Probe a task handler artifact without naming Dags
add 7a8bab885bc Build the probe test's params without the dropped required
add faf5e976f0c Go SDK: regenerate genmodels from the supervisor schema
add 9a8627af5d3 Go SDK: describe task handler params with JSON Schema
add 795b97cb9cd Go SDK: answer TaskHandlerParseRequest with registered
handlers
add 9fa167c731c Go SDK: clarify parse timeout and nested-null docs, test
anyOf
add bfadfbe16b6 Go SDK: regenerate genmodels for the named_open binding
add 5dc64b4eef2 Go SDK: record integrity and cache digests in bundle
metadata
add 5f5c3397476 Let the Executable coordinator probe a bundle's task
handlers
add b1f99665ded Read a Go bundle's stored cache digest without hashing it
add 6ef66f078a5 Document the task handler parse for Language SDK authors
add 609c46c0038 Go SDK: take the bundle digests from the staged executable
add de6d01540b6 Resolve the probed bundle path and tighten the digest docs
add 5e38a0ba4ee Move Go SDK comments back to the code they describe
add dd9027215fb Tidy names, mocks and cases in the Go probe and digest
tests
add 8f74493f2be Go SDK: answer a task handler parse with every handler
add 54363fc4a9d State that a task handler parse answer depends only on the
artifact
add dc549f98eb3 Go SDK: declare every lone struct handler as named
add b1151dc3b23 Describe the two task handler bindings for Language SDK
authors
add d2a52241701 TS SDK: answer TaskHandlerParseRequest from bundle.serve
add 4d8d187fe0d Let the Node coordinator probe a bundle's task handlers
add 7897b301e67 TS SDK: name the task handler lookup like its siblings
add 9da53941c0a Skip the TS real probe when node cannot report a version
add 9eb64e250c9 Patch the TS real probe test with decorators
add e5f58fdb157 TS SDK: answer a task handler parse with every handler
add 23843ba1f96 TS SDK: declare task handlers without listing their params
add 580c06b6102 Java SDK: sync the vendored supervisor schema from the
monorepo
add acacd23a093 Java SDK: record handler parameters in generated task
classes
add e2f46fd1206 Java SDK: answer TaskHandlerParseRequest with registered
handlers
add 2ccbdd01eec Java SDK: stamp a cache digest on the bundle JAR manifest
add 4442332a32c Java SDK: register stub Dags' Java tasks as task handlers
add 89db1346eba Let the Java coordinator probe a JAR's task handlers
add e77108bd367 Java SDK: list a value schema's enum constants in a stable
order
add 87eb647e1e1 Java SDK: check the task handler reply against the
supervisor schema
add afe446dc211 Say why the Java coordinator rejects a JAR it cannot probe
add cd309bbf950 Test the Java SDK schema sync hook's snapshot copy
add efafb54df8e Tidy names and mocks in the Java probe test
add 328f528483e Test the cache digest of classpath directories and non-zip
files
add 455e01d8af2 Java SDK: answer a task handler parse with every handler
add 949c9198622 Java SDK: declare task handler params without required
add a34978ba239 Let the coordinator registry list its task-handler bundles
add 95c2eb0dbb0 Add known task-handler artifacts to the Dag parse request
add 8f6748298dc Regenerate TS SDK supervisor types for known artifacts
add d1d52c0615d Sync the Java SDK supervisor schema for known artifacts
add 321c5abe1d1 Send the Dag-parsing child its known task-handler artifacts
add cf53781c583 Describe how known artifacts are scoped per Dag-parsing
child
add 115f7f79e13 Test that a Dag file gets only named bundles without a
fallback
add 7b0b329177a Build the known-artifact test params without required
add fb5c67bfa0e Carry task handler bindings and probed answers on the
parse result
add 33bff421e38 Regenerate TS SDK supervisor types for task handler
bindings
add cc022cbc46b Sync the Java SDK supervisor schema for task handler
bindings
add 814018206e9 Record probed artifacts and reconcile task handler bindings
add d562cfdb75b Index task handler artifacts by last_probed_at
add 1db081edd9e Delete task handler artifacts no stub task references
add 6bb87f99b9f Describe the binding reconcile and artifact sweep in
ADR-0013
add 01945cca37b Give the task handler race tests their own sessions
add 260880434ed Warn only about bindings of Dags missing from the parse
result
add 00b47d446d7 Align the artifact sweep with the other cleanup checks
add a937384f838 Build the binding and probed-artifact test params without
required
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (fead9511c86)
\
N -- N -- N
refs/heads/jason/core-taskhandler-refactor/10-task-handler-bindings-reconcile
(a937384f838)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../lang-sdk/0011-mixed-language-dag-processing.md | 14 +++++---
.../adr/lang-sdk/0012-lang-sdk-parse-protocol.md | 20 ++++++-----
.../0013-persisted-task-handler-bindings.md | 5 ++-
.../src/airflow/dag_processing/processor.py | 28 +++++++++------
.../tests/unit/dag_processing/test_collection.py | 4 +--
.../tests/unit/dag_processing/test_manager.py | 7 ++--
.../tests/unit/dag_processing/test_processor.py | 41 +++++++++++++---------
.../dag_processing/test_task_handler_processor.py | 4 +--
.../test_task_handler_processor_go.py | 18 ++++++----
.../test_task_handler_processor_java.py | 4 +--
.../test_task_handler_processor_ts.py | 19 +++-------
contributing-docs/30_new_language_sdk.rst | 28 +++++++++------
go-sdk/pkg/binding/declaration.go | 23 +++++-------
go-sdk/pkg/binding/declaration_test.go | 18 +++++-----
go-sdk/pkg/execution/genmodels/models.gen.go | 9 ++---
go-sdk/pkg/execution/integration_test.go | 5 +--
go-sdk/pkg/execution/task_handler_parse_test.go | 8 ++---
java-sdk/sdk/schema/schema.json | 28 +++++++--------
.../airflow/sdk/execution/TaskHandlerParse.kt | 11 +++---
.../airflow/sdk/execution/TaskHandlerParseTest.kt | 7 ++--
task-sdk/docs/lang-sdk-spec.rst | 7 ++--
.../airflow/sdk/execution_time/schema/schema.json | 28 +++++++--------
.../execution_time/schema/test_migrator.py | 12 +++----
ts-sdk/README.md | 2 +-
ts-sdk/src/coordinator/task-handler-parse.ts | 28 +++------------
ts-sdk/src/generated/supervisor.ts | 12 +++----
ts-sdk/tests/coordinator/integration.test.ts | 2 +-
.../tests/coordinator/task-handler-parse.test.ts | 24 ++++---------
28 files changed, 191 insertions(+), 225 deletions(-)
