This is an automated email from the ASF dual-hosted git repository.
jason810496 pushed a change to branch jason/lang-sdk-e2e/04-java-dag-importer
in repository https://gitbox.apache.org/repos/asf/airflow.git
omit a80facc76d2 Name the JAR manifest attributes the Java coordinator reads
omit 178e4681803 Test the Java Dag source placeholder for a JAR without a
manifest
omit 6ff2da485c9 Treat an unreadable JAR like one without a manifest
omit 12fa0f239a5 Document native Java Dag parsing
omit ca7dc2a707d Parse native Java Dags with their coordinator
omit 52b381cc1c2 Embed the Dag source in Java bundle JARs
omit b5b72f5ae4d Build the JVM command that parses a Dag JAR
omit 8159ac365b0 Read JAR manifests per the JAR specification
omit 8e8c9820d22 Start a coordinator's runtime from a fresh interpreter on
macOS
omit 75761970850 Keep a bundle's importers when the coordinator config is
broken
omit ac74a126a27 Store only native Dags that a Dag bag can rebuild
omit 0359eab0e99 Start a Lang-SDK parse runtime with default signal handling
omit 2db7f8f64ad Keep the first parse result from a Lang-SDK runtime
omit f743cc4c9e4 Run the Lang-SDK runtime in its own process group
omit c7007e70d66 Report an invalid frame from a Lang-SDK runtime as an
import error
omit 63af72aaa72 Name the missing task when a native Dag's edge points
nowhere
omit cd7689b2fdf Keep a native Dag's timezone when rebuilding it
omit aeb390767e1 Reject a native Dag whose task graph has a cycle
omit a0dfcd23390 Signal a Lang-SDK runtime's whole process tree
omit c39a58eaa2f Kill the Lang-SDK runtime when its parse is interrupted
omit 2eee8648c55 Parse coordinator-claimed Dag files with their runtime
omit 1fe5309883a Import native Dags through their coordinator
omit ecc19efc4b6 Parse a coordinator's Dag file outside the Dag processor
omit 23903ec049f Parse a Dag file with its coordinator's runtime
omit 7fb00390d88 Rebuild a Lang-SDK Dag as an airflow.sdk.DAG
omit ae498dcc9ec Keep a stub task's preset arg bindings when serializing it
omit 396263ed7f0 Find the coordinators that parse a Dag bundle
omit 40bf9b36a7c Harden claimed-file discovery errors and archive importers
omit 9ff9d0a6da5 Tidy the importer routing helpers and their tests
omit 1b150318965 Keep claimed files queued when their importer cannot list
them
omit a7a81df9f58 Claim Dag files by extension and keep .py, .pyc and .zip
legacy
omit b051d590807 Read a claimed file's Dag source through its importer
omit cf95650ca17 Queue and track files that a Task SDK importer claims
omit 12a840b2d22 Parse files that a Task SDK importer claims
omit a481344223b Reject empty subprocess coordinator artifact roots
omit 0aec44e8889 Reject concurrent subprocess coordinator execution
consistently
omit ce617c8474b Make task bundle access dependencies explicit
omit a59132f973f Preserve custom subprocess coordinator compatibility
omit 14d5448c276 Read the Dag bundle configuration through a single entry
point
omit ed24a7cdcdf Resolve coordinator Dag bundles without importing the task
runner
omit 5fa4098e409 Make JavaCoordinator main_class optional in co-located mode
omit da37c21686b Harden subprocess coordinator artifact bundle resolution
omit b87d2907a24 Document dag_bundle_name option for language-SDK
coordinators
omit 68693340f67 Add dag_bundle_name option to subprocess coordinators
add 0566d6cd5ef Add OpenSandbox backend for sandbox tools (#71676)
add 93978958ae1 Fix example_neptune_analytics by pinning an explicit
region (#73851)
add d6c4c71979e Bump prettier (#73857)
add c8ca02e4158 Update changelog to clarify new LLMBranchOperator
"branches" feature (#73850)
add ad3e67fccd2 Add Modal provider with `ModalHook` and connection type
(#73418)
add ce614baf9b0 Add Strands Agents support for `common.ai` toolsets
(#73587)
add dbac03d87dd Fix KubernetesExecutor not queuing tasks on Airflow 3.0.x
(#73764)
add b3bcb388742 Recognize grouped PR references in provider changelog
tooling (#73855)
add ae1e8385060 Let the sandbox toolset attach to a sandbox another task
provisioned (#73559)
add c1b878ad22c Exclude SQLAlchemy 2.1.0 (#73865)
add 10221c300eb Read the AWS task logs once more after the fetcher is
stopped (#73211)
add 86d5297ca4b Add dag_bundle_name option to subprocess coordinators
add 1a65f90325e Document dag_bundle_name option for language-SDK
coordinators
add fec090c4e62 Harden subprocess coordinator artifact bundle resolution
add aa32014083c Make JavaCoordinator main_class optional in co-located mode
add 6523b0581c3 Resolve coordinator Dag bundles without importing the task
runner
add ac1f620daec Read the Dag bundle configuration through a single entry
point
add 13b4a38ac26 Preserve custom subprocess coordinator compatibility
add 5e210e8f8d6 Make task bundle access dependencies explicit
add f79d7502c2c Reject concurrent subprocess coordinator execution
consistently
add 6dc4ee7384a Reject empty subprocess coordinator artifact roots
add b238a022c47 Reject blank subprocess coordinator artifact roots
add 61af924d65c Parse files that a Task SDK importer claims
add 687df08ff40 Queue and track files that a Task SDK importer claims
add 453d40ca237 Read a claimed file's Dag source through its importer
add 334f4c72a54 Claim Dag files by extension and keep .py, .pyc and .zip
legacy
add 844bb62d542 Keep claimed files queued when their importer cannot list
them
add de6df7075e7 Tidy the importer routing helpers and their tests
add a73513df7c1 Harden claimed-file discovery errors and archive importers
add e1228b0c91c Find the coordinators that parse a Dag bundle
add 942ff07bdc9 Keep a stub task's preset arg bindings when serializing it
add a72ebd5001a Rebuild a Lang-SDK Dag as an airflow.sdk.DAG
add ed655b8f92b Parse a Dag file with its coordinator's runtime
add d5b37f466d5 Parse a coordinator's Dag file outside the Dag processor
add b1a9bf3370e Import native Dags through their coordinator
add f6dd1237c6f Parse coordinator-claimed Dag files with their runtime
add 3492fac6413 Kill the Lang-SDK runtime when its parse is interrupted
add d654c84fbbf Signal a Lang-SDK runtime's whole process tree
add d7b07dfcefb Reject a native Dag whose task graph has a cycle
add 3718d8b9542 Keep a native Dag's timezone when rebuilding it
add 74c4a3aa974 Name the missing task when a native Dag's edge points
nowhere
add d8dc07f24fc Report an invalid frame from a Lang-SDK runtime as an
import error
add fb045a27537 Run the Lang-SDK runtime in its own process group
add 049df3e3ec1 Keep the first parse result from a Lang-SDK runtime
add e51166bbcdf Start a Lang-SDK parse runtime with default signal handling
add 8d6ff5d1198 Store only native Dags that a Dag bag can rebuild
add aaff5b1aaf3 Keep a bundle's importers when the coordinator config is
broken
add 0080efb1506 Start a coordinator's runtime from a fresh interpreter on
macOS
add d6216311ee1 Read JAR manifests per the JAR specification
add d24b1ceb489 Build the JVM command that parses a Dag JAR
add cd821a07b5a Embed the Dag source in Java bundle JARs
add daa85b9dd16 Parse native Java Dags with their coordinator
add 9c26a792936 Document native Java Dag parsing
add e47fd372efd Treat an unreadable JAR like one without a manifest
add 21ec98dcb45 Test the Java Dag source placeholder for a JAR without a
manifest
add d62246f6e49 Name the JAR manifest attributes the Java coordinator reads
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (a80facc76d2)
\
N -- N -- N refs/heads/jason/lang-sdk-e2e/04-java-dag-importer
(d62246f6e49)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.github/ISSUE_TEMPLATE/1-airflow_bug_report.yml | 1 +
.github/boring-cyborg.yml | 3 +
airflow-core/docs/extra-packages-ref.rst | 2 +
airflow-core/pyproject.toml | 2 +-
.../src/airflow/dag_processing/importer_routing.py | 7 +-
airflow-core/src/airflow/models/dagcode.py | 29 +-
airflow-core/tests/unit/models/test_dagcode.py | 23 +-
dev/breeze/doc/images/output_build-docs.svg | 4 +-
dev/breeze/doc/images/output_build-docs.txt | 2 +-
...tput_release-management_add-back-references.svg | 4 +-
...tput_release-management_add-back-references.txt | 2 +-
...elease-management_classify-provider-changes.svg | 4 +-
...elease-management_classify-provider-changes.txt | 2 +-
...management_generate-issue-content-providers.svg | 4 +-
...management_generate-issue-content-providers.txt | 2 +-
...ease-management_generate-providers-metadata.svg | 10 +-
...ease-management_generate-providers-metadata.txt | 2 +-
...e-management_prepare-provider-distributions.svg | 4 +-
...e-management_prepare-provider-distributions.txt | 2 +-
...e-management_prepare-provider-documentation.svg | 4 +-
...e-management_prepare-provider-documentation.txt | 2 +-
.../output_release-management_publish-docs.svg | 4 +-
.../output_release-management_publish-docs.txt | 2 +-
...output_sbom_generate-providers-requirements.svg | 2 +-
...output_sbom_generate-providers-requirements.txt | 2 +-
.../images/output_workflow-run_publish-docs.svg | 4 +-
.../images/output_workflow-run_publish-docs.txt | 2 +-
dev/breeze/src/airflow_breeze/global_constants.py | 2 +-
.../prepare_providers/provider_documentation.py | 11 +-
dev/breeze/tests/test_provider_documentation.py | 48 ++
devel-common/pyproject.toml | 2 +-
docs/spelling_wordlist.txt | 4 +
providers/amazon/docs/index.rst | 2 +-
providers/amazon/pyproject.toml | 2 +-
.../providers/amazon/aws/utils/task_log_fetcher.py | 38 +-
.../system/amazon/aws/example_neptune_analytics.py | 20 +-
.../unit/amazon/aws/utils/test_task_log_fetcher.py | 36 +-
providers/apache/hive/docs/index.rst | 2 +-
providers/apache/hive/pyproject.toml | 2 +-
providers/apache/impala/docs/index.rst | 8 +-
providers/apache/impala/pyproject.toml | 2 +-
.../kubernetes/executors/kubernetes_executor.py | 12 +-
.../executors/test_kubernetes_executor.py | 32 +
providers/common/ai/README.rst | 47 +-
providers/common/ai/docs/changelog.rst | 2 +-
providers/common/ai/docs/frameworks/index.rst | 119 ++++
providers/common/ai/docs/frameworks/strands.rst | 154 +++++
providers/common/ai/docs/index.rst | 48 +-
providers/common/ai/docs/installation.rst | 7 +-
providers/common/ai/docs/sandbox/backends.rst | 126 +++-
providers/common/ai/docs/sandbox/configuration.rst | 205 ++++--
providers/common/ai/docs/sandbox/index.rst | 86 ++-
providers/common/ai/docs/tool_approval.rst | 9 +-
providers/common/ai/provider.yaml | 5 +-
providers/common/ai/pyproject.toml | 4 +-
.../ai/example_dags/example_sandbox_toolset.py | 100 ++-
.../ai/example_dags/example_strands_agent.py | 89 +++
.../providers/common/ai/get_provider_info.py | 7 +-
.../airflow/providers/common/ai/operators/agent.py | 91 ++-
.../providers/common/ai/sandbox/__init__.py | 6 +
.../airflow/providers/common/ai/sandbox/base.py | 245 ++++++-
.../airflow/providers/common/ai/sandbox/modal.py | 246 +++++--
.../providers/common/ai/sandbox/opensandbox.py | 545 +++++++++++++++
.../src/airflow/providers/common/ai/sandbox/sbx.py | 17 +-
.../airflow/providers/common/ai/tools/__init__.py | 135 ++++
.../providers/common/ai/tools/_from_toolset.py | 100 +++
.../airflow/providers/common/ai/tools/strands.py | 108 +++
.../airflow/providers/common/ai/toolsets/hook.py | 17 +
.../common/ai/toolsets/langchain_bridge.py | 28 +-
.../providers/common/ai/toolsets/sandbox.py | 304 ++++++++-
.../airflow/providers/common/ai/toolsets/sql.py | 18 +
.../providers/common/ai/utils/coroutines.py} | 30 +-
.../common/ai/tests/system/common/ai/README.md | 37 +-
.../common/ai/example_sandbox_toolset_modal.py | 138 +++-
...x.py => example_sandbox_toolset_opensandbox.py} | 27 +-
.../common/ai/tests/unit/common/ai/conftest.py | 34 +
.../tests/unit/common/ai/operators/test_agent.py | 170 ++++-
.../ai/tests/unit/common/ai/sandbox/fake_modal.py | 17 +
.../ai/tests/unit/common/ai/sandbox/fake_tags.py | 63 ++
.../ai/tests/unit/common/ai/sandbox/test_base.py | 173 +++++
.../ai/tests/unit/common/ai/sandbox/test_modal.py | 207 +++++-
.../unit/common/ai/sandbox/test_opensandbox.py | 734 +++++++++++++++++++++
.../ai/tests/unit/common/ai/sandbox/test_sbx.py | 6 +
.../ai/tests/unit/common/ai/tools}/__init__.py | 0
.../unit/common/ai/tools/test__from_toolset.py | 155 +++++
.../ai/tests/unit/common/ai/tools/test_strands.py | 107 +++
.../ai/tests/unit/common/ai/tools/test_tools.py | 111 ++++
.../tests/unit/common/ai/toolsets/test_sandbox.py | 371 ++++++++++-
.../tests/unit/common/ai/utils/test_coroutines.py | 25 +-
providers/common/sql/docs/index.rst | 2 +-
providers/common/sql/pyproject.toml | 2 +-
providers/exasol/docs/index.rst | 8 +-
providers/exasol/pyproject.toml | 2 +-
.../fab/src/airflow/providers/fab/www/package.json | 2 +-
.../src/airflow/providers/fab/www/pnpm-lock.yaml | 10 +-
providers/jdbc/docs/index.rst | 2 +-
providers/jdbc/pyproject.toml | 2 +-
{airflow-core => providers/modal}/.gitignore | 0
{chart => providers/modal}/LICENSE | 0
NOTICE => providers/modal/NOTICE | 0
providers/{opensearch => modal}/README.rst | 21 +-
providers/modal/docs/.latest-doc-only-change.txt | 1 +
providers/{duckdb => modal}/docs/changelog.rst | 6 +-
providers/{keycloak => modal}/docs/commits.rst | 7 +-
providers/{mongo => modal}/docs/conf.py | 2 +-
providers/modal/docs/connections.rst | 85 +++
providers/modal/docs/index.rst | 131 ++++
.../docs/installing-providers-from-sources.rst | 0
providers/modal/docs/integration-logos/Modal.png | Bin 0 -> 10357 bytes
providers/{anthropic => modal}/docs/security.rst | 0
providers/modal/docs/usage.rst | 158 +++++
providers/modal/provider.yaml | 74 +++
providers/{vespa => modal}/pyproject.toml | 20 +-
.../modal/src/airflow}/__init__.py | 0
.../modal/src/airflow/providers}/__init__.py | 0
.../src/airflow/providers/modal}/__init__.py | 4 +-
.../src/airflow/providers/modal/exceptions.py | 18 +-
.../airflow/providers/modal/get_provider_info.py | 61 ++
.../src/airflow/providers/modal/hooks}/__init__.py | 0
.../src/airflow/providers/modal/hooks/modal.py | 279 ++++++++
providers/{airbyte => modal}/tests/conftest.py | 0
.../modal/tests/unit}/__init__.py | 0
.../modal/tests/unit/modal}/__init__.py | 0
.../modal/tests/unit/modal/hooks}/__init__.py | 0
.../modal/tests/unit/modal/hooks/test_modal.py | 372 +++++++++++
.../tests/unit/modal}/test_exceptions.py | 14 +-
providers/openlineage/docs/index.rst | 8 +-
providers/openlineage/pyproject.toml | 2 +-
providers/postgres/docs/index.rst | 2 +-
providers/postgres/pyproject.toml | 2 +-
providers/presto/docs/index.rst | 2 +-
providers/presto/pyproject.toml | 2 +-
providers/teradata/docs/index.rst | 2 +-
providers/teradata/pyproject.toml | 2 +-
providers/vertica/docs/index.rst | 8 +-
providers/vertica/pyproject.toml | 2 +-
providers/ydb/docs/index.rst | 8 +-
providers/ydb/pyproject.toml | 2 +-
pyproject.toml | 10 +
scripts/ci/docker-compose/remove-sources.yml | 1 +
scripts/ci/docker-compose/tests-sources.yml | 1 +
scripts/ci/prek/changelog_duplicates.py | 11 +-
scripts/tests/ci/prek/test_changelog_duplicates.py | 36 +-
.../airflow/sdk/coordinators/_bundle_metadata.py | 8 +-
.../coordinators/executable/test_coordinator.py | 8 +-
.../task_sdk/coordinators/java/test_coordinator.py | 8 +-
.../task_sdk/coordinators/node/test_coordinator.py | 8 +-
uv.lock | 112 +++-
148 files changed, 6623 insertions(+), 532 deletions(-)
create mode 100644 providers/common/ai/docs/frameworks/index.rst
create mode 100644 providers/common/ai/docs/frameworks/strands.rst
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/example_dags/example_strands_agent.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/sandbox/opensandbox.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/tools/__init__.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/tools/_from_toolset.py
create mode 100644
providers/common/ai/src/airflow/providers/common/ai/tools/strands.py
copy providers/{fab/src/airflow/providers/fab/www/airflow_flask_app.py =>
common/ai/src/airflow/providers/common/ai/utils/coroutines.py} (54%)
copy
providers/common/ai/tests/system/common/ai/{example_sandbox_toolset_sbx.py =>
example_sandbox_toolset_opensandbox.py} (80%)
create mode 100644
providers/common/ai/tests/unit/common/ai/sandbox/fake_tags.py
create mode 100644
providers/common/ai/tests/unit/common/ai/sandbox/test_opensandbox.py
copy {airflow-core/src/airflow/_shared =>
providers/common/ai/tests/unit/common/ai/tools}/__init__.py (100%)
create mode 100644
providers/common/ai/tests/unit/common/ai/tools/test__from_toolset.py
create mode 100644
providers/common/ai/tests/unit/common/ai/tools/test_strands.py
create mode 100644 providers/common/ai/tests/unit/common/ai/tools/test_tools.py
copy airflow-core/src/airflow/api_fastapi/core_api/datamodels/providers.py =>
providers/common/ai/tests/unit/common/ai/utils/test_coroutines.py (57%)
copy {airflow-core => providers/modal}/.gitignore (100%)
copy {chart => providers/modal}/LICENSE (100%)
copy NOTICE => providers/modal/NOTICE (100%)
copy providers/{opensearch => modal}/README.rst (77%)
create mode 100644 providers/modal/docs/.latest-doc-only-change.txt
copy providers/{duckdb => modal}/docs/changelog.rst (88%)
copy providers/{keycloak => modal}/docs/commits.rst (93%)
copy providers/{mongo => modal}/docs/conf.py (99%)
create mode 100644 providers/modal/docs/connections.rst
create mode 100644 providers/modal/docs/index.rst
copy providers/{airbyte => modal}/docs/installing-providers-from-sources.rst
(100%)
create mode 100644 providers/modal/docs/integration-logos/Modal.png
copy providers/{anthropic => modal}/docs/security.rst (100%)
create mode 100644 providers/modal/docs/usage.rst
create mode 100644 providers/modal/provider.yaml
copy providers/{vespa => modal}/pyproject.toml (91%)
copy {airflow-core/tests/integration =>
providers/modal/src/airflow}/__init__.py (100%)
copy {airflow-core/tests/integration =>
providers/modal/src/airflow/providers}/__init__.py (100%)
copy providers/{edge3/src/airflow/providers/edge3 =>
modal/src/airflow/providers/modal}/__init__.py (94%)
copy airflow-core/tests/unit/task/conftest.py =>
providers/modal/src/airflow/providers/modal/exceptions.py (63%)
create mode 100644
providers/modal/src/airflow/providers/modal/get_provider_info.py
copy {airflow-core/src/airflow/_shared =>
providers/modal/src/airflow/providers/modal/hooks}/__init__.py (100%)
create mode 100644 providers/modal/src/airflow/providers/modal/hooks/modal.py
copy providers/{airbyte => modal}/tests/conftest.py (100%)
copy {airflow-core/tests/integration =>
providers/modal/tests/unit}/__init__.py (100%)
copy {airflow-core/src/airflow/_shared =>
providers/modal/tests/unit/modal}/__init__.py (100%)
copy {airflow-core/src/airflow/_shared =>
providers/modal/tests/unit/modal/hooks}/__init__.py (100%)
create mode 100644 providers/modal/tests/unit/modal/hooks/test_modal.py
copy providers/{alibaba/tests/unit/alibaba/cloud =>
modal/tests/unit/modal}/test_exceptions.py (62%)
