kaxil commented on code in PR #73912: URL: https://github.com/apache/airflow/pull/73912#discussion_r4146104289
########## scripts/in_container/run_agent_framework_tests.sh: ########## @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Runs the common.ai adapter tests for agent frameworks that cannot join the workspace lock. +# +# Strands Agents caps mcp, and Google ADK caps opentelemetry and websockets, below the versions +# uv.lock resolves, so their adapter tests are skipped everywhere else in CI. This installs one +# framework into the CI image and runs the tests of the framework-neutral tools and their adapters; +# the other framework's tests skip themselves. One framework per invocation, so a bad release of +# one cannot mask the other. +# +# By default every package already in the image is held at its installed version with uv's +# --override, so the framework is tested against the same dependencies as the rest of Airflow and +# its caps on them are overridden. With --framework-pins the framework's own requirements win +# instead, which is the environment a user who installs it gets. +# +# The newest framework release older than the repository's uv exclude-newer window is installed. +set -euo pipefail + +TEST_PATH="providers/common/ai/tests/unit/common/ai/tools" + +framework="${1:-}" +case "${framework}" in + strands-agents) import_check="import strands" ;; + google-adk) import_check="import google.adk" ;; + *) + echo "Usage: $0 <strands-agents|google-adk> [--framework-pins]" >&2 + exit 1 + ;; +esac +FRAMEWORKS=("${framework}") + +framework_pins="false" +if [[ ${2:-} == "--framework-pins" ]]; then + framework_pins="true" +elif [[ -n ${2:-} ]]; then + echo "Unknown argument: ${2}. The only option after the framework is --framework-pins." >&2 + exit 1 +fi + +cd "${AIRFLOW_SOURCES:-/opt/airflow}" + +if [[ ${framework_pins} == "true" ]]; then + echo "Installing ${FRAMEWORKS[*]} with their own dependency pins" + uv pip install "${FRAMEWORKS[@]}" +else + overrides=$(mktemp) + before=$(mktemp) + after=$(mktemp) + trap 'rm -f "${overrides}" "${before}" "${after}"' EXIT + uv pip freeze | sort > "${before}" + # Only name==version lines: editable and local installs cannot be expressed as an override, + # and the frameworks do not depend on any of them. Overriding the rest means nothing the + # image ships should change; the check below is there in case something still does. + grep -E '^[A-Za-z0-9_.-]+==' "${before}" > "${overrides}" + echo "Installing ${FRAMEWORKS[*]}, holding the image's $(wc -l < "${overrides}") installed packages" + uv pip install --override "${overrides}" "${FRAMEWORKS[@]}" + uv pip freeze | sort > "${after}" + changed=$(comm -23 "${before}" "${after}") + if [[ -n ${changed} ]]; then + echo "Installing the frameworks changed packages the image already had:" >&2 + echo "${changed}" >&2 + exit 1 + fi +fi + +uv pip freeze | grep -iE '^(strands-agents|google-adk|mcp|opentelemetry-(api|sdk)|websockets|google-genai)==' + +# The adapter tests skip themselves when their framework is missing, so a broken install would +# otherwise pass as green. +python -c "${import_check}" + +# --skip-db-tests: the job runs with backend "none", which has no database to set up. +pytest "${TEST_PATH}" --skip-db-tests -p no:cacheprovider --color=yes -ra Review Comment: Kept on purpose. With `--framework-pins` each leg runs the framework-neutral tools against a different resolved environment (Strands pulls `mcp` down to 2.1.1, ADK pulls `opentelemetry` to 1.42.1 and `websockets` to 15.0.1), so the second run of `test_tools.py` and friends is not a duplicate, it is the tools under that leg's dependency set. In the image-versions mode it is a duplicate and costs about three seconds. Happy to split a shared leg out if a third framework makes it add up. ########## .github/workflows/agent-framework-tests.yml: ########## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +--- +name: Agent framework tests +on: # yamllint disable-line rule:truthy + workflow_call: + inputs: + runners: + description: "The array of labels (in json form) determining runners." + required: true + type: string + platform: + description: "Platform for the build - 'linux/amd64' or 'linux/arm64'" + required: true + type: string + default-python-version: + description: "Which version of python should be used by default" + required: true + type: string + use-uv: + description: "Whether to use uv" + required: true + type: string + canary-run: + description: > + On a canary run the frameworks' own dependency pins win, which is the environment a + user who installs them gets. Otherwise every package in the CI image keeps its version + and the frameworks' caps on them are overridden. + required: true + type: string +permissions: + contents: read +jobs: + tests: + timeout-minutes: 30 + name: >- + Agent framework tests: ${{ matrix.framework }} + (${{ inputs.canary-run == 'true' && 'framework pins' || 'image versions' }}) + runs-on: ${{ fromJSON(inputs.runners) }} + # amd64 only for now: ci-amd.yml and ci-arm.yml must stay in sync, so the platform gate lives here. + if: inputs.platform == 'linux/amd64' Review Comment: Thanks. The sync check between the two workflow files forced the choice, and it keeps the arm switch a one-line change later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
