wenjin272 opened a new pull request, #1214: URL: https://github.com/apache/flink-agents/pull/1214
Linked issue: Fixes #1194 ### Purpose of change Python agents can load Skills from job JARs through `Skills.from_classpath(...)`, and Java agents can load Skills from Python packages through `Skills.fromPackage(...)`. Both combinations also work through YAML, completing source support independently of the language declaring the agent. #### Runtime flow - Python classpath sources call `JavaResourceAdapter.extractClasspathSkills` with the operator's Flink user-code classloader. Java resolves the resource and copies/extracts it into the Python repository's temporary directory. - Java package sources cause `PythonBridgeManager` to initialize Python even for an otherwise Java-only agent. The resource context obtains the initialized interpreter when creating `SkillManager`; `PackageSkillRepository` invokes the Python package materializer into a Java-owned temporary directory. - The consuming repository discovers Skills and reads their bodies and attachments locally. Repository close releases owned temporary files, including through operator shutdown. Original JARs and package files remain intact. #### Key decisions Each `SkillManager` owns its source-handler map, so a handler can bind its operator's interpreter or bridge without storing runtime instances globally. Source resolution is shared with native repositories through internal materializers. Classpath fallback opens archive files without requiring a `.jar` suffix, covering Flink BLOB cache names and JARs without explicit directory entries. ### Behavioral Semantics #### Interaction decisions | Agent language | Classpath source | Python package source | |---|---|---| | Java | Resolve with the job classloader | Initialize/reuse Python; copy resources through the interpreter | | Python | Resolve through the operator's Java bridge | Resolve through Python package resources | Cross-language use requires the Flink runtime and the corresponding dependencies in the job environment. Declaring a source does not install its JAR or package. Missing runtime bridges fail loading rather than selecting another source. #### Behavioral contracts 1. API and YAML declarations support both cross-language directions, including `SKILL.md` and accompanying files. 2. Managers using the same resource names in different operator environments resolve through their own bridges/classloaders. 3. Closing repositories, or failing during materialization/partial loading, releases owned temporary directories. Existing duplicate-Skill precedence is preserved, and displaced repositories are still closed. 4. Missing sources or bridges fail initialization with source context. `load_skill` returns an error response on initialization failure in both languages, logs the exception chain, and does not misreport repeated failures as an empty configuration. URL origins in responses are redacted. 5. Existing native Java classpath and Python package sources continue to work, including archives and zip-imported Python packages. #### Failure behavior Repository initialization errors propagate to direct manager callers; partial results are cleaned up. The manager is marked initialized only after successful creation, allowing a later call to attempt initialization again. The built-in `load_skill` catches initialization exceptions and returns `ToolResponse.error`; no configured manager, unknown Skills, and missing attachments retain their error-response behavior. Repository cleanup failures retain the existing propagation behavior. ### Tests | Contract | Coverage | |---|---| | 1: Packaged sources through API/YAML | `PackageSkillsCrossLanguageTest` installs a generated wheel; `skills_cross_language_test.py` submits a resource JAR. Both verify body and attachment reads. Declaration serialization tests cover the added factories. | | 2: Operator isolation and job classloader | `CrossLanguageSkillSourceTest`, `test_classpath_repository.py`; includes reopening one manager's source after another manager is created and an extensionless JAR through a parent loader. | | 3: Ownership and cleanup | Cross-language E2Es, package/classpath repository tests, `SkillManagerTest`, `test_manager.py`, and `ResourceCacheTest`. | | 4: Failure responses and diagnostics | `LoadSkillToolTest`, `test_load_skill.py`, and missing-source/bridge tests; repeated calls and URL redaction are asserted. | | 5: Native source compatibility | Existing classpath/package repository suites plus zip-imported package tests. | Validation: Java clean reactor build with focused API/runtime Skills tests and two Java E2Es passed (170 tests); the final Java tool-response regressions passed (12 tests). Python Skills/API/YAML/cache suites passed (312 tests), as did two packaged-JAR E2Es and the final tool/classpath regressions (17 tests). Spotless, Ruff, and `git diff --check` passed. Distribution JARs were clean-built and the Python wheel rebuilt for cross-language verification. Validated with Java 11, Python 3.11, and Flink 2.3. Other supported version combinations, a remote multi-TaskManager deployment, and injected operator failover were not exercised. These runs are focused validation, not the full project test suite. ### API Adds Java `Skills.fromPackage(String, String)` and Python `Skills.from_classpath(*resources)`; existing native factories remain available. YAML keeps the existing `package`/`classpath` shape and gains the corresponding runtime paths. Java/Python runtime-global source registries are removed in favor of instance-owned handlers; these are internal runtime interfaces. Python `load_skill` now returns an error response for initialization failures that previously escaped as exceptions; Java preserves its response-based contract with more accurate diagnostics. ### Documentation - [ ] `doc-needed` - [ ] `doc-not-needed` - [x] `doc-included` Updates Skills usage/runtime requirements and refreshes generated YAML schemas and the bundled schema contract. ### Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No Generated-by: Codex 0.153.4 (GPT-6) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
