andrewmusselman opened a new issue, #29:
URL: https://github.com/apache/tooling-gofannon/issues/29
### Summary
There's no way to move a set of chained agents between Gofannon deployments,
share a pipeline with another ASF project, version-control a pipeline in git,
or back it up before breaking changes. The ASVS pipeline exists in
`apache/tooling-agents` as 11 separate `.py` files, but there's no manifest
tying them together and no system support for *ingesting* that directory as a
unit.
### Details
**Problem:**
Operators want a two-way format:
```
Gofannon UI ────[export]────▶ on-disk directory ────[git push]────▶
GitHub
Gofannon UI ◀────[import]──── on-disk directory ◀────[git clone]───
GitHub
```
Round-trip stable: export → import → export produces an equivalent bundle
(modulo timestamps and a manifest hash). Same operator workflow for local zip
and remote GitHub repo.
### Proposed solution
**On-disk layout:**
```
my-pipeline/
├── manifest.yaml # Top-level pipeline metadata
├── agents/
│ ├── asvs_orchestrate/
│ │ ├── agent.yaml # Per-agent metadata
│ │ ├── run.py # Agent source code
│ │ ├── tests/
│ │ │ ├── unit/
│ │ │ └── fixtures/
│ │ └── README.md
│ └── ...
├── integration_tests/
├── LICENSE
└── README.md
```
**Manifest fields:**
- `manifest_version: 1`
- `compatible_with.gofannon_platform: ">=0.8.0,<1.0.0"`
- Per-agent semver + pipeline-level semver
- `call_graph` describing inter-agent dependencies
- `external_dependencies.secrets` declarations (never values)
- `external_dependencies.mcp_servers` and `swagger_specs` references (URLs
only)
- `namespaces.reads` and `namespaces.writes` patterns
- `runtime_env` declarations for tunable env vars
**Per-agent `agent.yaml`:** input/output schemas, llm_settings (compose
model, invokable models with params), tools (MCP servers, Swagger specs),
env_vars (post-ISSUE-008).
**New endpoints:**
```
POST /pipelines/export
POST /pipelines/import
POST /pipelines/import/from-github
POST /pipelines/import/dry-run # Preview diff before applying
```
**Things to NOT export (security):** secret values, session state, data
store *contents*, user profile data, internal UUIDs.
**Phasing:**
1. **MVP:** Single-agent round-trip (~600 LOC backend + ~200 LOC frontend)
2. **Pipelines:** Multi-agent with call-graph, conflict resolution (~400 +
~250 LOC)
3. **GitHub:** Pull via tarball API, push via existing `asvs_push_github`
pattern (~300 LOC)
4. **Test fixtures:** Per-agent unit + pipeline-level integration;
lightweight CI runner (~500 LOC)
5. **Provenance / signing (conditional):** GPG or sigstore (~400 LOC)
### Acceptance Criteria
- [ ] Fixed: `manifest.yaml` and `agent.yaml` pydantic schemas defined
- [ ] Fixed: Phase 1 — single-agent export to zip, import from zip
- [ ] Fixed: Phase 2 — multi-agent export with call_graph; import resolves
dependencies in topological order
- [ ] Fixed: Phase 3 — GitHub pull (tarball API) and push integration
- [ ] Fixed: Phase 4 — test fixture format and standalone runner
- [ ] Fixed: Dry-run shows what would change without applying
- [ ] Test added: Round-trip stability (export → import → export produces
equivalent bundle)
- [ ] Test added: Conflict resolution on name collision (rename / replace /
abort)
- [ ] Test added: Atomicity (either whole bundle imports or nothing does)
- [ ] Documentation: Bundle format RFC, examples using existing ASVS pipeline
### References
- File: New module
`webapp/packages/api/user-service/services/pipeline_bundle_service.py`
- Reference layout: `apache/tooling-agents/ASVS/agents/` (informal prototype)
- Tracker: FIXES.md item #15
### Priority
**Medium / RFC** - Substantial feature. Becomes more capable after ISSUE-008
(env vars in manifest) and ISSUE-002 (workspace-scoped imports), but neither is
a strict blocker. Phase 1 alone unlocks ASVS pipeline as a reusable artifact
for other ASF projects.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
