nzw921rx opened a new pull request, #10876:
URL: https://github.com/apache/seatunnel/pull/10876
### Purpose of this pull request
Depends on: #10871
Closes: #10666
This PR delivers **Phase 2** of STIP-24 by introducing a standalone
**SeaTunnel Edge Agent** runtime and wiring it to the Phase 1 EdgeSocket
ingress path in Zeta.
Phase 1 focused on engine-side ingress capability.
Phase 2 focuses on edge-side collector runtime, durability, transport, and
distribution integration.
**Changes**:
- Add standalone `seatunnel-edge-agent` parent module with:
- `seatunnel-edge-agent-core`
- `seatunnel-edge-agent-transport`
- `seatunnel-edge-agent-connector`
- Add Edge Agent runtime bootstrap and lifecycle (`EdgeAgentMain`, bootstrap
loop), including YAML load/validation and graceful startup/shutdown flow.
- Add local input connectors for edge collection:
- `file`
- `log`
- `event`
with NDJSON normalization and unified record path.
- Add durable outbound buffering via SQLite WAL:
- pending/sending/acked state management
- retry/backoff handling for failed sends
- stale in-flight recovery on restart.
- Add EdgeSocket transport client for Phase 1 protocol interop:
- auth / batch send / commit polling
- reconnect + rediscovery behavior using job task-group addresses.
- Add edge agent launcher scripts and default config:
- `seatunnel-edge-agent/bin/seatunnel-edge-agent.sh`
- `seatunnel-edge-agent/bin/seatunnel-edge-agent.cmd`
- `seatunnel-edge-agent/conf/agent.yaml`
- Integrate agent packaging into existing `seatunnel-dist` standard
assemblies:
- `assembly-bin.xml`
- `assembly-bin-ci.xml`
- `assembly-src.xml`
(no extra parallel assembly entry maintained).
- Add/adjust Edge Agent E2E module structure:
- `seatunnel-e2e/seatunnel-edge-agent-e2e`
- Update docs in both English and Chinese:
- architecture docs
- module README
- packaging/runtime path conventions aligned with current dist structure.
### Does this PR introduce _any_ user-facing change?
Yes.
- Introduces a new standalone **Edge Agent runtime** for edge-side data
collection and forwarding.
- Adds user-facing Edge Agent scripts and default YAML config.
- Adds/clarifies user-facing runtime and architecture documentation for Edge
Agent.
- Integrates agent artifacts into standard SeaTunnel dist assembly outputs.
Compatibility note:
- No breaking change for existing Zeta/connector jobs.
- This phase extends capabilities and runtime packaging only.
### How was this patch tested?
- Unit tests for key runtime and transport behaviors, including:
- config parsing/validation paths
- transport protocol and reconnect logic
- parser/runtime utility coverage
- Existing EdgeSocket Phase 1 tests remain as protocol compatibility
baseline.
- Manual packaging verification checklist:
- build `seatunnel-dist`
- verify agent scripts are included in distribution `bin/`
- verify `agent.yaml` is included in distribution `conf/`
- verify edge-agent runtime jars are present in distribution `lib/`
- Documentation consistency checks between code/config and `docs/en` +
`docs/zh`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
