qqeasonchen opened a new pull request, #5328:
URL: https://github.com/apache/eventmesh/pull/5328
### Motivation
Connector plane hardening batch — three P0/P1 items from the connector
review (#5296 follow-up):
1. **P0 data-loss hardening** (follow-up to #5231 / #5232 / #5233): those
three bugs crashed the *old* `openconnect` framework (already removed by the
uni-arch redesign); the crash classes (`SourceWorker`,
`RecordOffsetManagement`, `CommonProtocol`) no longer exist. But the *failure
mode* — one bad event killing the loop and silently stranding the rest of the
batch — could still happen in the new `ConnectorRuntime`. This PR closes that
class of bugs:
- `runSourceOnce()`: per-event isolation — a publish that throws (bad
CloudEvent, broken URL) is logged + skipped + counted; the rest of the batch
still publishes; `null` events in a `poll()` batch are skipped, not NPE.
- `runSinkOnce()`: per-delivery ACK isolation — one ACK failure no longer
loses the remaining ACKs/offset; EventMesh re-delivers.
- `offsetStore.put()` failures no longer abort the publish/commit path
(checkpoint lost is logged, event flow continues).
- `getSourcePublishFailures()` counter exposes skipped events to the
admin API.
2. **P1 connector-api module split** (plan doc included at
`docs/contrib/connector-api-split-plan.md`): the 7 SPI interfaces
(`SourceConnector`, `SinkConnector`, `EventMeshEndpoint`, `HttpCaller`,
`ConnectorOffsetStore`, `CloudEventSerializer`, `PollEntry`) move from
`eventmesh-connector-runtime` into a new `:eventmesh-connector-api` module
(deps: cloudevents-core only). The package name stays
`org.apache.eventmesh.connector`, so all 23 plugins keep their imports — only
their `build.gradle` dependency swaps. Enforcement is two-layer:
- compile-time: plugins no longer have the runtime on their classpath;
- ArchUnit: new `ruleConnectorPluginsDependOnlyOnSpi` (FAIL mode, extends
#5305 guard) + a canary negative test proving the rule fires.
3. **P0 plugin unit tests** (#4642 follow-up): first tests ever for
`connector-file` (5+5), `connector-kafka` (1, Kafka `MockProducer`),
`connector-pulsar` (2, smoke), `connector-rocketmq` (3, smoke). File connectors
gain `close*Quietly()` so tests release file handles on Windows.
Also: `eventmesh.properties` gains a commented Connector Runtime section;
`dist-connector` packaging ships the api jar in `apps/` next to the runtime jar.
### Modifications
- `eventmesh-connector-api/` — new module: 7 SPI interfaces + package-info
- `eventmesh-connector-runtime/` — SPI files removed; `api
project(':eventmesh-connector-api')` dependency; `ConnectorRuntime` hardening +
4 new tests
- 23 × `eventmesh-connector-plugin/*/build.gradle` — dep swap runtime → api
- `eventmesh-architecture-guard/` — new rule + canary test (11/11 green)
- `build.gradle` (root) — dist-connector ships api jar
- `eventmesh-runtime/conf/eventmesh.properties` — connector section
-
`eventmesh-connector-plugin/eventmesh-connector-{file,kafka,pulsar,rocketmq}/src/test/`
— new tests
### Documentation
- `docs/contrib/connector-api-split-plan.md` — split rationale, migration
steps (M1/M2/M3), risks, acceptance criteria
### Verifying this patch
Local (JDK 21, Windows): `:eventmesh-connector-api:build`,
`:eventmesh-connector-runtime:build`,
`:connector-{file,kafka,pulsar,rocketmq}:build` (incl. checkstyle),
`:eventmesh-runtime:compileJava`, `:eventmesh-architecture-guard:test` 11/11 —
all green. 33 new/updated unit tests total (17 runtime + 16 plugin).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
