raman20 opened a new pull request, #3789: URL: https://github.com/apache/iggy/pull/3789
## Which issue does this PR address? Relates to #3764 ## Rationale The Apache Iggy connectors subsystem needed a native `iggy_source` plugin to poll messages from an external Iggy stream/topic and publish them into local Iggy streams via the connectors runtime. ## What changed? The connectors runtime lacked a source plugin capable of ingesting streaming messages from an external Iggy cluster. This PR implements `iggy_connector_iggy_source` as a `cdylib` shared library plugin loaded dynamically by the connectors runtime. It polls messages from a remote Iggy cluster, maintains atomic offset state persistence via MessagePack (`State`), preserves original 128-bit message IDs and user headers, and adheres strictly to the maintainer source connector guidelines (sleep-first poll, non-blocking lock discipline, ascending offset sequence). ## Local Execution - Passed - Pre-commit hooks ran Verified locally inside devcontainer: - `cargo fmt --all --check` (Passed) - `cargo clippy -p iggy_connector_iggy_source --all-targets -- -D warnings` (Passed with 0 warnings) - `cargo test -p iggy_connector_iggy_source` (**Passed 8/8 unit tests**) - Live end-to-end testing in devcontainer with `iggy-server` and `iggy-connectors` runtime. ## AI Usage 1. Which tools? Google Antigravity 2. Scope of usage? Learning data-flow & implementation of `iggy_source` plugin, unit test suite, and plugin documentation. 3. How did you verify the generated code works correctly? Verified via unit tests (`cargo test`), zero-warning clippy check, and live message polling and ingestion inside devcontainer. 4. Can you explain every line of the code if asked? Yes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
