shbhmrzd opened a new pull request, #4272:
URL: https://github.com/apache/iggy/pull/4272
## Which issue does this PR address?
Relates to #2500.
This supersedes #3588, which was closed by the stale bot. It includes the
review follow-ups from that PR and merges the latest `master`.
## Rationale
Iggy currently has database-specific connectors. A generic JDBC source lets
Iggy read from PostgreSQL, MySQL, Oracle, SQL Server, H2, and other
JDBC-compliant databases through their existing JDBC drivers.
## What changed?
- Adds a JDBC source connector backed by an embedded JVM and the standard
`java.sql` API.
- Supports bulk and incremental polling, including persisted offsets through
`{last_offset}` substitution.
- Maps JDBC values to JSON while preserving precision for decimal and BIGINT
values and encoding binary values as base64.
- Classifies SQL failures by SQLState and clears pending Java exceptions
safely.
- Adds connector configuration, per-database examples, release artifact
wiring, and PostgreSQL integration coverage.
- Updates the shared connector test harness for current `master`, including
persisted topic durability required by source connectors.
## How to review
| Area | Where to look |
|---|---|
| Source logic and JNI lifecycle |
`core/connectors/sources/jdbc_source/src/lib.rs` |
| Configuration and usage | `core/connectors/sources/jdbc_source/README.md`
and `config.toml` |
| Integration coverage | `core/integration/tests/connectors/jdbc/` |
| Shared harness compatibility | `core/integration/tests/connectors/mod.rs` |
## Verification
- `cargo fmt --all`
- `cargo sort --no-format --workspace`
- `cargo check -p iggy_connector_jdbc_source`
- `cargo clippy -p iggy_connector_jdbc_source --all-features --all-targets
-- -D warnings`
- `cargo test -p iggy_connector_jdbc_source --all-features` (65 passed)
- `cargo clippy -p iggy_connector_sdk --all-targets --all-features -- -D
warnings`
- `cargo test -p iggy_connector_sdk --all-features` (220 passed)
- `cargo clippy -p integration --all-targets --all-features -- -D warnings`
- `IGGY_TEST_CLUSTER_NODES=1 cargo test -p integration --
connectors::jdbc::` (7 passed)
The local three-node integration harness did not reach VSR mesh readiness on
this machine, before any JDBC test ran. The JDBC suite passes in the
harness-supported single-node mode above.
## Limitations
- JNI permits one JVM per process, so JDBC source and sink connectors should
run in separate connector-runtime processes.
- JDBC calls are synchronous on the runtime worker thread.
- A JVM and database-specific JDBC driver JAR are required at runtime.
## AI Usage
Claude Code was used substantially for implementation, tests, documentation,
and review hardening. The changes were verified with the commands above and can
be explained line by line.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
