david-streamlio opened a new pull request, #105:
URL: https://github.com/apache/pulsar-connectors/pull/105
Fixes #46
### Motivation
The OpenMLDB JDBC sink had only mock-based unit tests; nothing exercised it
against a real OpenMLDB cluster. Attempting to do so surfaced that the
connector was unusable in a current build, so this PR is a dependency fix plus
the first integration test, verified green on a Linux x86-64 host.
### Modifications
**Commit 1 — `[fix][io]` upgrade OpenMLDB client 0.4.4-hotfix1 → 0.9.2.**
The pinned client was broken three ways: its native SDK (`libsql_jsdk.so`)
references `_dl_sym@GLIBC_PRIVATE`, removed in glibc 2.34, and hard-crashes the
JVM on any modern Linux host; its JDBC driver has no
`META-INF/services/java.sql.Driver` entry, so `DriverManager` never discovers
it (`No suitable driver`); and 0.9.2 is the newest version with a matching
`4pdosc/openmldb` Docker image. The 0.9.2 SDK needs two module-scoped
`resolutionStrategy.force` pins against the platform-enforced versions: Curator
4.2.0 (`NodeCache.getListenable()` signature changed in Curator 5.x →
`NoSuchMethodError` on connect) and protobuf-java 3.21.12 (the SDK's generated
classes call `makeExtensionsImmutable()`, removed in protobuf 3.22 →
`NoSuchMethodError` on insert). NARs never bundle protobuf, so the protobuf
force affects only compile/test classpaths.
**Commit 2 — `[improve][test]` add `OpenMLDBJdbcSinkIntegrationTest`.**
Starts an OpenMLDB 0.9.2 cluster via Testcontainers (the image's `init.sh`;
ZooKeeper on `localhost:2181`, zkPath `/openmldb`) with host networking,
creates the database/table, writes an Avro `GenericRecord` through
`OpenMLDBJdbcAutoSchemaSink`, and reads the row back over a fresh JDBC
connection. Host networking is required because the driver discovers
tablet/nameserver endpoints via ZooKeeper and dials them directly — which also
makes the test Linux-only, as is the driver's native SDK. Two driver
limitations shape the test: no `key`/`nonKey` config (the driver supports only
INSERT prepared statements; `key`/`nonKey` makes the sink prepare UPDATE/DELETE
→ `unsupported sql`), and positional `ResultSet` getters (label-based getters
throw `current do not support this method`).
### Verifying this change
Run on Linux x86-64 (Ubuntu 24.04, Docker 29.1.3, OpenJDK 21):
```
$ ./gradlew :jdbc:pulsar-io-jdbc-openmldb:test --rerun-tasks
Gradle suite > Gradle test >
org.apache.pulsar.io.jdbc.OpenMLDBJdbcSinkIntegrationTest >
testOpenWritesAndReadsBack PASSED
BUILD SUCCESSFUL in 1m 35s
```
Mutation check (test is not vacuous): with the read-back assertion
deliberately broken, the test fails against the value actually stored in
OpenMLDB, then passes again after reverting:
```
Gradle suite > Gradle test >
org.apache.pulsar.io.jdbc.OpenMLDBJdbcSinkIntegrationTest >
testOpenWritesAndReadsBack FAILED
java.lang.AssertionError: expected [WRONG_value_mutation_check] but
found [ValueOfField1]
```
### Documentation
- [x] `doc-not-needed` (dependency fix + test)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
