pjfanning opened a new pull request, #1919: URL: https://github.com/apache/pekko-connectors/pull/1919
Replaces the `harisekhon/hbase:2.1` container with `hbase-shaded-testing-util`, which starts a mini ZooKeeper, mini HDFS, master and region server inside the test JVM. Client versions are unchanged at **2.6.6** — this is only about how we run the tests. ### Why `harisekhon/hbase` has no tag newer than `2.1`, published in 2018, and there is no official apache hbase image. Being pinned to a server that old is what makes a client upgrade look impossible: hbase 3 clients default to `RpcConnectionRegistry`, and a 2.1 master has no `ClientMetaService` to answer them, which is why #1912 fails every test at connection setup. Whatever we decide about 3.x later, the test server should not be the thing deciding it. An in-JVM cluster is how hbase itself, Phoenix, and the Spark and Flink hbase connectors do integration testing. It runs the same code paths a real cluster runs, minus the multi-node parts, which is the right fidelity here — we are testing that the Pekko Streams stages drive the hbase client correctly. `hbase-shaded-testing-util` is the flavour that pairs with the `hbase-shaded-client` we already depend on. ### What changed - `hbase-shaded-testing-util` added in `Test` scope. - New `HBaseMiniCluster` test helper, started once per test JVM and shared by the Scala and Java suites (`Test / fork := true` is already set for this module). - `scripts/hbase-server.sh`, the `hbase` docker-compose service, and the workflow `pre_cmd` are gone. Two settings are needed to bring the cluster up on a current JDK, both commented in the helper: - master and region server web UIs off (`hbase.master.info.port` / `hbase.regionserver.info.port` = -1) — their JAXB stack needs `java.lang` opened to reflect, and we have no use for the web UI in tests; - `hbase.wal.provider` = `filesystem` — the default asyncfs writer reflects into protobuf internals in a way the shaded hbase/hadoop combination does not survive. ZooKeeper is pinned to its default port, so a plain `HBaseConfiguration.create()` still reaches the cluster and the `@@snip` documentation snippets are unchanged. ### Verification Local `sbt "hbase/test"`: 14 tests (7 Scala, 7 Java) pass in ~80s with no docker running. `checkCodeStyle` and `headerCheck` pass. The `hdfs` module already runs `MiniDFSCluster` in its tests on the same JDK, so this is not a new kind of dependency for the build. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
