The GitHub Actions job "Scalafmt" on pekko-connectors.git/fix-hbase-source-leak has failed. Run started by GitHub user pjfanning (triggered by pjfanning).
Head commit for run: 28627f4fa03d63cc7618dca015ce5a5446192c2a / PJ Fanning <[email protected]> HBase: close the scanner and connection in the source stage Motivation: HBaseSourceLogic acquires three resources and released one. implicit val connection: Connection = connect(settings.conf) lazy val table: Table = getOrCreateTable(...).get override def preStart(): Unit = val scanner = table.getScanner(scan) // local val, never closed results = scanner.iterator() override def postStop(): Unit = table.close() // connection never closed The Connection is a ConnectionImplementation: a ZooKeeper connection, an RpcClient and a thread pool. Every materialization of HTableStage.source leaks one for the life of the JVM. The ResultScanner is never closed either, so the server-side scanner lease is held on the RegionServer until it times out, on every run. The sibling HBaseFlowStage already closes both table and connection, so this is an omission in the source rather than a deliberate difference. Modification: Keep the scanner in a field and close all three in postStop, each independently so that a failure to close one still releases the rest. Failures are logged rather than rethrown, matching HBaseFlowStage: the stage has already stopped, so failing here would only mask why. The previous code called failStage from postStop, which is too late to have any effect. `table` is lazy, so a `tableOpened` flag records whether preStart forced it; otherwise postStop would create a table only to close it. If preStart fails part way — getScanner throwing on a missing table is the common case — whatever was opened is still released. The close sequence is extracted to HBaseSourceLogic.closeAll so it can be tested without an HBase cluster. Result: Connections, tables and scanners are released when the source stops, on both the normal and the failure path. Tests: Added HBaseSourceLogicSpec covering the ordinary path, a scanner that fails to close, a table that fails to close, and all three failing. Each asserts the later resources are still closed. Needs no HBase server, which matters because the hbase integration tests are disabled in CI. 4 tests pass. Ran hbase/mimaReportBinaryIssues (clean) and the module scalafmt checks. References: Compare HBaseFlowStage.postStop, which closes table and connection. Report URL: https://github.com/apache/pekko-connectors/actions/runs/33276904578 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
