zhang-arvin opened a new pull request, #11883: URL: https://github.com/apache/seatunnel/pull/11883
## Purpose Fix the MySQL sink "No operations allowed after statement closed" error in long-running streaming jobs (#11769). ## Root Cause Three interrelated issues cause stale connections and statement leaks: 1. **ConnectionPoolManager caches connections that HikariCP can evict**: When HikariCP evicts a connection (due to `idleTimeout` or `maxLifetime`), the connection is closed but the reference remains in the `ConnectionPoolManager.connectionMap`. Subsequent calls to `getConnection()` return the stale (closed) connection, causing "No operations allowed after statement closed" errors. 2. **JdbcConnectionValidationUtils ignores user-configured connectionTestQuery**: The `getConnectionValidationQuery()` method only returns a validation query for the Xugu driver. For MySQL and other drivers, it returns `Optional.empty()`, so `isConnectionValid()` uses `Connection.isValid()` which can return false positives in environments with proxies, NATs, or firewalls. 3. **HikariCP pool-level properties are silently ignored**: User-configured properties like `max-lifetime`, `keepalive-time`, `connection-test-query` are passed via `addDataSourceProperty()` which routes them to the underlying JDBC DataSource, not to HikariCP's pool configuration. These properties are silently ignored, preventing proper connection lifecycle management. ## Changes ### 1. ConnectionPoolManager.getConnection() - Check if the cached connection has been closed by HikariCP - If closed, evict the stale reference and obtain a fresh connection from the pool - This prevents "No operations allowed after statement closed" errors in long-running streaming jobs ### 2. JdbcConnectionValidationUtils.getConnectionValidationQuery() - After checking the driver-specific hook (Xugu), fall back to the user-configured `connectionTestQuery` / `connection-test-query` from the `properties` block - This enables SQL-based liveness checks instead of `Connection.isValid()` which can return false positives ### 3. JdbcSinkWriter.applyConnectionValidation() - Apply all HikariCP pool-level properties (maxLifetime, keepaliveTime, validationTimeout, idleTimeout, connectionTimeout, maximumPoolSize, minimumIdle) from user properties config - Add isHikariPoolProperty() filter to prevent these properties from being redundantly passed to addDataSourceProperty() ### Files Changed - `seatunnel-connectors-v2/connector-jdbc/src/main/java/.../sink/ConnectionPoolManager.java` - `seatunnel-connectors-v2/connector-jdbc/src/main/java/.../internal/connection/JdbcConnectionValidationUtils.java` - `seatunnel-connectors-v2/connector-jdbc/src/main/java/.../sink/JdbcSinkWriter.java` ### Related Issues - Closes #11769 - Related to PR #11771 (partial fix for connection-test-query only) ### Verification - All existing tests pass: JdbcSinkWriterTest (9 tests), JdbcConnectionValidationUtilsTest (2 tests) - Connection validation query from user properties is now properly applied to both HikariCP pool and isConnectionValid() check -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
