CodersAcademy006 opened a new pull request, #58925: URL: https://github.com/apache/spark/pull/58925
### What changes were proposed in this pull request? This PR introduces configurable connection-retry attempts (`connectionRetryAttempts`) and retry backoff delay (`connectionRetryDelayMs`) for Spark JDBC readers and writers. When connecting to database clusters (such as AWS RDS during failover or intermittent network blips), connection attempts may transiently fail. Currently, Spark JDBC readers fail immediately upon receiving an initial connection failure without backoff. Key architectural design of this change: 1. **Decoupled Retry Wrapper**: The connection retry logic is encapsulated in `JdbcUtils.createConnectionFactory`, leaving `JdbcDialect.createConnectionFactory` untouched so custom dialect implementations automatically inherit retry behavior. 2. **Transient Exception Filtering**: Retries are strictly restricted to transient connection errors (`SQLTransientConnectionException` or `SQLException` with SQLState matching `"08*"`). Non-transient errors (such as invalid credentials or authorization failure) fail immediately without retrying. 3. **Task Interruption Safety**: Before and during retry backoff sleeps, `TaskContext.isInterrupted()` is checked to ensure task cancellation is handled immediately. ### Why are the changes needed? To prevent pipeline failures caused by temporary DB connection unavailability during database failovers or transient network hiccups. ### Does this PR introduce any user-facing change? Yes. Adds two new JDBC options: - `connectionRetryAttempts`: Number of retries for connection establishment (default `0`, maintaining existing behavior). - `connectionRetryDelayMs`: Delay between connection retries in milliseconds (default `1000`). ### How was this patch tested? Added `JDBCConnectionRetrySuite` covering: - Default 0-retry behavior on normal connection - Successful retries on `SQLTransientConnectionException` - Successful retries on connection SQLState `"08001"` - Immediate failure without retries on non-transient auth error (`28000`) - Preservation of original `SQLException` upon retry exhaustion Closes #58474. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
