dongjoon-hyun opened a new pull request, #614: URL: https://github.com/apache/spark-connect-swift/pull/614
### What changes were proposed in this pull request? This PR proposes to enable gRPC keepalive (HTTP/2 PING) by default and to support the following connection string parameters with the same names and meanings as the Scala and Python clients (SPARK-58094). | Parameter | Default | Scala/Python default | | --- | --- | --- | | `grpc_keepalive_enabled` | `true` | `true` | | `grpc_keepalive_time_ms` | `300000` (5 minutes) | `60000` | | `grpc_keepalive_timeout_ms` | `20000` | `20000` | | `grpc_keepalive_without_calls` | `true` | `true` | - `SparkConnectClient.init(remote:)` parses them. Like `grpc_max_message_size`, a non-positive or non-integer `grpc_keepalive_time_ms` or `grpc_keepalive_timeout_ms` throws `SparkConnectError.InvalidArgument`. Like `use_ssl` and the Python client, a boolean parameter is `true` only when its value is `true`, ignoring case. - `getGRPCClient()` sets `HTTP2ClientTransport.Config.Connection.keepalive` of the transport. It stays `nil` (no keepalive) with `grpc_keepalive_enabled=false`. The default keepalive time is 5 minutes instead of 60 seconds because of Apache Spark 4.0, 4.1 and 4.2 servers. They use the gRPC Java defaults, `permitKeepAliveTime=5m` and `permitKeepAliveWithoutCalls=false`. On the third early ping since they last sent anything, they send `GOAWAY(ENHANCE_YOUR_CALM, "too_many_pings")` and close the connection with all its in-flight RPCs. The Scala and Python clients recover by doubling their keepalive time (gRFC A8), but `grpc-swift-nio-transport` [2.11.0](https://github.com/grpc/grpc-swift-nio-transport/releases/tag/2.11.0) (2026-10-13) doesn't. With 60 seconds, every attempt of an RPC on which the server stays silent for 3-4 minutes, e.g. `AnalyzePlan` with a long schema inference, would be killed. Apache Spark 4.3 servers allow a ping every 10 seconds even without calls, so their users can set `grpc_keepalive_time_ms=60000` to get the same behavior as the Scala and Python clients. `grpc_keepalive_without_calls=false` is not recommended with `grpc-swift-nio-transport` 2.11.0. It stops the keepalive timer when the last stream of a connection closes and doesn't restart it for new streams. Since the Swift client shares one connection, keepalive stops after the first RPC. ### Why are the changes needed? The Swift client used neither keepalive nor RPC deadlines. When a NAT gateway, a load balancer or a proxy silently drops a connection without TCP RST/FIN, a blocked RPC like `StreamingQuery.awaitTermination()` or a long `DataFrame.collect()` waits forever because nothing detects it. With this PR, such a connection is closed within `grpc_keepalive_time_ms + grpc_keepalive_timeout_ms` (5 minutes and 20 seconds by default). The RPC fails with `UNAVAILABLE`, is retried by `RetryPolicy`, and `ExecutePlan` is resumed by `ReattachExecute`. ### Does this PR introduce _any_ user-facing change? Yes. - The client sends a keepalive PING every 5 minutes by default, and an RPC on a silently dropped connection is retried or reattached instead of waiting forever. `grpc_keepalive_enabled=false` restores the previous behavior. - `grpc_keepalive_*` parameters are honored instead of being ignored with `Unknown parameter: ...`. An invalid value such as `grpc_keepalive_time_ms=abc` throws `SparkConnectError.InvalidArgument`. ### How was this patch tested? Pass the CIs with the newly added test cases in `SparkConnectClientTests`. In addition, I tested manually against `apache/spark:4.2.0` through a TCP relay. The relay can freeze its existing connections silently, reading and dropping bytes in both directions while keeping the sockets open, like a NAT gateway which dropped the mapping. New connections are forwarded as usual. 1. `spark.sql("SELECT reflect('java.lang.Thread', 'sleep', 30000L)").collect()` while the relay froze the connection 3 seconds later | Connection string | Result | | --- | --- | | `grpc_keepalive_enabled=false` (previous behavior) | Blocked until the relay closed the frozen connection 63 seconds later | | `grpc_keepalive_time_ms=10000;grpc_keepalive_timeout_ms=5000` | Sent a PING, closed the connection 5 seconds later with `GOAWAY(keepalive_expired)`, reattached on a new connection, and returned in 30.4 seconds | | The above with `grpc_keepalive_without_calls=false` | No PING was sent and blocked like the first row | 2. Keepalive enforcement of the Apache Spark 4.2.0 server | Case | Result | | --- | --- | | Idle with `grpc_keepalive_time_ms=10000` | The server closed the connection 30 seconds (3 PINGs) after it opened | | `spark.read.json(path).schema` taking 10.2 seconds with the default, with `grpc_keepalive_time_ms=1000` | The server killed each of the first three attempts after 3 seconds, while each abandoned attempt kept running on the server | ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 5.5 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
