NagaSurya11 opened a new issue, #4851: URL: https://github.com/apache/arrow-adbc/issues/4851
### What happened? The Spark driver defines `adbc.spark.connect_timeout_ms` (`SparkParameters.ConnectTimeoutMilliseconds`, added with the timeout options in #2312/#2313). It is only honored by the HTTP connection: - `SparkHttpConnection` parses it and sets `ConnectTimeoutMilliseconds` ([SparkHttpConnection.cs](https://github.com/apache/arrow-adbc/blob/3df4635caeeed89fa9c71d136278a0e540316a84/csharp/src/Drivers/Apache/Spark/SparkHttpConnection.cs)). - `SparkStandardConnection` (binary Thrift, `SparkServerType.Standard`) never reads the option. Its `ValidateOptions()` only handles the data type conversion and TLS options ([SparkStandardConnection.cs](https://github.com/apache/arrow-adbc/blob/3df4635caeeed89fa9c71d136278a0e540316a84/csharp/src/Drivers/Apache/Spark/SparkStandardConnection.cs)). `HiveServer2Connection.OpenAsync` uses `ConnectTimeoutMilliseconds` as the cancellation timeout for creating the transport/protocol and for `OpenSession` ([HiveServer2Connection.cs](https://github.com/apache/arrow-adbc/blob/3df4635caeeed89fa9c71d136278a0e540316a84/csharp/src/Drivers/Apache/Hive2/HiveServer2Connection.cs)). Because the standard connection never overrides the property, it stays at the 30000 ms default no matter what the user sets. There is no error either: the option is silently ignored. Impact: servers that do real work while a session is opening, for example Apache Kyuubi launching a Spark engine on a cold start, can take longer than 30 s and the connection is cancelled. What I expected: `adbc.spark.connect_timeout_ms` behaves the same on both connection types (including `0` = no limit, and an `ArgumentOutOfRangeException` for invalid values). ### Stack Trace _No response_ ### How can we reproduce the bug? 1. Connect with the Spark driver over binary Thrift (not HTTP) to a server whose `OpenSession` takes longer than 30 s (e.g. Kyuubi with no running engine), with `adbc.spark.connect_timeout_ms=600000`. 2. Expected: the session open may take up to 10 minutes. Per the code above, the 30 s default applies instead, because `SparkStandardConnection` never applies the option. 3. The same option over the HTTP transport is honored. Fix: parse the option in `SparkStandardConnection.ValidateOptions()` the same way `SparkHttpConnection` does. A patch is ready (12 lines, builds with `dotnet build`) and I can open a PR. ### Environment/Setup C# Apache driver (csharp/src/Drivers/Apache/Spark) at apache/arrow-adbc main 3df4635ca. Seen through dbt-oss 2.0.5 (dbt Labs fork of the driver) against Apache Kyuubi, Linux. Source-checked against current apache/arrow-adbc main as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
