dillitz opened a new pull request, #45842: URL: https://github.com/apache/spark/pull/45842
### What changes were proposed in this pull request? Follow up to https://github.com/apache/spark/pull/40447. Allows to configure the currently hardcoded max message of 128MB on the client side for both the Scala and Python clients. Adds the option to the Scala client and improves the way we handle `channelOptions` in Python's `ChannelBuiler`. ### Why are the changes needed? Usability: Users are hitting the 128MB limit for very large (single) rows. This gives the option to increase the limit. ### Does this PR introduce _any_ user-facing change? Scala: Adds option to set `grpcMaxMessageSize` to `SparkConnectClient.Builder` Python: No. ### How was this patch tested? Tests added. ### Was this patch authored or co-authored using generative AI tooling? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
