beliefer opened a new pull request, #41527: URL: https://github.com/apache/spark/pull/41527
### What changes were proposed in this pull request? `SparkConnectStreamHandler` treats the proto requests from connect client and send the responses back to connect client. `SparkConnectStreamHandler` holds a component `StreamObserver` to send responses. Currently, SparkConnectPlanner also knows `StreamObserver` passed by method parameters. So the behavior introduces some issues. - Visibility `SparkConnectStreamHandler` holds the `StreamObserver` and shouldn't expose it. Now, we expose `StreamObserver` to `SparkConnectPlanner` and the latter is public, so every developers could use it. - Safety Based on the visibility issue, this is usually not a secure strategy. - Code Design Because `SparkConnectStreamHandler` is fully responsible for responding to RPC, which is a division of responsibilities in good programming. The better code design, the easier to extend function. So I think we should keep the `StreamObserver` could be accessed only with `SparkConnectStreamHandler`. This PR wraps the detail of `StreamObserver` into `SparkConnectStreamHandler`, `SparkConnectPlanner` only need call `sendResponse` if the response is ready. This PR want decouple the process handle commands and the other process send responses on server side. Note: As we disscussed on https://github.com/apache/spark/pull/41379, this PR doesn't delay to send any response "right now" and expects that it will be returned. ### Why are the changes needed? Decouple handle command and send response on server side. ### Does this PR introduce _any_ user-facing change? 'No'. Just update the inner implementation. ### How was this patch tested? Exists test cases. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
