grundprinzip commented on issue #126: URL: https://github.com/apache/spark-connect-go/issues/126#issuecomment-3197391044
The server-side query will time-out by default after 5 minutes without the client connecting back. In addition with Spark 4 there are now a bunch more ways to cancel queries: 1. you can use tags to tag a session and then interrupt by tag see https://spark.apache.org/docs/latest/api/python/reference/pyspark.sql/api/pyspark.sql.SparkSession.interruptTag.html however, this isn't yet fully implemented as the public APIs in the Go DataFrame and session. 2. Spark 4 supports as well cancelling by operation ID - https://github.com/apache/spark/blob/master/sql/connect/common/src/main/protobuf/spark/connect/base.proto#L775-L786 however, as well this isn't implemented in the client yet. 3. The client should properly deal with client side cancelling of the request and cancel the query appropriately. Generally, this seems like some work that would be great for someone to take and I'm happy to advise and review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
