juliuszsompolski commented on code in PR #43745:
URL: https://github.com/apache/spark/pull/43745#discussion_r1389176478
##########
connector/connect/client/jvm/src/main/scala/org/apache/spark/sql/SparkSession.scala:
##########
@@ -251,7 +251,7 @@ class SparkSession private[sql] (
.addAllPosArguments(args.map(lit(_).expr).toImmutableArraySeq.asJava)))
val plan = proto.Plan.newBuilder().setCommand(cmd)
// .toBuffer forces that the iterator is consumed and closed
- val responseSeq = client.execute(plan.build()).toBuffer.toSeq
+ val responseSeq = client.execute(plan.build()).toSeq
Review Comment:
Currently the test seems to pass. But I think the commands that had to do
this forced execution may not get coverage...
If it works, it would be good to add a test to that uses one of these
commands that used to do `.toBuffer` to assert that they release execution even
without `toBuffer`... Unfortunately, the `ReattachableExecuteSuite` that has
access to server state does not have access to client `SparkSession` to test
it... So it would have to be tested in connect-client-jvm tests with some
pulling out of the ExecutePlanResponseReattachableIterator and asserting that
it got closed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
