haiyangsun-db commented on code in PR #56397:
URL: https://github.com/apache/spark/pull/56397#discussion_r3386330652
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/externalUDF/ExternalUDFExec.scala:
##########
@@ -60,10 +60,12 @@ trait ExternalUDFExec extends UnaryExecNode {
/**
* Creates a [[WorkerSession]] via [[SparkEnv#getExternalUDFDispatcher]].
- * Registers session cancellation on task failure and session termination
- * on task completion. The provided function receives the session
- * and must return the result iterator. The function may use the
- * session but MUST NOT cancel or close it.
+ * Finalizes the session on task completion (which fires on both success and
+ * failure). [[WorkerSession#close]] is the single finalizer: it fetches the
+ * `FinishResponse` if processing completed, or cancels anything still in
+ * flight and waits for the `CancelResponse`. The provided function receives
+ * the session and must return the result iterator. The function CAN but MUST
+ * NOT close the session.
Review Comment:
I will remember to fix this in the next PR (as the CI is all green, don't
like to re-trigger all CI jobs again for changing the comment).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
