HeartSaVioR commented on code in PR #48373:
URL: https://github.com/apache/spark/pull/48373#discussion_r1820001517
##########
python/pyspark/sql/pandas/group_ops.py:
##########
@@ -503,7 +503,11 @@ def transformWithStateUDF(
statefulProcessorApiClient.set_implicit_key(key)
result = statefulProcessor.handleInputRows(key, inputRows)
- return result
+ try:
+ yield result
+ finally:
+ statefulProcessor.close()
Review Comment:
I don't feel like current interface would give you such information - we'll
probably need to have another control message to send the signal from JVM to
Python (UDF). I agree this may take time, but probably need to mark it as a
blocker so that we address before the release.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
