jingz-db commented on code in PR #49156:
URL: https://github.com/apache/spark/pull/49156#discussion_r1887720909
##########
python/pyspark/sql/streaming/stateful_processor_util.py:
##########
@@ -25,3 +25,4 @@ class TransformWithStateInPandasFuncMode(Enum):
PROCESS_DATA = 1
PROCESS_TIMER = 2
COMPLETE = 3
+ PRE_INIT = 4
Review Comment:
This is a stateful processor handle state introduced only for internal use.
It will be passed into our UDF `transformWithStateUDF` in `group_ops.py` as an
internal parameter and we will use this parameter to decide whether we are
going to process data, timer, or if we are at pre-init stage. The public
exposed StatefulProcessorHandleState will only be set on the JVM.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
