Jason Teoh created SPARK-56248:
----------------------------------
Summary: Python TransformWithState stateful processor
serialization improvements
Key: SPARK-56248
URL: https://issues.apache.org/jira/browse/SPARK-56248
Project: Spark
Issue Type: Improvement
Components: Structured Streaming
Affects Versions: 4.1.1
Reporter: Jason Teoh
The current Python TWS stateful processor's serialization path is a hot path
executed for every row. Looking at the code, there is room for improvement
around Row construction, closure creation, and attribute lookups.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
