Jason Teoh created SPARK-53822:
----------------------------------
Summary: Add python transform_with_state test case for output
schemas containing composite StructType/nested StructType like ArrayType/MapType
Key: SPARK-53822
URL: https://issues.apache.org/jira/browse/SPARK-53822
Project: Spark
Issue Type: Test
Components: Structured Streaming
Affects Versions: 4.0.1
Reporter: Jason Teoh
Existing test case today does cover composite schemas, but only within the
state API. The output type of that test case is still strings (e.g., dicts are
json dumped into a string).
This ticket is to add a test case where the output of the transform_with_state
operation itself contains nested types.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
