Jonathan Albrecht created SPARK-51092:
-----------------------------------------
Summary: FlatMapGroupsWithState: Change the dataType of the
timestampTimeoutAttribute from IntegerType to LongType
Key: SPARK-51092
URL: https://issues.apache.org/jira/browse/SPARK-51092
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.5.4, 4.0.0, 3.5.5, 4.1.0
Reporter: Jonathan Albrecht
The {{dataType}} of the {{timestampTimeoutAttribute}} is IntegerType in
FlatMapGroupsWithStateExecHelper.scala for the v1 state manager but it is
serialized and stored in memory as a Long everywhere.
This causes the {{timestampTimeout}} value to be corrupted on big endian
platforms. This is seen in unit tests relating to the v1 state manager when run
on big endian platforms.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
