Jungtaek Lim created SPARK-61986:
------------------------------------
Summary: NullPointerException for a NULL window join key in
stream-stream join state format V4
Key: SPARK-61986
URL: https://issues.apache.org/jira/browse/SPARK-61986
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 5.0.0
Reporter: Jungtaek Lim
With {{spark.sql.streaming.join.stateFormatVersion=4}} and the watermark on an
equality key, a row whose key is a NULL window fails the query with an NPE in
{{SymmetricHashJoinStateManagerV4.append}} ({{getStruct(idx, 2).getLong(1)}}).
A NULL timestamp
key is read as time 0 and evicted at once, unlike V2 and V3. Fix: a NULL key
has no event
time and gets no index entry.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]