Jungtaek Lim created SPARK-61986:
------------------------------------

             Summary: NullPointerException for a NULL window join key in 
stream-stream join state format V4
                 Key: SPARK-61986
                 URL: https://issues.apache.org/jira/browse/SPARK-61986
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 5.0.0
            Reporter: Jungtaek Lim


With {{spark.sql.streaming.join.stateFormatVersion=4}} and the watermark on an
equality key, a row whose key is a NULL window fails the query with an NPE in
{{SymmetricHashJoinStateManagerV4.append}} ({{getStruct(idx, 2).getLong(1)}}). 
A NULL timestamp
key is read as time 0 and evicted at once, unlike V2 and V3. Fix: a NULL key 
has no event
time and gets no index entry.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to