Jungtaek Lim created SPARK-61348:
------------------------------------

             Summary: Reject numeric state bounds in stream-stream outer joins
                 Key: SPARK-61348
                 URL: https://issues.apache.org/jira/browse/SPARK-61348
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 5.0.0
            Reporter: Jungtaek Lim


The analyzer accepts a stream-stream outer join whose only bound is on a 
numeric state column, for example {{n > CAST(rightTime AS BIGINT)}}. Runtime 
range eviction only handles timestamp state columns, so the join can never 
finalize its unmatched rows. It keeps state indefinitely and never emits them.

The analyzer should require a timestamp state column when it validates a range 
bound, and reject such outer joins.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to