[
https://issues.apache.org/jira/browse/SPARK-55147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim reassigned SPARK-55147:
------------------------------------
Assignee: Nicholas Chew
> Improve the retrieval operation in new stream-stream stateful operator to
> scope the timestamp range for time-interval join
> --------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-55147
> URL: https://issues.apache.org/jira/browse/SPARK-55147
> Project: Spark
> Issue Type: Task
> Components: Structured Streaming
> Affects Versions: 4.2.0
> Reporter: Jungtaek Lim
> Assignee: Nicholas Chew
> Priority: Major
> Labels: pull-request-available
>
> With SPARK-55144, the state format of the primary store would support for
> "prefix scan + range scan on event time". That allows us to perform the
> retrieval operation of time interval join more efficiently, via only scanning
> through the timestamp range which is constructed with "time interval
> condition".
> This ticket tracks the effort of modifying the state store API and state
> manager, and modifying stream-stream join operator to integrate the change.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
