[
https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tathagata Das updated SPARK-18791:
----------------------------------
Description: Stream stream join is a much requested, but missing feature in
Structured Streaming. While the join API exists in Datasets and DataFrames, it
throws UnsupportedOperationException when applied between two streaming
Datasets/DataFrames. To support this, we have to maintain the same semantics as
other Structured Streaming operations - the output of the operation after
consuming a stream of data until position/offset X, should be the same as a
batch join on all the data (was: Just a placeholder for now. Please comment
with your requirements.)
> Stream-Stream Joins
> -------------------
>
> Key: SPARK-18791
> URL: https://issues.apache.org/jira/browse/SPARK-18791
> Project: Spark
> Issue Type: New Feature
> Components: Structured Streaming
> Reporter: Michael Armbrust
>
> Stream stream join is a much requested, but missing feature in Structured
> Streaming. While the join API exists in Datasets and DataFrames, it throws
> UnsupportedOperationException when applied between two streaming
> Datasets/DataFrames. To support this, we have to maintain the same semantics
> as other Structured Streaming operations - the output of the operation after
> consuming a stream of data until position/offset X, should be the same as a
> batch join on all the data
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
