[
https://issues.apache.org/jira/browse/SPARK-34205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
L. C. Hsieh updated SPARK-34205:
--------------------------------
Description: Dataset doesn't have pipe API but RDD has it. Although for
normal Dataset, user can convert a Dataset to RDD and call RDD.pipe, for
streaming Dataset it is not possible. So that being said, this is actually a
requirement from Structured Streaming, but we need to add pipe API to Dataset
to enable it in Structured Streaming. (was: Dataset doesn't have pipe API but
RDD has it. Although for normal Dataset, user can convert a Dataset to RDD and
call RDD.pipe, for streaming Dataset it is not possible.)
> Add pipe API to Dataset
> -----------------------
>
> Key: SPARK-34205
> URL: https://issues.apache.org/jira/browse/SPARK-34205
> Project: Spark
> Issue Type: New Feature
> Components: SQL, Structured Streaming
> Affects Versions: 3.2.0
> Reporter: L. C. Hsieh
> Assignee: L. C. Hsieh
> Priority: Major
>
> Dataset doesn't have pipe API but RDD has it. Although for normal Dataset,
> user can convert a Dataset to RDD and call RDD.pipe, for streaming Dataset it
> is not possible. So that being said, this is actually a requirement from
> Structured Streaming, but we need to add pipe API to Dataset to enable it in
> Structured Streaming.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
