[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16641327#comment-16641327
]
Jungtaek Lim commented on SPARK-24630:
--------------------------------------
[~Jackey Lee]
For DDL it would be better to participate discussion on Spark dev. mailing list
which Ryan Blue is driving.
For 'stream' keyword, I'm not sure how using the keyword 'stream' makes huge
difference. It can be a kind of marker to verify at least one of sources is
stream as well as sink is stream, but while I guess this proposal is inspired
by Streaming SQL in Flink or Calcite, even Flink they no longer using keyword
'stream'. "STREAM" is not available in BNF-grammar in here:
[https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql.html]
Spark is on the same page, even without stream keyword, once Spark determines
whether one of source(s) and sink are both stream, query will work as
structured streaming.
> SPIP: Support SQLStreaming in Spark
> -----------------------------------
>
> Key: SPARK-24630
> URL: https://issues.apache.org/jira/browse/SPARK-24630
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.2.0, 2.2.1
> Reporter: Jackey Lee
> Priority: Minor
> Labels: SQLStreaming
> Attachments: SQLStreaming SPIP.pdf
>
>
> At present, KafkaSQL, Flink SQL(which is actually based on Calcite),
> SQLStream, StormSQL all provide a stream type SQL interface, with which users
> with little knowledge about streaming, can easily develop a flow system
> processing model. In Spark, we can also support SQL API based on
> StructStreamig.
> To support for SQL Streaming, there are two key points:
> 1, Analysis should be able to parse streaming type SQL.
> 2, Analyzer should be able to map metadata information to the corresponding
> Relation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
