[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16561798#comment-16561798
]
Jackey Lee edited comment on SPARK-24630 at 7/30/18 11:38 AM:
--------------------------------------------------------------
For Stream Table DDL，we have a better way to deal with(such as the following
example).
Firstly, we used a more compatible way to handle Stream Table, user no need to
learn other more syntax;
Secondly, the Stream Table, kafka_sql_test, can be used to not only Source but
also Sink. Such as, user A can write Stream Data into kafka_sql_test, and user
B can read from kafka_sql_test. This is a closed loop in data processing, and
users can run the stream processing of the entire business smoothly through
SQLStreaming.
Thirdly, the Stream Table created by this sql just seems like Hive Table. User
even can run batch queries on this Stream Table.
Finally, using this way to deal with STREAM Table, fewer code will be added or
changed in spark source.
{code:sql}
create table kafka_sql_test using kafka
options(
table.isstreaming = 'true',
subscribe = 'topic',
kafka.bootstrap.servers = 'localhost:9092')
{code}
was (Author: jackey lee):
For Stream Table DDL，we have a better way to deal with(such as the following
example).
Firstly, we used a more compatible way to handle Stream Table, user no need to
learn other more syntax;
Secondly, the Stream Table, kafka_sql_test, can be used to not only Source but
also Sink. Such as, user A can write Stream Data into kafka_sql_test, and user
B can read from kafka_sql_test. This is a closed loop in data processing, and
users can run the stream processing of the entire business smoothly through
SQLStreaming.
Finally, using this way to deal with STREAM Table, fewer code will be added or
changed in spark source.
{code:sql}
create table kafka_sql_test using kafka
options(
table.isstreaming = 'true',
subscribe = 'topic',
kafka.bootstrap.servers = 'localhost:9092')
{code}
> SPIP: Support SQLStreaming in Spark
> -----------------------------------
>
> Key: SPARK-24630
> URL: https://issues.apache.org/jira/browse/SPARK-24630
> Project: Spark
> Issue Type: Improvement
> Components: Structured Streaming
> Affects Versions: 2.2.0, 2.2.1
> Reporter: Jackey Lee
> Priority: Minor
> Labels: SQLStreaming
> Attachments: SQLStreaming SPIP.pdf
>
>
> At present, KafkaSQL, Flink SQL(which is actually based on Calcite),
> SQLStream, StormSQL all provide a stream type SQL interface, with which users
> with little knowledge about streaming, can easily develop a flow system
> processing model. In Spark, we can also support SQL API based on
> StructStreamig.
> To support for SQL Streaming, there are two key points:
> 1, Analysis should be able to parse streaming type SQL.
> 2, Analyzer should be able to map metadata information to the corresponding
> Relation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
