GitHub user stczwd opened a pull request:
https://github.com/apache/spark/pull/22575
[SPARK-24630][SS][WIP] Support SQLStreaming in Spark
## What changes were proposed in this pull request?
This patch propose new support of SQLStreaming in Spark, Please refer
[SPARK-24630](https://issues.apache.org/jira/browse/SPARK-24630) for more
details.
This patch supports:
1. Support create stream table, which can be used as Source and Sink in
SQLStreaming;
`create table kafka_sql_test using kafka
options(
isStreaming = 'true',
subscribe = 'topic',
kafka.bootstrap.servers = 'localhost:9092')`
2. Add keyword 'STREAM' in sql to support SQLStreaming queries;
`select stream * from kafka_sql_test`
3. As for those complex queries, they all can be supported as long as SQL
and StructStreaming support.
## How was this patch tested?
Some UTs are added to verify sqlstreaming.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/stczwd/spark sqlstreaming
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22575.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22575
----
commit af26ea77157a7ff4e0a2c5eecec64c57f73c425d
Author: Jackey Lee <qcsd2011@...>
Date: 2018-09-28T01:04:17Z
Support SQLStreaming in Spark: Add keyword 'STREAM'; Support create stream
table
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
