It will be great to add pure-SQL support to structured streaming. I think it goes without saying that how important SQL support is, but we should make a completed design first.
Looking at the Kafka streaming syntax <https://www.confluent.io/blog/ksql-streaming-sql-for-apache-kafka/>, it has CREATE STREAM, it has WINDOW TUMBLING. Shall we check other streaming systems with SQL support, and justify places where we are going to differ? We should also take into account the full lifecycle: 1. how to restart a streaming query from checkpoint? 2. how to stop a streaming query? 3. how to check status/progress of a streaming query? 4. ... Basically, we should check what functions the DataStreamReader/Writer API support, and see if we can support it with SQL as well. Thanks for your proposal! Wenchen On Mon, Oct 22, 2018 at 11:15 AM JackyLee <qcsd2...@163.com> wrote: > The code of SQLStreaming has been pushed: > > https://github.com/apache/spark/pull/22575 > > > > -- > Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: dev-unsubscr...@spark.apache.org > >
