Re: Support SqlStreaming in spark

Hi all, >From what I read there is an effort here to globally standardize SQL Streaming (Flink people, Google at others are working with SQL standardization body) https://arxiv.org/abs/1905.12133v1

Re: Support SqlStreaming in spark

Hi all, I have rewritten the design doc based on previous discussing. https://docs.google.com/document/d/19degwnIIcuMSELv6BQ_1VQI5AIVcvGeqOm5xE2-aRA0 Would be interested to hear what others think. Regards, Genmao Yu -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: Support SqlStreaming in spark

Hi all, I have rewritten the design doc based on previous discussing. https://docs.google.com/document/d/19degwnIIcuMSELv6BQ_1VQI5AIVcvGeqOm5xE2-aRA0 Would be interested to hear what others think. Regards, Genmao Yu -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

Re: Support SqlStreaming in spark

Hi All, I think there are few more updates are added in the design document compare to last document where few folks has reviewed and provided inputs., requesting all experts to review the design document and help us to baseline the design for the SPIP 'Support SQL streaming' in spark

Re: Support SqlStreaming in spark

Hi, Wenchen Thank you for your recognition of Streaming on sql. I have written the SQLStreaming design document: https://docs.google.com/document/d/19degwnIIcuMSELv6BQ_1VQI5AIVcvGeqOm5xE2-aRA0/edit# Your Questions are answered in here:

Re: Support SqlStreaming in spark

No problem -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Support SqlStreaming in spark

Hi JackyLee, Can you put the answers to these questions in the design doc? e.g. if we don't want to support manipulating a streaming query, then is `SELECT STREAM ...` a blocking action? And how users can create a Spark application with multiple streaming jobs? How users can run Structured

Re: Support SqlStreaming in spark

Hi wenchen I have been working at SQLStreaming for a year, and I have promoted it in company. I have seen the design for Kafka or the Calcite, and I believe my design is better than them. They support pure-SQL not table API for streaming. Users can only use the specified Streaming

Re: Support SqlStreaming in spark

Hi wenchen and Arun Mahadevan Thanks for your reply. SQLStreaming is not just a way to support pure-SQL, but also a way to define table api for Streaming. I have redefined the SQLStreaming to make it support table API. User can use sql or table API to run SQLStreaming. I will

Re: Support SqlStreaming in spark

There has been efforts to come up with a unified syntax for streaming (see [1] [2]), but I guess there will be differences based on the streaming features supported by a system. Agree it needs a detailed design and it can be as close to the Spark batch SQL syntax as possible. Also I am not sure

Re: Support SqlStreaming in spark

It will be great to add pure-SQL support to structured streaming. I think it goes without saying that how important SQL support is, but we should make a completed design first. Looking at the Kafka streaming syntax , it has

Re: Support SqlStreaming in spark

The code of SQLStreaming has been pushed: https://github.com/apache/spark/pull/22575 -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ - To unsubscribe e-mail: dev-unsubscr...@spark.apache.org

Re: Support SqlStreaming in spark

Spark JIRA: https://issues.apache.org/jira/projects/SPARK/issues/SPARK-24630 Benefits: Firstly, users, who are unfamiliar with streaming, can easily use SQL to run StructStreaming especially when migrating offline tasks to real time processing tasks. Secondly, support SQL API in StructStreaming

Re: Support SqlStreaming in spark

Structured Streaming supports standard SQL as the batch queries, so the users can switch their queries between batch and streaming easily. Could you clarify what problems SqlStreaming solves and what are the benefits of the new syntax? Best Regards, Ryan On Thu, Jun 14, 2018 at 7:06 PM, JackyLee

Re: Re: Support SqlStreaming in spark

tlr4/org/ > apache/spark/sql/catalyst/parser/pstl/PstlSqlBase.g4. > > -- > *From:* JackyLee > *Sent:* Thursday, June 14, 2018 7:06:17 PM > *To:* dev@spark.apache.org > *Subject:* Support SqlStreaming in spark > > Hello > > Nowadays, more and more streaming pr

Re:Re: Support SqlStreaming in spark

rg Subject: Support SqlStreaming in spark Hello Nowadays, more and more streaming products begin to support SQL streaming, such as KafaSQL, Flink SQL and Storm SQL. To support SQL Streaming can not only reduce the threshold of streaming, but also make streaming easier to be accepted by everyone.

Support SqlStreaming in spark

Hello Nowadays, more and more streaming products begin to support SQL streaming, such as KafaSQL, Flink SQL and Storm SQL. To support SQL Streaming can not only reduce the threshold of streaming, but also make streaming easier to be accepted by everyone. At present, StructStreaming is