The repo you give may solve some of SqlStreaming problems, but not friendly enough, user need to learn this new syntax.
-- Jacky Lee Mail：qcsd2...@163.com At 2018-06-15 11:48:01, "Bowden, Chris" <chris.bow...@microfocus.com> wrote: Not sure if there is a question in here, but if you are hinting that structured streaming should support a sql interface, spark has appropriate extensibility hooks to make it possible. However, the most powerful construct in structured streaming is quite difficult to find a sql equivalent for (e.g., flatMapGroupsWithState). This repo could use some cleanup but is an example of providing a sql interface to a subset of structured streaming's functionality: https://github.com/vertica/pstl/blob/master/pstl/src/main/antlr4/org/apache/spark/sql/catalyst/parser/pstl/PstlSqlBase.g4. From: JackyLee <qcsd2...@163.com> Sent: Thursday, June 14, 2018 7:06:17 PM To:dev@spark.apache.org Subject: Support SqlStreaming in spark Hello Nowadays, more and more streaming products begin to support SQL streaming, such as KafaSQL, Flink SQL and Storm SQL. To support SQL Streaming can not only reduce the threshold of streaming, but also make streaming easier to be accepted by everyone. At present, StructStreaming is relatively mature, and the StructStreaming is based on DataSet API, which make it possibal to provide a SQL portal for structstreaming and run structstreaming in SQL. To support for SQL Streaming, there are two key points: 1, Analysis should be able to parse streaming type SQL. 2, Analyzer should be able to map metadata information to the corresponding Relation. Running StructStreaming in SQL can bring some benefits. 1, Reduce the entry threshold of StructStreaming and attract users more easily. 2, Encapsulate the meta information of source or sink into table, maintain and manage uniformly, and make users more accessible. 3. Metadata permissions management, which is based on hive, can control StructStreaming's overall authority management scheme more closely. We have found some ways to solve this problem. It's a pleasure to discuss it with you. Thanks, Jackey Lee -- Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/ --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
