[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 I hive send an email to Ryan Blue. > > > Can you send a mail to Ryan blue for adding this SPIP topic in tomorrow meeting. Meeting will be conducted tomorrow 05:00 pm PST. If you confirm

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22575 > > Can you send a mail to Ryan blue for adding this SPIP topic in tomorrow meeting. Meeting will be conducted tomorrow 05:00 pm PST. If you confirm then we can also attend the meeting. >

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 > Can you send a mail to Ryan blue for adding this SPIP topic in tomorrow meeting. Meeting will be conducted tomorrow 05:00 pm PST. If you confirm then we can also attend the meeting. I

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22575 Can you send a mail to Ryan blue for adding this SPIP topic in tomorrow meeting. Meeting will be conducted tomorrow 05:00 pm PST. If you confirm then we can also attend the meeting.

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 > ![image](https://user-images.githubusercontent.com/12999161/49129177-ab056680-f2f4-11e8-8f71-4695ebc045c1.png) I have removed the 'stream' keyword. > There is a DatasourceV2 community

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22575 cc @koeninger --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22575 ![image](https://user-images.githubusercontent.com/12999161/49129177-ab056680-f2f4-11e8-8f71-4695ebc045c1.png) There is a DatasourceV2 community synch meetup tomorrow which is

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 @sujithjay Please refer [SPARK-24630](https://issues.apache.org/jira/browse/SPARK-24630) for more details. --- - To

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-25 Thread sujith71955
Github user sujith71955 commented on the issue: https://github.com/apache/spark/pull/22575 @stczwd Can you provide a detail design document for this PR, by mentioning the cenarios is been handled and constraints if any. this wll give a complete pitcture about this PR. Thanks ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-20 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 @cloud-fan @zsxwing @tdas @xuanyuanking This patch has been submitted for a long time. Do you have any questions? Can this patch be merged in? ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-12 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 > Currently Dataframe API support "writeStream.start()" api to run streaming in background, so that query can be executed on that sink, also multiple stream to stream processing can happen in single

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-12 Thread gvramana
Github user gvramana commented on the issue: https://github.com/apache/spark/pull/22575 > > How should we do if we wanna join two kafka stream and sink the result to another stream? > > `insert into kafka_sql_out select stream t1.value from (select cast(value as string),

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-08 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 @tdas @zsxwing @cloud-fan Hi, any other questions block this patch for merge in? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-02 Thread shijinkui
Github user shijinkui commented on the issue: https://github.com/apache/spark/pull/22575 @cloud-fan Hi, Wenchen. Is it ready for merge in? This PR is very useful and is what I want to develop and need. Once Spark support StreamSQL, it will be easier for developping streaming job.

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-31 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/22575 Nice! I am looking forward to it. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-25 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 @WangTaoTheTonic @cloud-fan @xuanyuanking I have removed the stream keyword. Table API is supoorted in SQLStreaming now. ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22575 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-22 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 > ql and normal sql? how could users define watermark with SQL? Yes, the 'stream' keyword is the only difference from normal sql. We can use configuration to define watermark. ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 do we have a full story about stream sql? is the `STREAM` keyword the only difference between stream sql and normal sql? also cc @tdas @zsxwing ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 do we have a full story about stream sql? is the `STREAM` keyword the only difference between stream sql and normal sql? how could users define watermark with SQL? also cc @tdas @zsxwing

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 do we have a full story about stream sql? is the `STREAM` keyword the only difference between stream sql and normal sql? how could users define watermark with SQL? also cc @tdas @zsxwing

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 do we have a full story about stream sql? is the `STREAM` keyword the only difference between stream sql and normal sql? how could users define watermark with SQL? also cc @tdas @zsxwing

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 do we have a full story about stream sql? is the `STREAM` keyword the only difference between stream sql and normal sql? how could users define watermark with SQL? also cc @tdas @zsxwing

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22575 Do we have a full story about streaming SQL? is the `STREAM` keyword the only difference between stream sql and normal sql? also cc @tdas @zsxwing ---

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-14 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 cc @xuanyuanking --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-12 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 > How should we do if we wanna join two kafka stream and sink the result to another stream? `insert into kafka_sql_out select stream t1.value from (select cast(value as string), timestamp as

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-11 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/22575 How should we do if we wanna join two kafka stream and sink the result to another stream? --- - To unsubscribe, e-mail:

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-10-11 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 @WangTaoTheTonic Adding 'stream' keyword has two purposes: - **Mark the entire sql query as a stream query and generate the SQLStreaming plan tree.** - **Mark the table type as