Hey guys, We have been trying out Fink SQL to implement use-cases like "compute the last X minutes of data" and we'd like to return the computed data programatically from Scala/Java, is there a way to write a program to run SQL queries over Flink in a reactive manner?
We used for now for experimenting purposes the sql-client utility and out approach was the following: 1. Create the Source table, backed by Kafka. 2. Create the Sink table, also backed by Kafka. 3. Define and run the INSERT query into the sink. We used here hopping windows, count and group by operators. 4. Define and run the SELECT query to display the data. Here the query uses sub-selects, to retrieve only results for the last window. What we would need now is: 1. Define the DDL and DML queries in steps 1-3 into a file and use the sql-client to send this to the flink cluster for running.What I got from https://issues.apache.org/jira/browse/FLINK-12828 is that the sql-client cannot be used for now for this purpose. What would be the alternative here? 2. Write a Scala/Java program that would run the query in step 4 and return the result to its users, e.g. via a REST interface in a continuous manner, e.g. by exposing a reactive interface. The JDBC client exposed by Flink is not reactive, as the JDBC spec is not reactive. Is there an alternative API we could use? 3. We would like to get incomplete results for the current minute, e.g. if we run the query at time 14:34:23, we should get the data for the window 14:30:00 - 14:35:00, not for the window 14:29:00 - 14:34:00. What would be the best way to achieve these for now? Looking forward for you answers. Thanks, Cristi -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
