Spark SQL and Streaming Results

2015-06-05 Thread Pietro Gentile
Hi all, what is the best way to perform Spark SQL queries and obtain the result tuplas in a stremaing way. In particullar, I want to aggregate data and obtain the first and incomplete results in a fast way. But it should be updated until the aggregation be completed. Best Regards.

Re: Spark SQL and Streaming Results

2015-06-05 Thread Todd Nist
There use to be a project, StreamSQL ( https://github.com/thunderain-project/StreamSQL), but it appears a bit dated and I do not see it in the Spark repo, but may have missed it. @TD Is this project still active? I'm not sure what the status is but it may provide some insights on how to achieve

Re: Spark SQL and Streaming Results

2015-06-05 Thread Tathagata Das
You could take at RDD *async operations, their source code. May be that can help if getting some early results. TD On Fri, Jun 5, 2015 at 8:41 AM, Pietro Gentile pietro.gentile89.develo...@gmail.com wrote: Hi all, what is the best way to perform Spark SQL queries and obtain the result

Re: Spark SQL and Streaming Results

2015-06-05 Thread Tathagata Das
I am not sure. Saisai may be able to say more about it. TD On Fri, Jun 5, 2015 at 5:35 PM, Todd Nist tsind...@gmail.com wrote: There use to be a project, StreamSQL ( https://github.com/thunderain-project/StreamSQL), but it appears a bit dated and I do not see it in the Spark repo, but may