You could build a rest API, but you may have issue if you want to return
back arbitrary binary data. A more complex but robust alternative is to use
some RPC libraries like Akka, Thrift, etc.
TD
On Mon, Feb 23, 2015 at 12:45 AM, Nikhil Bafna
wrote:
>
> Tathagata - Yes, I'm thinking on that line
Tathagata - Yes, I'm thinking on that line.
The problem is how to send to send the query to the backend? Bundle a http
server into a spark streaming job, that will accept the parameters?
--
Nikhil Bafna
On Mon, Feb 23, 2015 at 2:04 PM, Tathagata Das wrote:
> You will have a build a split infra
You will have a build a split infrastructure - a front end that takes the
queries from the UI and sends them to the backend, and the backend (running
the Spark Streaming app) will actually run the queries on table created in
the contexts. The RPCs necessary between the frontend and backend will nee
Yes. As my understanding, it would allow me to write SQLs to query a spark
context. But, the query needs to be specified within a job & deployed.
What I want is to be able to run multiple dynamic queries specified at
runtime from a dashboard.
--
Nikhil Bafna
On Sat, Feb 21, 2015 at 8:37 PM, Te
Have you looked at
http://spark.apache.org/docs/1.2.0/api/scala/index.html#org.apache.spark.sql.SchemaRDD
?
Cheers
On Sat, Feb 21, 2015 at 4:24 AM, Nikhil Bafna
wrote:
>
> Hi.
>
> My use case is building a realtime monitoring system over
> multi-dimensional data.
>
> The way I'm planning to go
Hi.
My use case is building a realtime monitoring system over multi-dimensional
data.
The way I'm planning to go about it is to use Spark Streaming to store
aggregated count over all dimensions in 10 sec interval.
Then, from a dashboard, I would be able to specify a query over some
dimensions, w