RE: Creating a front-end for output from Spark/PySpark

2014-11-25 Thread Mohammed Guller
Two options that I can think of: 1) Use the Spark SQL Thrift/JDBC server. 2) Develop a web app using some framework such as Play and expose a set of REST APIs for sending queries. Inside your web app backend, you initialize the Spark SQL context only once when your app initializes.

Re: Creating a front-end for output from Spark/PySpark

2014-11-23 Thread Alex Kamil
Alaa, one option is to use Spark as a cache, importing subset of data from hbase/phoenix that fits in memory, and using jdbcrdd to get more data on cache miss. The front end can be created with pyspark and flusk, either as rest api translating json requests to sparkSQL dialect, or simply