some Ideas on expressing Spark SQL using JSON

[kant kodali]

Hi All,

I am thinking to express Spark SQL using JSON in the following the way.

For Example:

*Query using Spark DSL*

DS.filter(col("name").equalTo("john"))
        .groupBy(functions.window(df1.col("TIMESTAMP"), "24 hours",
"24 hours"), df1.col("hourlyPay"))
        .agg(sum("hourlyPay").as("total"));


*Query using JSON*



​
​
The Goal is to design a DSL in JSON such that users can and express SPARK
SQL queries in JSON so users can send Spark SQL queries over rest and get
the results out. Now, I am sure there are BI tools and notebooks like
Zeppelin that can accomplish the desired behavior however I believe there
maybe group of users who don't want to use those BI tools or notebooks
instead they want all the communication from front end to back end using
API's.

Also another goal would be the DSL design in JSON should closely mimic the
underlying Spark SQL DSL.

Please feel free to provide some feedback or criticize to whatever extent
you like!

Thanks!