Because sparks dsl partially supports compile time type safety. E.g. the
compiler will notify you that a sql function was misspelled when using the
dsl opposed to the plain sql string which is only parsed at runtime.
Sathish Kumaran Vairavelu <vsathishkuma...@gmail.com> schrieb am Di. 25.
Juli 2017 um 23:42:
> Just a thought. SQL itself is a DSL. Why DSL on top of another DSL?
> On Tue, Jul 25, 2017 at 4:47 AM kant kodali <kanth...@gmail.com> wrote:
>
>> Hi All,
>>
>> I am thinking to express Spark SQL using JSON in the following the way.
>>
>> For Example:
>>
>> *Query using Spark DSL*
>>
>> DS.filter(col("name").equalTo("john"))
>> .groupBy(functions.window(df1.col("TIMESTAMP"), "24 hours", "24
>> hours"), df1.col("hourlyPay"))
>> .agg(sum("hourlyPay").as("total"));
>>
>>
>> *Query using JSON*
>>
>>
>>
>>
>>
>> The Goal is to design a DSL in JSON such that users can and express SPARK
>> SQL queries in JSON so users can send Spark SQL queries over rest and get
>> the results out. Now, I am sure there are BI tools and notebooks like
>> Zeppelin that can accomplish the desired behavior however I believe there
>> maybe group of users who don't want to use those BI tools or notebooks
>> instead they want all the communication from front end to back end using
>> API's.
>>
>> Also another goal would be the DSL design in JSON should closely mimic
>> the underlying Spark SQL DSL.
>>
>> Please feel free to provide some feedback or criticize to whatever extent
>> you like!
>>
>> Thanks!
>>
>>
>>
