I see. Thanks for the advice!
Jan 20, 2020, 09:47 by hemant2...@gmail.com: > Livy has both statement based(scala) as well as batch processing(code jar > packaged). I think first statement based approach is what you might want to > look at. > Data has to residing in some source. > > Thanks, > Hemant > > On Mon, 20 Jan 2020 at 2:04 PM, <> hamishberri...@tutanota.com> > wrote: > >> Sorry didn't explain well. Livy seems to me similar to job server by >> interacting through restful api interface for job submission. The task I >> want to achieve is more like general restful api interface where user would >> provide some parameters for business operations (a bit like readText or a >> custom date source with embedded http). So when the spark receive those >> requests in e.g custom date source, the spark job can then perform normal >> operations such as aggregation, window functions, and so on. >> >> Thank you for the suggestions and help. >> >> >> >> >> Jan 20, 2020, 00:26 by >> chris.t...@gmail.com>> : >> >>> >>> >>> Maybe something like Livy, otherwise roll your own REST API and have it >>> start a Spark job. >>> >>> On Mon, 20 Jan 2020 at 06:55, <>>> hamishberri...@tutanota.com>>> > wrote: >>> >>>> I am new to Spark. The task I want to accomplish is let client send http >>>> requests, then spark process that request for further operations. However >>>> searching Spark's website docs >>>> >>>> >>>> https://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.package >>>> >>>> >>>> https://spark.apache.org/docs/latest/ >>>> >>>> I do not find any places mentioning about this. Also most of the internet >>>> result are more lated to spark job server. >>>> >>>> Any places I should start if I want to use Spark for such purpose? >>>> >>>> Thanks >>>> >>> >>> >>> -- >>> Chris >>> >> >>
