With Structured Streaming ,Spark would provide apis over spark sql engine. Its like once you have the structured stream and dataframe created out of this , you can do ad-hoc querying on the DF , which means you are actually querying the stram without having to store or transform. I have not used it yet but seems it will be like start streaming data from source as son as you define it.
Thanks Deepak On Fri, May 6, 2016 at 1:37 PM, madhu phatak <phatak....@gmail.com> wrote: > Hi, > As I was playing with new structured streaming API, I noticed that spark > starts processing as and when the data appears. It's no more seems like > micro batch processing. Is spark structured streaming will be an event > based processing? > > -- > Regards, > Madhukara Phatak > http://datamantra.io/ > -- Thanks Deepak www.bigdatabig.com www.keosha.net
