I am not sure if spark natively support this functionality. Custom poller class can query HTTP resources as per configured interval and dump it on HDFS / other stores in csv or json format. Using lambda arch (aws) or invoking sc context you can use these values for further processing
On Thursday 10 December 2015, Sourav Mazumder <sourav.mazumde...@gmail.com> wrote: > What I want is to be able to connect to a source system which exposes data > stream over http. > > Essentially a Reciever in onStart() should starting a threads which will > access a http uri to get the data say after every 10 seconds. There can be > multiple such Recievers each polling different http uris. An example can be > accessing a http uri for inventory update and sales update in parallel. > > Regards, > Sourav > > On Wed, Dec 9, 2015 at 7:53 PM, Vijay Gharge <vijay.gha...@gmail.com > <javascript:_e(%7B%7D,'cvml','vijay.gha...@gmail.com');>> wrote: > >> Not very clear. Can you elaborate your use case ? >> >> On Thursday 10 December 2015, Sourav Mazumder < >> sourav.mazumde...@gmail.com >> <javascript:_e(%7B%7D,'cvml','sourav.mazumde...@gmail.com');>> wrote: >> >>> Hi All, >>> >>> Currently is there a way using which one can connect to a http server to >>> get data as a dstream at a given frequency ? >>> >>> Or one has to write own utility for the same ? >>> >>> Regards, >>> Sourav >>> >> >> >> -- >> Regards, >> Vijay Gharge >> >> >> >> > -- Regards, Vijay Gharge
