@Soumya Simanta Right now its just a prove of concept. Later I will have a real stream. Its EEG files of brain. Later it can be used for real time analysis of eeg streams.
@Mayur The size is huge yes. SO its better to do in distributed manner and as I said above I want to read as stream because later i will have stream data. This is a prove a concept. Regards, Laeeq On Saturday, May 17, 2014 7:03 PM, Mayur Rustagi <mayur.rust...@gmail.com> wrote: The real question is why are looking to consume file as a Stream 1. Too big to load as RDD 2. Operate in sequential manner. Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi On Sat, May 17, 2014 at 5:12 AM, Soumya Simanta <soumya.sima...@gmail.com> wrote: File is just a steam with a fixed length. Usually streams don't end but in this case it would. > > >On the other hand if you real your file as a steam may not be able to use the >entire data in the file for your analysis. Spark (give enough memory) can >process large amounts of data quickly. > >On May 15, 2014, at 9:52 AM, Laeeq Ahmed <laeeqsp...@yahoo.com> wrote: > > >Hi, >> >>I have data in a file. Can I read it as Stream in spark? I know it seems odd >>to read file as stream but it has practical applications in real life if I >>can read it as stream. It there any other tools which can give this file as >>stream to Spark or I have to make batches manually which is not what I want. >>Its a coloumn of a million values. >> >>Regards, >>Laeeq >> >>
