File is just a steam with a fixed length. Usually streams don't end but in this case it would.
On the other hand if you real your file as a steam may not be able to use the entire data in the file for your analysis. Spark (give enough memory) can process large amounts of data quickly. > On May 15, 2014, at 9:52 AM, Laeeq Ahmed <laeeqsp...@yahoo.com> wrote: > > Hi, > > I have data in a file. Can I read it as Stream in spark? I know it seems odd > to read file as stream but it has practical applications in real life if I > can read it as stream. It there any other tools which can give this file as > stream to Spark or I have to make batches manually which is not what I want. > Its a coloumn of a million values. > > Regards, > Laeeq >
