This is cool. Thanks Akhil. ᐧ
On Sun, May 17, 2015 at 11:25 AM, Akhil Das <[email protected]> wrote: > With file timestamp, you can actually see the finding new files logic > from here > <https://github.com/apache/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala#L172> > > Thanks > Best Regards > > On Fri, May 15, 2015 at 2:25 AM, Vadim Bichutskiy < > [email protected]> wrote: > >> How does textFileStream work behind the scenes? How does Spark Streaming >> know what files are new and need to be processed? Is it based on time >> stamp, file name? >> >> Thanks, >> Vadim >> ᐧ >> > >
