Fwd: [Spark Streaming] The FileInputDStream newFilesOnly=false does not work in 1.2 since

[Terry Hole]

Hi,

I am trying to move from 1.1 to 1.2 and found that the newFilesOnly=false
(Intend to include old files) does not work anymore. It works great in 1.1,
this should be introduced by the last change of this class.



Does this flag behavior change or is it a regression?

Issue should be caused by this code:
>From line 157 in FileInputDStream.scala
    val modTimeIgnoreThreshold = math.max(
        initialModTimeIgnoreThreshold,   // initial threshold based on
newFilesOnly setting
        currentTime - durationToRemember.milliseconds  // trailing end of
the remember window
      )


Regards

- Terry