Yes doesn't this need access to the actually configured conf object from the context? This is just making a new conf object. On Apr 9, 2015 8:08 AM, "emres" <[email protected]> wrote:
> Github user emres commented on a diff in the pull request: > > https://github.com/apache/spark/pull/5438#discussion_r28055824 > > --- Diff: > streaming/src/main/scala/org/apache/spark/streaming/dstream/FileInputDStream.scala > --- > @@ -331,19 +331,22 @@ private[streaming] > object FileInputDStream { > > /** > - * Minimum duration of remembering the information of selected > files. Files with mod times > - * older than this "window" of remembering will be ignored. So if > new files are visible > - * within this window, then the file will get selected in the next > batch. > + * Minimum duration of remembering the information of selected > files. Defaults to 1 minute. > + * > + * Files with mod times older than this "window" of remembering > will be ignored. So if new > + * files are visible within this window, then the file will get > selected in the next batch. > */ > - private val MIN_REMEMBER_DURATION = Minutes(1) > + private val minRememberDurationMin = Minutes(new SparkConf() > --- End diff -- > > @srowen I'm a little confused. Do you mean something like: "inside > ``private[streaming] class FileInputDStream[K, V, F <: > NewInputFormat[K,V]]`` class create a ``private val conf = new > SparkConf()`` and then use that ``conf`` inside ``private[streaming] object > FileInputDStream``"? > > Or something totally different? > > > --- > If your project is set up for it, you can reply to this email and have your > reply appear on GitHub as well. If your project does not have this feature > enabled and wishes so, or if the feature is enabled but not working, please > contact infrastructure at [email protected] or file a JIRA ticket > with INFRA. > --- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
