[
https://issues.apache.org/jira/browse/NIFI-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15753933#comment-15753933
]
ASF GitHub Bot commented on NIFI-3213:
--------------------------------------
Github user ijokarumawak commented on the issue:
https://github.com/apache/nifi/pull/1335
I believe the old behavior that always postpone to emit files with the
latest timestamp is counter intuitive and can be problematic with some use
cases if user would like to schedule it with longer run schedule. Please
correct me if I'm missing any important purpose for this behavior.
I had to fix many unit test cases because those are written with an
assumption that the latest file should be skipped. Again I believe those test
cases became more natural and understandable by this PR, but I may be missing
something.
Thanks for reviewing in advance!
> ListFile always skips files with the latest timestamp in an iteration even if
> the files have existed a while ago
> ----------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-3213
> URL: https://issues.apache.org/jira/browse/NIFI-3213
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
>
> NIFI-1484 add few lines of code to avoid files to be emitted if those have
> the latest timestamp within an iteration of listing, because it may still be
> written at the same time.
> While it doesn't affect much if ListFiles processor is scheduled with a short
> period of time, such as few ms, but it does affect negatively if an user
> scheduled it with longer run schedule such as "1 day" or with cron scheduler.
> For example, user would expect to process list of files per daily basis. Even
> if a file is saved few hours ago, the processor will skip this, because the
> file has the latest timestamp within the iteration.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
