[
https://issues.apache.org/jira/browse/NIFI-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16048867#comment-16048867
]
Koji Kawamura commented on NIFI-3213:
-------------------------------------
[~bende] Sorry for my late response. As you concerned, this JIRA can list
without waiting additional cycle, it doesn't go into that else statement. We
shouldn't use System.nanoTime to compare with file timestamp as System.nanoTime
uses arbitrary origin and differ from one JVM to another.
Even before this JIRA is merged, filesystems those do not provide timestamps in
milliseconds precision have had a problem that ListFile can miss some of the
files those are written with the same timestamp in seconds precision. I created
NIFI-4069 to address your concern and also to work with those filesystems with
less accurate timestamp.
It'd be appreciated if you can take a look on NIFI-4069 and its PR. Let's
discuss further at NIFI-4069.
> ListFile always skips files with the latest timestamp in an iteration even if
> the files have existed a while ago
> ----------------------------------------------------------------------------------------------------------------
>
> Key: NIFI-3213
> URL: https://issues.apache.org/jira/browse/NIFI-3213
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1
> Reporter: Koji Kawamura
> Assignee: Koji Kawamura
> Fix For: 1.2.0
>
>
> NIFI-1484 add few lines of code to avoid files to be emitted if those have
> the latest timestamp within an iteration of listing, because it may still be
> written at the same time.
> While it doesn't affect much if ListFiles processor is scheduled with a short
> period of time, such as few ms, but it does affect negatively if an user
> scheduled it with longer run schedule such as "1 day" or with cron scheduler.
> For example, user would expect to process list of files per daily basis. Even
> if a file is saved few hours ago, the processor will skip this, because the
> file has the latest timestamp within the iteration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
